2024-04-25T00:03:46.332 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:03:46.332 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244 --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/orig.config.yaml 2024-04-25T00:03:46.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:03:46.552 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672244' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7ppejsaNvvQ6Yri1OK3XJSj0lEoQFlRymh6WmFInINLHJ6R89W9yPc1aSvqj67uw+BDCVxEzYAtILVdojzRFo= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFOhJKppAYFUwT+nOYadluy8FSlaItbVO3JkQdXpINimrr3ZkuhItpD/XsGh92mHP7wCu862z50Or1Vj4NUesSc= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZPSrqqGE5gE2ExaO1LMZoiyxwGbX4wDfzy2EylBoDpC0r9IAzQqt/W06X88+EpOoFQ1HZZRwNiPxos8584isg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:03:46.552 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:03:46.553 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:03:46.553 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:03:46.554 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:03:46.580 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:03:46.581 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:03:46.581 INFO:teuthology.packaging:ref: None 2024-04-25T00:03:46.581 INFO:teuthology.packaging:tag: None 2024-04-25T00:03:46.581 INFO:teuthology.packaging:branch: squid 2024-04-25T00:03:46.581 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:03:46.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T00:03:46.897 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T00:03:46.898 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:03:46.904 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:03:46.904 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:03:46.929 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:03:46.952 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:03:46.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:03:46.998 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:58:52.701206', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE7ppejsaNvvQ6Yri1OK3XJSj0lEoQFlRymh6WmFInINLHJ6R89W9yPc1aSvqj67uw+BDCVxEzYAtILVdojzRFo='} 2024-04-25T00:03:47.020 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:58:52.702718', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFOhJKppAYFUwT+nOYadluy8FSlaItbVO3JkQdXpINimrr3ZkuhItpD/XsGh92mHP7wCu862z50Or1Vj4NUesSc='} 2024-04-25T00:03:47.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:58:52.704006', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKZPSrqqGE5gE2ExaO1LMZoiyxwGbX4wDfzy2EylBoDpC0r9IAzQqt/W06X88+EpOoFQ1HZZRwNiPxos8584isg='} 2024-04-25T00:03:47.045 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:03:47.059 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T00:03:47.059 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T00:03:47.059 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T00:03:47.059 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:03:47.221 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4dff663160>, signals=[15]) 2024-04-25T00:03:47.221 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:03:47.228 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:03:47.228 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:03:47.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:47.309 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-25T00:03:47.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:47.383 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-25T00:03:47.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:47.462 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:03:47.468 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-25T00:03:47.487 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-25T00:03:47.487 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-04-25T00:03:47.545 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:03:47.546 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:03:47.547 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-25T00:03:47.576 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-25T00:03:47.594 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-25T00:03:47.594 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:03:47.651 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:03:47.652 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:03:47.652 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:03:47.652 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:03:47.652 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-25T00:03:47.676 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-25T00:03:47.694 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-25T00:03:47.694 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:03:47.752 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:03:47.753 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:03:47.753 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-25T00:03:47.775 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:03:47.850 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:03:47.902 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:03:47.903 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:03:47.906 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:03:47.909 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:03:47.926 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:03:47.950 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:03:47.950 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:03:47.965 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:03:47.968 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:03:47.992 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:03:48.075 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:03:48.098 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:03:48.098 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T00:03:48.098 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T00:03:48.099 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:03:48.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:03:48.116 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-25T00:03:48.174 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:03:48.174 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-437.el9.x86_64 2024-04-25T00:03:48.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-04-25T00:03:50.329 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-25T00:03:53.043 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T00:04:00.543 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-04-25T00:04:02.877 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T00:04:04.156 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T00:04:09.412 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T00:04:11.410 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:04:11.410 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:04:11.504 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T00:04:11.505 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-25T00:04:11.505 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-25T00:04:11.623 DEBUG:teuthology.orchestra.run.smithi045:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:04:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:04:09 AM UTC. 2024-04-25T00:04:12.310 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T00:04:12.319 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:12.319 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-04-25T00:04:12.319 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:12.319 INFO:teuthology.orchestra.run.smithi045.stdout:Reinstalling: 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:12.320 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 255 k 2024-04-25T00:04:12.321 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 0 2024-04-25T00:04:12.321 INFO:teuthology.orchestra.run.smithi045.stderr:Operation aborted. 2024-04-25T00:04:12.377 INFO:teuthology.orchestra.run.smithi045.stdout:Is this ok [y/N]: 2024-04-25T00:04:12.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum reinstall -y kernel || true 2024-04-25T00:04:12.891 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:04:09 AM UTC. 2024-04-25T00:04:13.011 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout:Reinstalling: 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-25T00:04:13.012 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-25T00:04:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:13.013 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 255 k 2024-04-25T00:04:13.013 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 0 2024-04-25T00:04:13.013 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-25T00:04:13.164 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-437.el9.x86_64.rpm 3.3 MB/s | 255 kB 00:00 2024-04-25T00:04:13.164 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:13.164 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.6 MB/s | 255 kB 00:00 2024-04-25T00:04:13.197 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-25T00:04:13.372 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-25T00:04:13.372 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-25T00:04:13.377 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-25T00:04:13.378 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-25T00:04:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-25T00:04:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:04:13.590 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:04:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:04:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout:Reinstalled: 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:04:14.988 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-25T00:04:15.154 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:04:15.202 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:04:15.203 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:04:15.203 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:04:15.203 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:15.203 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:04:15.216 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:04:15.216 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:04:15.218 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:04:15.221 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:04:15.238 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:04:15.275 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:04:15.326 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:04:15.326 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:04:15.328 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:04:15.330 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:04:15.459 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:04:15.466 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:04:15.466 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:04:15.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:04:15.484 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:04:15.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:04:15.501 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:04:15.503 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:04:15.526 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:04:15.533 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.546 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.571 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.581 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.601 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.606 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:04:15.608 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:04:15.629 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:04:15.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:04:15.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:04:15.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:04:15.677 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:04:15.751 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:04:15.751 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:04:15.754 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:04:15.756 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:04:15.772 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:15.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:15.907 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:15.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:16.029 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:04:16.029 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:04:16.094 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:16.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:16.183 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:16.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:16.310 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:04:16.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:04:16.376 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:16.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:04:16.465 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:16.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:04:16.588 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:04:16.588 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:04:16.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-25T00:04:16.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-25T00:04:16.658 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-25T00:04:16.692 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:04:16.693 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:04:16.737 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:04:17.182 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:04:17.246 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:04:17.246 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:04:17.330 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:04:17.380 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T00:04:17.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-04-25T00:04:17.431 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-04-25T00:04:17.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-04-25T00:04:17.502 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-04-25T00:04:17.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-04-25T00:04:17.571 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-04-25T00:04:17.573 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:04:17.573 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-04-25T00:04:17.597 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-04-25T00:04:17.597 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-04-25T00:04:17.619 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-04-25T00:04:17.619 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-04-25T00:04:17.639 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-04-25T00:04:17.639 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-04-25T00:04:17.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:04:17.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:04:17.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:04:17.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:04:17.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:04:17.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:04:17.717 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:04:17.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:04:17.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:04:17.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:04:17.794 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:04:17.810 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:04:17.830 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:04:17.831 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi071.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:08:07.891 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-04-25T00:08:07.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T00:08:07.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:08:08.028 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-25T00:08:08.111 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T00:08:08.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-25T00:08:08.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:08:08.188 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-25T00:08:08.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-25T00:08:08.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-25T00:08:08.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:08:08.343 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-25T00:08:08.425 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-25T00:08:08.426 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:08:08.470 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:08:08.470 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:08:08.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:08:08.473 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:08:08.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:08:08.475 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:08:08.475 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:08:08.503 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:08:08.504 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:08:08.505 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:08:08.516 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:08:08.519 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:08:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:08:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-04-25T00:08:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-04-25T00:08:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-04-25T00:08:08.549 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-04-25T00:08:08.554 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-04-25T00:08:08.557 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-04-25T00:08:08.563 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:08:08.568 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:08:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:08:08.575 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:08:08.581 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:08:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:08:08.630 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.634 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.640 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.643 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:08:08.648 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:08:08.744 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:08:08.744 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:08:08.744 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:08:08.744 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:08:08.750 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:08:08.750 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:08:08.751 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:08:08.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:08:08.754 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:08:08.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:08:08.755 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:08:08.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:08:08.914 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:08:08.914 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:08:09.083 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:08:09.083 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:08:09.090 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:08:09.090 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:08:09.148 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:08:09.148 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:08:09.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:08:09.177 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:08:09.178 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:08:09.190 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:08:09.190 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:08:09.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:08:09.198 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:08:09.198 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:08:09.198 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:08:09.218 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:08:09.218 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:08:09.225 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:08:09.225 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:08:09.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:08:09.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:08:09.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:08:09.327 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:08:09.363 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:08:09.364 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:08:09.393 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-04-25T00:08:09.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-04-25T00:08:09.427 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-04-25T00:08:09.427 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-04-25T00:08:09.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-04-25T00:08:09.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-04-25T00:08:09.632 INFO:teuthology.orchestra.run.smithi045.stdout:48 files removed 2024-04-25T00:08:09.654 INFO:teuthology.orchestra.run.smithi122.stdout:48 files removed 2024-04-25T00:08:09.656 INFO:teuthology.orchestra.run.smithi071.stdout:48 files removed 2024-04-25T00:08:09.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:08:09.681 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:08:09.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:08:10.265 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 259 kB/s | 83 kB 00:00 2024-04-25T00:08:10.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 214 kB/s | 83 kB 00:00 2024-04-25T00:08:10.352 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-04-25T00:08:10.691 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-25T00:08:10.699 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-25T00:08:10.728 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-25T00:08:11.058 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T00:08:11.065 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T00:08:11.137 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T00:08:11.833 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T00:08:12.074 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T00:08:13.129 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-04-25T00:08:15.039 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T00:08:17.041 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 19 MB 00:03 2024-04-25T00:08:17.809 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-04-25T00:08:19.031 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-25T00:08:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-25T00:08:21.775 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-25T00:08:22.693 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-25T00:08:22.859 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 21 MB 00:02 2024-04-25T00:08:23.392 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T00:08:24.118 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-25T00:08:24.852 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T00:08:25.053 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-25T00:08:27.892 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T00:08:30.019 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:30.019 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:30.030 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:08:30.031 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:08:30.032 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:08:30.032 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:08:30.034 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:08:30.037 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:08:30.037 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:08:30.098 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-25T00:08:30.116 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-25T00:08:30.116 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:08:30.117 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:08:30.118 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:08:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:08:30.120 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:08:30.121 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:08:30.122 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:08:30.123 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:08:30.124 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:08:30.125 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:08:30.126 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:08:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:08:30.128 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:08:30.129 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 4 Packages 2024-04-25T00:08:30.130 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:08:30.131 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T00:08:30.131 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 220 M 2024-04-25T00:08:30.132 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-25T00:08:30.135 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-25T00:08:31.511 INFO:teuthology.orchestra.run.smithi122.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T00:08:31.711 INFO:teuthology.orchestra.run.smithi122.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.3 MB/s | 890 kB 00:00 2024-04-25T00:08:31.769 INFO:teuthology.orchestra.run.smithi122.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.4 MB/s | 145 kB 00:00 2024-04-25T00:08:31.953 INFO:teuthology.orchestra.run.smithi122.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-04-25T00:08:32.070 INFO:teuthology.orchestra.run.smithi122.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-25T00:08:32.120 INFO:teuthology.orchestra.run.smithi122.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 8.9 MB/s | 1.5 MB 00:00 2024-04-25T00:08:32.212 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.212 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.223 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:08:32.224 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.225 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:08:32.225 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:08:32.227 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:08:32.229 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:08:32.230 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:08:32.290 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-25T00:08:32.311 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.312 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.315 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-25T00:08:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:08:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:08:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-04-25T00:08:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:08:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:08:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:08:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:08:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:08:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:08:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:08:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:08:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:08:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:08:32.328 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-25T00:08:32.328 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-25T00:08:32.328 INFO:teuthology.orchestra.run.smithi071.stdout:Install 148 Packages 2024-04-25T00:08:32.328 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 4 Packages 2024-04-25T00:08:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 220 M 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:08:32.329 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:08:32.330 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:08:32.331 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:08:32.395 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:08:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:08:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:08:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:08:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:08:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:08:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:08:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:08:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:08:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:08:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:08:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:08:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:08:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 4 Packages 2024-04-25T00:08:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:08:32.421 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 220 M 2024-04-25T00:08:32.421 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-25T00:08:32.654 INFO:teuthology.orchestra.run.smithi122.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-04-25T00:08:33.186 INFO:teuthology.orchestra.run.smithi045.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T00:08:33.396 INFO:teuthology.orchestra.run.smithi071.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-25T00:08:33.795 INFO:teuthology.orchestra.run.smithi045.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T00:08:33.870 INFO:teuthology.orchestra.run.smithi045.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.9 MB/s | 145 kB 00:00 2024-04-25T00:08:33.929 INFO:teuthology.orchestra.run.smithi122.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 9.3 MB/s | 17 MB 00:01 2024-04-25T00:08:34.138 INFO:teuthology.orchestra.run.smithi122.stdout:(9/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 7.1 MB/s | 21 MB 00:02 2024-04-25T00:08:34.163 INFO:teuthology.orchestra.run.smithi122.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 108 kB/s | 25 kB 00:00 2024-04-25T00:08:34.262 INFO:teuthology.orchestra.run.smithi045.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-25T00:08:34.280 INFO:teuthology.orchestra.run.smithi122.stdout:(11/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 6.3 MB/s | 10 MB 00:01 2024-04-25T00:08:34.305 INFO:teuthology.orchestra.run.smithi122.stdout:(12/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 222 kB 00:00 2024-04-25T00:08:34.347 INFO:teuthology.orchestra.run.smithi122.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 405 kB/s | 27 kB 00:00 2024-04-25T00:08:34.372 INFO:teuthology.orchestra.run.smithi122.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 472 kB/s | 32 kB 00:00 2024-04-25T00:08:34.396 INFO:teuthology.orchestra.run.smithi045.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-04-25T00:08:34.448 INFO:teuthology.orchestra.run.smithi122.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-25T00:08:34.482 INFO:teuthology.orchestra.run.smithi122.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.4 MB/s | 745 kB 00:00 2024-04-25T00:08:34.524 INFO:teuthology.orchestra.run.smithi122.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-25T00:08:34.571 INFO:teuthology.orchestra.run.smithi045.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-04-25T00:08:34.624 INFO:teuthology.orchestra.run.smithi122.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.4 MB/s | 501 kB 00:00 2024-04-25T00:08:34.708 INFO:teuthology.orchestra.run.smithi122.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 542 kB/s | 45 kB 00:00 2024-04-25T00:08:34.808 INFO:teuthology.orchestra.run.smithi122.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-25T00:08:34.908 INFO:teuthology.orchestra.run.smithi122.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-25T00:08:34.979 INFO:teuthology.orchestra.run.smithi071.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 562 kB/s | 890 kB 00:01 2024-04-25T00:08:35.008 INFO:teuthology.orchestra.run.smithi122.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.1 MB/s | 321 kB 00:00 2024-04-25T00:08:35.110 INFO:teuthology.orchestra.run.smithi122.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.9 MB/s | 299 kB 00:00 2024-04-25T00:08:35.201 INFO:teuthology.orchestra.run.smithi122.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T00:08:35.318 INFO:teuthology.orchestra.run.smithi122.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 729 kB/s | 85 kB 00:00 2024-04-25T00:08:35.329 INFO:teuthology.orchestra.run.smithi071.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 414 kB/s | 145 kB 00:00 2024-04-25T00:08:35.405 INFO:teuthology.orchestra.run.smithi045.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 4.7 MB/s | 4.7 MB 00:01 2024-04-25T00:08:35.835 INFO:teuthology.orchestra.run.smithi122.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 6.0 MB/s | 3.1 MB 00:00 2024-04-25T00:08:35.896 INFO:teuthology.orchestra.run.smithi071.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T00:08:35.902 INFO:teuthology.orchestra.run.smithi122.stdout:(27/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.9 MB/s | 5.3 MB 00:01 2024-04-25T00:08:35.927 INFO:teuthology.orchestra.run.smithi122.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-25T00:08:35.977 INFO:teuthology.orchestra.run.smithi122.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 422 kB/s | 32 kB 00:00 2024-04-25T00:08:36.011 INFO:teuthology.orchestra.run.smithi122.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.6 MB/s | 141 kB 00:00 2024-04-25T00:08:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-25T00:08:36.911 INFO:teuthology.orchestra.run.smithi122.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.8 MB/s | 3.5 MB 00:00 2024-04-25T00:08:37.012 INFO:teuthology.orchestra.run.smithi122.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.4 MB/s | 249 kB 00:00 2024-04-25T00:08:37.104 INFO:teuthology.orchestra.run.smithi122.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 537 kB/s | 49 kB 00:00 2024-04-25T00:08:37.196 INFO:teuthology.orchestra.run.smithi122.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 167 kB/s | 15 kB 00:00 2024-04-25T00:08:37.305 INFO:teuthology.orchestra.run.smithi122.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-04-25T00:08:37.330 INFO:teuthology.orchestra.run.smithi122.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 264 kB 00:00 2024-04-25T00:08:37.422 INFO:teuthology.orchestra.run.smithi122.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 199 kB/s | 18 kB 00:00 2024-04-25T00:08:37.464 INFO:teuthology.orchestra.run.smithi122.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T00:08:37.589 INFO:teuthology.orchestra.run.smithi122.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-04-25T00:08:37.692 INFO:teuthology.orchestra.run.smithi122.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T00:08:37.792 INFO:teuthology.orchestra.run.smithi122.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-25T00:08:37.817 INFO:teuthology.orchestra.run.smithi122.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T00:08:37.854 INFO:teuthology.orchestra.run.smithi045.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 4.2 MB/s | 10 MB 00:02 2024-04-25T00:08:37.876 INFO:teuthology.orchestra.run.smithi122.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T00:08:37.904 INFO:teuthology.orchestra.run.smithi045.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 506 kB/s | 25 kB 00:00 2024-04-25T00:08:37.960 INFO:teuthology.orchestra.run.smithi122.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T00:08:38.027 INFO:teuthology.orchestra.run.smithi122.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-04-25T00:08:38.052 INFO:teuthology.orchestra.run.smithi122.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T00:08:38.086 INFO:teuthology.orchestra.run.smithi122.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T00:08:38.153 INFO:teuthology.orchestra.run.smithi122.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-25T00:08:38.195 INFO:teuthology.orchestra.run.smithi122.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-25T00:08:38.237 INFO:teuthology.orchestra.run.smithi122.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-04-25T00:08:38.262 INFO:teuthology.orchestra.run.smithi122.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-25T00:08:38.305 INFO:teuthology.orchestra.run.smithi071.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 813 kB/s | 2.4 MB 00:02 2024-04-25T00:08:38.320 INFO:teuthology.orchestra.run.smithi122.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-04-25T00:08:38.346 INFO:teuthology.orchestra.run.smithi122.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-25T00:08:38.396 INFO:teuthology.orchestra.run.smithi122.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi045.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.4 MB/s | 17 MB 00:03 2024-04-25T00:08:38.439 INFO:teuthology.orchestra.run.smithi122.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-25T00:08:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 222 kB 00:00 2024-04-25T00:08:38.538 INFO:teuthology.orchestra.run.smithi045.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 466 kB/s | 27 kB 00:00 2024-04-25T00:08:38.547 INFO:teuthology.orchestra.run.smithi122.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-25T00:08:38.572 INFO:teuthology.orchestra.run.smithi122.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T00:08:38.597 INFO:teuthology.orchestra.run.smithi045.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 543 kB/s | 32 kB 00:00 2024-04-25T00:08:38.631 INFO:teuthology.orchestra.run.smithi122.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-04-25T00:08:38.689 INFO:teuthology.orchestra.run.smithi122.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-25T00:08:38.696 INFO:teuthology.orchestra.run.smithi071.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.3 MB/s | 4.7 MB 00:02 2024-04-25T00:08:38.739 INFO:teuthology.orchestra.run.smithi122.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-04-25T00:08:38.765 INFO:teuthology.orchestra.run.smithi122.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-25T00:08:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.0 MB/s | 745 kB 00:00 2024-04-25T00:08:38.807 INFO:teuthology.orchestra.run.smithi122.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-25T00:08:38.856 INFO:teuthology.orchestra.run.smithi045.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-25T00:08:38.857 INFO:teuthology.orchestra.run.smithi122.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-25T00:08:38.899 INFO:teuthology.orchestra.run.smithi122.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T00:08:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.8 MB/s | 127 kB 00:00 2024-04-25T00:08:38.924 INFO:teuthology.orchestra.run.smithi122.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-25T00:08:39.057 INFO:teuthology.orchestra.run.smithi045.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.7 MB/s | 501 kB 00:00 2024-04-25T00:08:39.167 INFO:teuthology.orchestra.run.smithi122.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-25T00:08:39.225 INFO:teuthology.orchestra.run.smithi122.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-25T00:08:39.342 INFO:teuthology.orchestra.run.smithi122.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T00:08:39.384 INFO:teuthology.orchestra.run.smithi122.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-25T00:08:39.451 INFO:teuthology.orchestra.run.smithi122.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T00:08:39.577 INFO:teuthology.orchestra.run.smithi122.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T00:08:39.643 INFO:teuthology.orchestra.run.smithi122.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T00:08:39.744 INFO:teuthology.orchestra.run.smithi122.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-25T00:08:39.828 INFO:teuthology.orchestra.run.smithi122.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T00:08:39.999 INFO:teuthology.orchestra.run.smithi045.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.6 MB/s | 5.3 MB 00:00 2024-04-25T00:08:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 905 kB/s | 45 kB 00:00 2024-04-25T00:08:40.099 INFO:teuthology.orchestra.run.smithi045.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.6 MB/s | 133 kB 00:00 2024-04-25T00:08:40.149 INFO:teuthology.orchestra.run.smithi045.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 3.2 MB/s | 162 kB 00:00 2024-04-25T00:08:40.242 INFO:teuthology.orchestra.run.smithi045.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.4 MB/s | 321 kB 00:00 2024-04-25T00:08:40.286 INFO:teuthology.orchestra.run.smithi122.stdout:(75/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.9 MB/s | 48 MB 00:06 2024-04-25T00:08:40.325 INFO:teuthology.orchestra.run.smithi045.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 299 kB 00:00 2024-04-25T00:08:40.336 INFO:teuthology.orchestra.run.smithi122.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 487 kB/s | 248 kB 00:00 2024-04-25T00:08:40.362 INFO:teuthology.orchestra.run.smithi122.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-25T00:08:40.375 INFO:teuthology.orchestra.run.smithi045.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T00:08:40.426 INFO:teuthology.orchestra.run.smithi045.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T00:08:40.430 INFO:teuthology.orchestra.run.smithi071.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 2.8 MB/s | 21 MB 00:07 2024-04-25T00:08:40.454 INFO:teuthology.orchestra.run.smithi122.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 7.4 MB/s | 6.0 MB 00:00 2024-04-25T00:08:40.479 INFO:teuthology.orchestra.run.smithi122.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 135 kB/s | 19 kB 00:00 2024-04-25T00:08:40.497 INFO:teuthology.orchestra.run.smithi071.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 380 kB/s | 25 kB 00:00 2024-04-25T00:08:40.521 INFO:teuthology.orchestra.run.smithi122.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-25T00:08:40.547 INFO:teuthology.orchestra.run.smithi122.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 934 kB/s | 172 kB 00:00 2024-04-25T00:08:40.613 INFO:teuthology.orchestra.run.smithi122.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T00:08:40.639 INFO:teuthology.orchestra.run.smithi122.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T00:08:40.697 INFO:teuthology.orchestra.run.smithi122.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 923 kB/s | 77 kB 00:00 2024-04-25T00:08:40.723 INFO:teuthology.orchestra.run.smithi122.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-04-25T00:08:40.835 INFO:teuthology.orchestra.run.smithi122.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T00:08:40.875 INFO:teuthology.orchestra.run.smithi122.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-25T00:08:40.876 INFO:teuthology.orchestra.run.smithi045.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 6.9 MB/s | 3.1 MB 00:00 2024-04-25T00:08:40.899 INFO:teuthology.orchestra.run.smithi122.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 904 kB/s | 159 kB 00:00 2024-04-25T00:08:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 3.4 MB/s | 171 kB 00:00 2024-04-25T00:08:40.949 INFO:teuthology.orchestra.run.smithi122.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 692 kB/s | 51 kB 00:00 2024-04-25T00:08:40.974 INFO:teuthology.orchestra.run.smithi122.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-25T00:08:40.984 INFO:teuthology.orchestra.run.smithi045.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 546 kB/s | 32 kB 00:00 2024-04-25T00:08:40.999 INFO:teuthology.orchestra.run.smithi122.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-04-25T00:08:41.042 INFO:teuthology.orchestra.run.smithi122.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 618 kB/s | 42 kB 00:00 2024-04-25T00:08:41.043 INFO:teuthology.orchestra.run.smithi045.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-25T00:08:41.059 INFO:teuthology.orchestra.run.smithi122.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 750 kB/s | 44 kB 00:00 2024-04-25T00:08:41.101 INFO:teuthology.orchestra.run.smithi122.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 868 kB/s | 36 kB 00:00 2024-04-25T00:08:41.126 INFO:teuthology.orchestra.run.smithi122.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-25T00:08:41.168 INFO:teuthology.orchestra.run.smithi122.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-25T00:08:41.218 INFO:teuthology.orchestra.run.smithi122.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-25T00:08:41.235 INFO:teuthology.orchestra.run.smithi122.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 861 kB/s | 58 kB 00:00 2024-04-25T00:08:41.302 INFO:teuthology.orchestra.run.smithi122.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T00:08:41.327 INFO:teuthology.orchestra.run.smithi122.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-04-25T00:08:41.386 INFO:teuthology.orchestra.run.smithi122.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-04-25T00:08:41.411 INFO:teuthology.orchestra.run.smithi122.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-04-25T00:08:41.489 INFO:teuthology.orchestra.run.smithi071.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 3.7 MB/s | 10 MB 00:02 2024-04-25T00:08:41.552 INFO:teuthology.orchestra.run.smithi045.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 6.9 MB/s | 3.5 MB 00:00 2024-04-25T00:08:41.564 INFO:teuthology.orchestra.run.smithi071.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 222 kB 00:00 2024-04-25T00:08:41.587 INFO:teuthology.orchestra.run.smithi122.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-25T00:08:41.623 INFO:teuthology.orchestra.run.smithi071.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 464 kB/s | 27 kB 00:00 2024-04-25T00:08:41.690 INFO:teuthology.orchestra.run.smithi071.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 475 kB/s | 32 kB 00:00 2024-04-25T00:08:41.729 INFO:teuthology.orchestra.run.smithi122.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T00:08:41.912 INFO:teuthology.orchestra.run.smithi122.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 107 kB/s | 19 kB 00:00 2024-04-25T00:08:41.932 INFO:teuthology.orchestra.run.smithi071.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 745 kB 00:00 2024-04-25T00:08:41.938 INFO:teuthology.orchestra.run.smithi122.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T00:08:41.980 INFO:teuthology.orchestra.run.smithi122.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 787 kB/s | 309 kB 00:00 2024-04-25T00:08:41.999 INFO:teuthology.orchestra.run.smithi071.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-25T00:08:42.005 INFO:teuthology.orchestra.run.smithi122.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-25T00:08:42.066 INFO:teuthology.orchestra.run.smithi071.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-25T00:08:42.072 INFO:teuthology.orchestra.run.smithi122.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T00:08:42.097 INFO:teuthology.orchestra.run.smithi122.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 414 kB/s | 49 kB 00:00 2024-04-25T00:08:42.139 INFO:teuthology.orchestra.run.smithi122.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T00:08:42.224 INFO:teuthology.orchestra.run.smithi071.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.1 MB/s | 501 kB 00:00 2024-04-25T00:08:42.298 INFO:teuthology.orchestra.run.smithi122.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-25T00:08:42.323 INFO:teuthology.orchestra.run.smithi122.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-25T00:08:42.357 INFO:teuthology.orchestra.run.smithi122.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 738 kB/s | 43 kB 00:00 2024-04-25T00:08:42.382 INFO:teuthology.orchestra.run.smithi122.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-04-25T00:08:42.416 INFO:teuthology.orchestra.run.smithi122.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-25T00:08:42.441 INFO:teuthology.orchestra.run.smithi122.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T00:08:42.542 INFO:teuthology.orchestra.run.smithi122.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T00:08:42.567 INFO:teuthology.orchestra.run.smithi122.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T00:08:42.634 INFO:teuthology.orchestra.run.smithi122.stdout:(120/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-25T00:08:42.644 INFO:teuthology.orchestra.run.smithi045.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-04-25T00:08:42.670 INFO:teuthology.orchestra.run.smithi122.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 280 kB/s | 35 kB 00:00 2024-04-25T00:08:42.693 INFO:teuthology.orchestra.run.smithi122.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-25T00:08:42.710 INFO:teuthology.orchestra.run.smithi045.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-25T00:08:42.727 INFO:teuthology.orchestra.run.smithi122.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T00:08:42.752 INFO:teuthology.orchestra.run.smithi122.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-25T00:08:42.769 INFO:teuthology.orchestra.run.smithi045.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 843 kB/s | 49 kB 00:00 2024-04-25T00:08:42.794 INFO:teuthology.orchestra.run.smithi122.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-25T00:08:42.811 INFO:teuthology.orchestra.run.smithi122.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 393 kB/s | 23 kB 00:00 2024-04-25T00:08:42.828 INFO:teuthology.orchestra.run.smithi045.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 263 kB/s | 15 kB 00:00 2024-04-25T00:08:42.853 INFO:teuthology.orchestra.run.smithi122.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-25T00:08:42.878 INFO:teuthology.orchestra.run.smithi122.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-25T00:08:42.895 INFO:teuthology.orchestra.run.smithi045.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 264 kB 00:00 2024-04-25T00:08:42.928 INFO:teuthology.orchestra.run.smithi122.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-04-25T00:08:42.945 INFO:teuthology.orchestra.run.smithi122.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-04-25T00:08:42.987 INFO:teuthology.orchestra.run.smithi122.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T00:08:43.004 INFO:teuthology.orchestra.run.smithi122.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-25T00:08:43.046 INFO:teuthology.orchestra.run.smithi122.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-25T00:08:43.053 INFO:teuthology.orchestra.run.smithi045.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T00:08:43.063 INFO:teuthology.orchestra.run.smithi122.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T00:08:43.097 INFO:teuthology.orchestra.run.smithi122.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T00:08:43.120 INFO:teuthology.orchestra.run.smithi045.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 274 kB/s | 18 kB 00:00 2024-04-25T00:08:43.122 INFO:teuthology.orchestra.run.smithi122.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-04-25T00:08:43.147 INFO:teuthology.orchestra.run.smithi122.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 890 kB/s | 90 kB 00:00 2024-04-25T00:08:43.173 INFO:teuthology.orchestra.run.smithi122.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-04-25T00:08:43.187 INFO:teuthology.orchestra.run.smithi045.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-25T00:08:43.198 INFO:teuthology.orchestra.run.smithi122.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T00:08:43.223 INFO:teuthology.orchestra.run.smithi122.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-04-25T00:08:43.248 INFO:teuthology.orchestra.run.smithi122.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T00:08:43.274 INFO:teuthology.orchestra.run.smithi122.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T00:08:43.299 INFO:teuthology.orchestra.run.smithi122.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T00:08:43.312 INFO:teuthology.orchestra.run.smithi045.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-25T00:08:43.316 INFO:teuthology.orchestra.run.smithi122.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-25T00:08:43.341 INFO:teuthology.orchestra.run.smithi071.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 4.8 MB/s | 5.3 MB 00:01 2024-04-25T00:08:43.350 INFO:teuthology.orchestra.run.smithi122.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-04-25T00:08:43.375 INFO:teuthology.orchestra.run.smithi122.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-25T00:08:43.379 INFO:teuthology.orchestra.run.smithi045.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T00:08:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 777 kB/s | 45 kB 00:00 2024-04-25T00:08:43.400 INFO:teuthology.orchestra.run.smithi122.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T00:08:43.458 INFO:teuthology.orchestra.run.smithi071.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.2 MB/s | 133 kB 00:00 2024-04-25T00:08:43.471 INFO:teuthology.orchestra.run.smithi045.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T00:08:43.517 INFO:teuthology.orchestra.run.smithi071.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.7 MB/s | 162 kB 00:00 2024-04-25T00:08:43.534 INFO:teuthology.orchestra.run.smithi122.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T00:08:43.580 INFO:teuthology.orchestra.run.smithi045.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T00:08:43.634 INFO:teuthology.orchestra.run.smithi071.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.7 MB/s | 321 kB 00:00 2024-04-25T00:08:43.664 INFO:teuthology.orchestra.run.smithi045.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T00:08:43.731 INFO:teuthology.orchestra.run.smithi045.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T00:08:43.784 INFO:teuthology.orchestra.run.smithi071.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 299 kB 00:00 2024-04-25T00:08:43.797 INFO:teuthology.orchestra.run.smithi045.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T00:08:43.868 INFO:teuthology.orchestra.run.smithi071.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-25T00:08:43.873 INFO:teuthology.orchestra.run.smithi045.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T00:08:43.935 INFO:teuthology.orchestra.run.smithi122.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-04-25T00:08:43.951 INFO:teuthology.orchestra.run.smithi071.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T00:08:43.985 INFO:teuthology.orchestra.run.smithi122.stdout:(150/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.6 MB/s | 3.4 MB 00:00 2024-04-25T00:08:44.160 INFO:teuthology.orchestra.run.smithi122.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-25T00:08:44.327 INFO:teuthology.orchestra.run.smithi122.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 2.4 MB 00:00 2024-04-25T00:08:44.333 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:08:44.333 INFO:teuthology.orchestra.run.smithi122.stdout:Total 15 MB/s | 220 MB 00:14 2024-04-25T00:08:44.410 INFO:teuthology.orchestra.run.smithi071.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 6.7 MB/s | 3.1 MB 00:00 2024-04-25T00:08:44.461 INFO:teuthology.orchestra.run.smithi071.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-25T00:08:44.523 INFO:teuthology.orchestra.run.smithi045.stdout:(47/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.3 MB/s | 48 MB 00:06 2024-04-25T00:08:44.536 INFO:teuthology.orchestra.run.smithi071.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 422 kB/s | 32 kB 00:00 2024-04-25T00:08:44.565 INFO:teuthology.orchestra.run.smithi045.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 996 kB/s | 689 kB 00:00 2024-04-25T00:08:44.587 INFO:teuthology.orchestra.run.smithi071.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.8 MB/s | 141 kB 00:00 2024-04-25T00:08:44.765 INFO:teuthology.orchestra.run.smithi045.stdout:(49/152): ceph-common-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 21 MB 00:11 2024-04-25T00:08:44.790 INFO:teuthology.orchestra.run.smithi045.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 475 kB/s | 107 kB 00:00 2024-04-25T00:08:44.882 INFO:teuthology.orchestra.run.smithi045.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-25T00:08:44.899 INFO:teuthology.orchestra.run.smithi045.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-04-25T00:08:44.941 INFO:teuthology.orchestra.run.smithi045.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-25T00:08:44.991 INFO:teuthology.orchestra.run.smithi045.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T00:08:44.995 INFO:teuthology.orchestra.run.smithi071.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.6 MB/s | 3.5 MB 00:00 2024-04-25T00:08:45.017 INFO:teuthology.orchestra.run.smithi045.stdout:(55/152): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-04-25T00:08:45.042 INFO:teuthology.orchestra.run.smithi045.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T00:08:45.075 INFO:teuthology.orchestra.run.smithi045.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T00:08:45.100 INFO:teuthology.orchestra.run.smithi045.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-25T00:08:45.201 INFO:teuthology.orchestra.run.smithi045.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-25T00:08:45.235 INFO:teuthology.orchestra.run.smithi045.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-25T00:08:45.260 INFO:teuthology.orchestra.run.smithi045.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T00:08:45.293 INFO:teuthology.orchestra.run.smithi045.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-25T00:08:45.360 INFO:teuthology.orchestra.run.smithi045.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-25T00:08:45.377 INFO:teuthology.orchestra.run.smithi045.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-25T00:08:45.419 INFO:teuthology.orchestra.run.smithi045.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T00:08:45.442 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-25T00:08:45.469 INFO:teuthology.orchestra.run.smithi045.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-04-25T00:08:45.585 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-25T00:08:45.585 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-25T00:08:45.711 INFO:teuthology.orchestra.run.smithi045.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-04-25T00:08:45.762 INFO:teuthology.orchestra.run.smithi071.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-04-25T00:08:45.829 INFO:teuthology.orchestra.run.smithi071.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.6 MB/s | 249 kB 00:00 2024-04-25T00:08:45.880 INFO:teuthology.orchestra.run.smithi071.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 989 kB/s | 49 kB 00:00 2024-04-25T00:08:45.886 INFO:teuthology.orchestra.run.smithi045.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-25T00:08:45.930 INFO:teuthology.orchestra.run.smithi071.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 307 kB/s | 15 kB 00:00 2024-04-25T00:08:45.945 INFO:teuthology.orchestra.run.smithi045.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T00:08:46.005 INFO:teuthology.orchestra.run.smithi071.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.4 MB/s | 264 kB 00:00 2024-04-25T00:08:46.037 INFO:teuthology.orchestra.run.smithi045.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-04-25T00:08:46.130 INFO:teuthology.orchestra.run.smithi071.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-25T00:08:46.171 INFO:teuthology.orchestra.run.smithi045.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-04-25T00:08:46.181 INFO:teuthology.orchestra.run.smithi071.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 365 kB/s | 18 kB 00:00 2024-04-25T00:08:46.196 INFO:teuthology.orchestra.run.smithi045.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-25T00:08:46.229 INFO:teuthology.orchestra.run.smithi045.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T00:08:46.296 INFO:teuthology.orchestra.run.smithi045.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T00:08:46.388 INFO:teuthology.orchestra.run.smithi045.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-25T00:08:46.480 INFO:teuthology.orchestra.run.smithi045.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T00:08:46.506 INFO:teuthology.orchestra.run.smithi045.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-04-25T00:08:46.539 INFO:teuthology.orchestra.run.smithi045.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-25T00:08:46.564 INFO:teuthology.orchestra.run.smithi045.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T00:08:46.598 INFO:teuthology.orchestra.run.smithi045.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T00:08:46.657 INFO:teuthology.orchestra.run.smithi045.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T00:08:46.673 INFO:teuthology.orchestra.run.smithi071.stdout:(38/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.8 MB/s | 48 MB 00:06 2024-04-25T00:08:46.698 INFO:teuthology.orchestra.run.smithi071.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 155 kB/s | 80 kB 00:00 2024-04-25T00:08:46.740 INFO:teuthology.orchestra.run.smithi045.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T00:08:46.765 INFO:teuthology.orchestra.run.smithi045.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-25T00:08:46.799 INFO:teuthology.orchestra.run.smithi045.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T00:08:46.833 INFO:teuthology.orchestra.run.smithi045.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-04-25T00:08:46.858 INFO:teuthology.orchestra.run.smithi045.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T00:08:46.873 INFO:teuthology.orchestra.run.smithi071.stdout:(40/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 17 MB 00:08 2024-04-25T00:08:46.891 INFO:teuthology.orchestra.run.smithi045.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T00:08:46.917 INFO:teuthology.orchestra.run.smithi045.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T00:08:46.950 INFO:teuthology.orchestra.run.smithi045.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-04-25T00:08:46.976 INFO:teuthology.orchestra.run.smithi045.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-04-25T00:08:47.132 INFO:teuthology.orchestra.run.smithi071.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 450 kB/s | 195 kB 00:00 2024-04-25T00:08:47.151 INFO:teuthology.orchestra.run.smithi045.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-04-25T00:08:47.176 INFO:teuthology.orchestra.run.smithi045.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 208 kB/s | 42 kB 00:00 2024-04-25T00:08:47.209 INFO:teuthology.orchestra.run.smithi045.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-04-25T00:08:47.235 INFO:teuthology.orchestra.run.smithi045.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T00:08:47.268 INFO:teuthology.orchestra.run.smithi045.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-04-25T00:08:47.294 INFO:teuthology.orchestra.run.smithi045.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-25T00:08:47.327 INFO:teuthology.orchestra.run.smithi045.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T00:08:47.350 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-25T00:08:47.350 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-25T00:08:47.352 INFO:teuthology.orchestra.run.smithi045.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-25T00:08:47.386 INFO:teuthology.orchestra.run.smithi045.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T00:08:47.411 INFO:teuthology.orchestra.run.smithi045.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T00:08:47.445 INFO:teuthology.orchestra.run.smithi045.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-25T00:08:47.482 INFO:teuthology.orchestra.run.smithi071.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 416 kB/s | 253 kB 00:00 2024-04-25T00:08:47.591 INFO:teuthology.orchestra.run.smithi071.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-04-25T00:08:47.608 INFO:teuthology.orchestra.run.smithi071.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 850 kB/s | 106 kB 00:00 2024-04-25T00:08:47.641 INFO:teuthology.orchestra.run.smithi071.stdout:(45/152): libgfortran-11.4.1-3.el9.x86_64.rpm 830 kB/s | 803 kB 00:00 2024-04-25T00:08:47.678 INFO:teuthology.orchestra.run.smithi045.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-04-25T00:08:47.692 INFO:teuthology.orchestra.run.smithi071.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-25T00:08:47.717 INFO:teuthology.orchestra.run.smithi071.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-25T00:08:47.751 INFO:teuthology.orchestra.run.smithi071.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-25T00:08:47.837 INFO:teuthology.orchestra.run.smithi045.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T00:08:47.893 INFO:teuthology.orchestra.run.smithi071.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-04-25T00:08:47.943 INFO:teuthology.orchestra.run.smithi071.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-25T00:08:47.960 INFO:teuthology.orchestra.run.smithi071.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-25T00:08:47.986 INFO:teuthology.orchestra.run.smithi071.stdout:(52/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-25T00:08:48.002 INFO:teuthology.orchestra.run.smithi071.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 777 kB/s | 33 kB 00:00 2024-04-25T00:08:48.012 INFO:teuthology.orchestra.run.smithi045.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-25T00:08:48.027 INFO:teuthology.orchestra.run.smithi071.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 313 kB 00:00 2024-04-25T00:08:48.044 INFO:teuthology.orchestra.run.smithi071.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 433 kB/s | 18 kB 00:00 2024-04-25T00:08:48.054 INFO:teuthology.orchestra.run.smithi045.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-04-25T00:08:48.070 INFO:teuthology.orchestra.run.smithi071.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-04-25T00:08:48.088 INFO:teuthology.orchestra.run.smithi045.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-25T00:08:48.138 INFO:teuthology.orchestra.run.smithi045.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-04-25T00:08:48.145 INFO:teuthology.orchestra.run.smithi071.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-25T00:08:48.171 INFO:teuthology.orchestra.run.smithi071.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T00:08:48.180 INFO:teuthology.orchestra.run.smithi045.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-25T00:08:48.187 INFO:teuthology.orchestra.run.smithi071.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 385 kB/s | 45 kB 00:00 2024-04-25T00:08:48.221 INFO:teuthology.orchestra.run.smithi071.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-25T00:08:48.222 INFO:teuthology.orchestra.run.smithi045.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T00:08:48.247 INFO:teuthology.orchestra.run.smithi071.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T00:08:48.272 INFO:teuthology.orchestra.run.smithi071.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T00:08:48.289 INFO:teuthology.orchestra.run.smithi045.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T00:08:48.297 INFO:teuthology.orchestra.run.smithi071.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-25T00:08:48.322 INFO:teuthology.orchestra.run.smithi071.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T00:08:48.356 INFO:teuthology.orchestra.run.smithi071.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-04-25T00:08:48.398 INFO:teuthology.orchestra.run.smithi071.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T00:08:48.498 INFO:teuthology.orchestra.run.smithi045.stdout:(111/152): protobuf-compiler-3.14.0-13.el9.x86_ 820 kB/s | 863 kB 00:01 2024-04-25T00:08:48.540 INFO:teuthology.orchestra.run.smithi045.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-04-25T00:08:48.598 INFO:teuthology.orchestra.run.smithi045.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 748 kB/s | 43 kB 00:00 2024-04-25T00:08:48.599 INFO:teuthology.orchestra.run.smithi071.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T00:08:48.649 INFO:teuthology.orchestra.run.smithi045.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-04-25T00:08:48.674 INFO:teuthology.orchestra.run.smithi071.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T00:08:48.690 INFO:teuthology.orchestra.run.smithi045.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-25T00:08:48.724 INFO:teuthology.orchestra.run.smithi071.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-25T00:08:48.749 INFO:teuthology.orchestra.run.smithi071.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-25T00:08:48.757 INFO:teuthology.orchestra.run.smithi045.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T00:08:48.799 INFO:teuthology.orchestra.run.smithi045.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-04-25T00:08:48.833 INFO:teuthology.orchestra.run.smithi071.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-25T00:08:48.891 INFO:teuthology.orchestra.run.smithi071.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T00:08:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-25T00:08:48.950 INFO:teuthology.orchestra.run.smithi071.stdout:(73/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.6 MB/s | 4.7 MB 00:00 2024-04-25T00:08:48.966 INFO:teuthology.orchestra.run.smithi045.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-25T00:08:48.992 INFO:teuthology.orchestra.run.smithi045.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-04-25T00:08:49.025 INFO:teuthology.orchestra.run.smithi071.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-25T00:08:49.051 INFO:teuthology.orchestra.run.smithi071.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-25T00:08:49.084 INFO:teuthology.orchestra.run.smithi071.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-25T00:08:49.109 INFO:teuthology.orchestra.run.smithi071.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-04-25T00:08:49.134 INFO:teuthology.orchestra.run.smithi045.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-04-25T00:08:49.135 INFO:teuthology.orchestra.run.smithi071.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T00:08:49.152 INFO:teuthology.orchestra.run.smithi071.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T00:08:49.227 INFO:teuthology.orchestra.run.smithi071.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-25T00:08:49.234 INFO:teuthology.orchestra.run.smithi045.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T00:08:49.252 INFO:teuthology.orchestra.run.smithi071.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-04-25T00:08:49.284 INFO:teuthology.orchestra.run.smithi045.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-25T00:08:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T00:08:49.318 INFO:teuthology.orchestra.run.smithi045.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 532 kB/s | 18 kB 00:00 2024-04-25T00:08:49.320 INFO:teuthology.orchestra.run.smithi071.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-25T00:08:49.345 INFO:teuthology.orchestra.run.smithi071.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T00:08:49.362 INFO:teuthology.orchestra.run.smithi071.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T00:08:49.398 INFO:teuthology.orchestra.run.smithi071.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.0 MB/s | 269 kB 00:00 2024-04-25T00:08:49.401 INFO:teuthology.orchestra.run.smithi045.stdout:(125/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-04-25T00:08:49.418 INFO:teuthology.orchestra.run.smithi045.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T00:08:49.423 INFO:teuthology.orchestra.run.smithi071.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T00:08:49.457 INFO:teuthology.orchestra.run.smithi071.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T00:08:49.468 INFO:teuthology.orchestra.run.smithi045.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-25T00:08:49.490 INFO:teuthology.orchestra.run.smithi071.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-25T00:08:49.494 INFO:teuthology.orchestra.run.smithi045.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-25T00:08:49.516 INFO:teuthology.orchestra.run.smithi071.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-04-25T00:08:49.594 INFO:teuthology.orchestra.run.smithi045.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-04-25T00:08:49.619 INFO:teuthology.orchestra.run.smithi045.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-25T00:08:49.624 INFO:teuthology.orchestra.run.smithi071.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-04-25T00:08:49.691 INFO:teuthology.orchestra.run.smithi071.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-04-25T00:08:49.694 INFO:teuthology.orchestra.run.smithi045.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-25T00:08:49.719 INFO:teuthology.orchestra.run.smithi045.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-04-25T00:08:49.741 INFO:teuthology.orchestra.run.smithi071.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 377 kB/s | 44 kB 00:00 2024-04-25T00:08:49.753 INFO:teuthology.orchestra.run.smithi045.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2024-04-25T00:08:49.767 INFO:teuthology.orchestra.run.smithi071.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T00:08:49.778 INFO:teuthology.orchestra.run.smithi045.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-25T00:08:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-04-25T00:08:49.804 INFO:teuthology.orchestra.run.smithi045.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T00:08:49.817 INFO:teuthology.orchestra.run.smithi071.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-25T00:08:49.829 INFO:teuthology.orchestra.run.smithi045.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-04-25T00:08:49.842 INFO:teuthology.orchestra.run.smithi071.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T00:08:49.846 INFO:teuthology.orchestra.run.smithi045.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 974 kB/s | 90 kB 00:00 2024-04-25T00:08:49.871 INFO:teuthology.orchestra.run.smithi045.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T00:08:49.876 INFO:teuthology.orchestra.run.smithi071.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-04-25T00:08:49.902 INFO:teuthology.orchestra.run.smithi071.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T00:08:49.913 INFO:teuthology.orchestra.run.smithi045.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T00:08:49.927 INFO:teuthology.orchestra.run.smithi071.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T00:08:49.939 INFO:teuthology.orchestra.run.smithi045.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 532 kB/s | 36 kB 00:00 2024-04-25T00:08:49.952 INFO:teuthology.orchestra.run.smithi071.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 887 kB/s | 45 kB 00:00 2024-04-25T00:08:49.964 INFO:teuthology.orchestra.run.smithi045.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 503 kB/s | 59 kB 00:00 2024-04-25T00:08:49.989 INFO:teuthology.orchestra.run.smithi045.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T00:08:49.994 INFO:teuthology.orchestra.run.smithi071.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 332 kB/s | 22 kB 00:00 2024-04-25T00:08:50.014 INFO:teuthology.orchestra.run.smithi045.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T00:08:50.039 INFO:teuthology.orchestra.run.smithi045.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T00:08:50.065 INFO:teuthology.orchestra.run.smithi045.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-25T00:08:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-04-25T00:08:50.090 INFO:teuthology.orchestra.run.smithi045.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T00:08:50.148 INFO:teuthology.orchestra.run.smithi045.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T00:08:50.199 INFO:teuthology.orchestra.run.smithi045.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T00:08:50.311 INFO:teuthology.orchestra.run.smithi071.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T00:08:50.316 INFO:teuthology.orchestra.run.smithi045.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T00:08:50.458 INFO:teuthology.orchestra.run.smithi045.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-25T00:08:50.512 INFO:teuthology.orchestra.run.smithi071.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 98 kB/s | 19 kB 00:00 2024-04-25T00:08:50.537 INFO:teuthology.orchestra.run.smithi071.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 698 kB/s | 309 kB 00:00 2024-04-25T00:08:50.587 INFO:teuthology.orchestra.run.smithi071.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-25T00:08:50.646 INFO:teuthology.orchestra.run.smithi071.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-04-25T00:08:50.679 INFO:teuthology.orchestra.run.smithi071.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 800 kB/s | 548 kB 00:00 2024-04-25T00:08:50.705 INFO:teuthology.orchestra.run.smithi071.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T00:08:50.783 INFO:teuthology.orchestra.run.smithi045.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-04-25T00:08:50.805 INFO:teuthology.orchestra.run.smithi071.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T00:08:50.872 INFO:teuthology.orchestra.run.smithi071.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T00:08:50.923 INFO:teuthology.orchestra.run.smithi071.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 874 kB/s | 43 kB 00:00 2024-04-25T00:08:50.983 INFO:teuthology.orchestra.run.smithi045.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.8 MB/s | 3.4 MB 00:00 2024-04-25T00:08:50.986 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:08:50.986 INFO:teuthology.orchestra.run.smithi045.stdout:Total 12 MB/s | 220 MB 00:18 2024-04-25T00:08:51.015 INFO:teuthology.orchestra.run.smithi071.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-04-25T00:08:51.073 INFO:teuthology.orchestra.run.smithi071.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-04-25T00:08:51.090 INFO:teuthology.orchestra.run.smithi071.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-25T00:08:51.116 INFO:teuthology.orchestra.run.smithi071.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-25T00:08:51.141 INFO:teuthology.orchestra.run.smithi071.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T00:08:51.174 INFO:teuthology.orchestra.run.smithi071.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-25T00:08:51.192 INFO:teuthology.orchestra.run.smithi071.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-04-25T00:08:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T00:08:51.275 INFO:teuthology.orchestra.run.smithi071.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-25T00:08:51.301 INFO:teuthology.orchestra.run.smithi071.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-25T00:08:51.326 INFO:teuthology.orchestra.run.smithi071.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-25T00:08:51.351 INFO:teuthology.orchestra.run.smithi071.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T00:08:51.402 INFO:teuthology.orchestra.run.smithi071.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-04-25T00:08:51.427 INFO:teuthology.orchestra.run.smithi071.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-25T00:08:51.452 INFO:teuthology.orchestra.run.smithi071.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-25T00:08:51.477 INFO:teuthology.orchestra.run.smithi071.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-25T00:08:51.502 INFO:teuthology.orchestra.run.smithi071.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 929 kB/s | 46 kB 00:00 2024-04-25T00:08:51.528 INFO:teuthology.orchestra.run.smithi071.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T00:08:51.553 INFO:teuthology.orchestra.run.smithi071.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T00:08:51.587 INFO:teuthology.orchestra.run.smithi071.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-25T00:08:51.612 INFO:teuthology.orchestra.run.smithi071.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-25T00:08:51.637 INFO:teuthology.orchestra.run.smithi071.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T00:08:51.663 INFO:teuthology.orchestra.run.smithi071.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-25T00:08:51.688 INFO:teuthology.orchestra.run.smithi071.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-25T00:08:51.722 INFO:teuthology.orchestra.run.smithi071.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-25T00:08:51.747 INFO:teuthology.orchestra.run.smithi071.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T00:08:51.772 INFO:teuthology.orchestra.run.smithi071.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-04-25T00:08:51.806 INFO:teuthology.orchestra.run.smithi071.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 867 kB/s | 51 kB 00:00 2024-04-25T00:08:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:(142/152): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-04-25T00:08:51.873 INFO:teuthology.orchestra.run.smithi071.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T00:08:51.899 INFO:teuthology.orchestra.run.smithi071.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-04-25T00:08:51.924 INFO:teuthology.orchestra.run.smithi071.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-25T00:08:51.966 INFO:teuthology.orchestra.run.smithi071.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T00:08:52.008 INFO:teuthology.orchestra.run.smithi071.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-25T00:08:52.067 INFO:teuthology.orchestra.run.smithi071.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T00:08:52.136 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-25T00:08:52.296 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-25T00:08:52.296 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-25T00:08:52.808 INFO:teuthology.orchestra.run.smithi071.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-04-25T00:08:53.100 INFO:teuthology.orchestra.run.smithi071.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-04-25T00:08:53.280 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-25T00:08:53.364 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:08:53.445 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:08:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:08:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:08:53.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:08:53.784 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:08:53.800 INFO:teuthology.orchestra.run.smithi071.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-04-25T00:08:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:08:54.190 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-25T00:08:54.191 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-25T00:08:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:08:54.347 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:08:54.377 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:08:54.451 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:08:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:08:54.641 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:08:54.698 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:08:54.911 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:08:54.917 INFO:teuthology.orchestra.run.smithi071.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 1.1 MB/s | 3.4 MB 00:02 2024-04-25T00:08:54.923 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:08:54.923 INFO:teuthology.orchestra.run.smithi071.stdout:Total 9.7 MB/s | 220 MB 00:22 2024-04-25T00:08:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:08:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:08:55.104 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:08:55.126 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:08:55.150 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:08:55.350 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:08:55.384 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:08:55.490 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:08:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:08:55.626 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:08:55.661 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:08:55.734 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:08:55.769 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:08:55.813 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:08:55.845 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:08:55.873 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:08:55.944 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:08:55.999 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:08:56.026 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:08:56.037 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-25T00:08:56.070 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:08:56.147 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:08:56.178 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-25T00:08:56.178 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-25T00:08:56.181 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:08:56.224 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:08:56.253 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:08:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:08:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:08:56.443 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:08:56.586 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:08:56.637 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:08:57.164 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:08:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:08:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:08:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:08:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:08:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:08:57.557 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:08:57.622 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:08:57.677 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:08:57.750 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:08:57.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:08:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:08:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:08:57.901 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:08:57.943 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:08:57.951 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-25T00:08:57.952 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-25T00:08:57.995 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:08:58.130 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:08:58.175 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:08:58.206 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:08:58.247 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:08:58.341 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:08:58.381 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:08:58.421 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:08:58.446 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:08:58.471 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:08:58.546 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:08:58.673 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:08:58.721 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:08:58.749 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:08:58.783 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:08:59.016 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:08:59.064 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:08:59.093 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:08:59.128 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:08:59.162 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:08:59.205 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:08:59.237 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:08:59.326 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:08:59.376 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:08:59.455 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:08:59.553 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:08:59.601 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:09:00.244 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-25T00:09:00.322 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:09:00.344 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:09:00.373 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:09:00.407 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:09:00.509 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:09:00.565 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:09:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:09:00.653 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:09:00.701 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:09:00.741 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:09:00.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:09:00.805 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:09:00.851 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:09:00.877 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:09:00.907 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:09:00.985 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:09:01.205 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:09:01.313 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:01.464 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:01.559 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:01.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:09:01.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:09:01.739 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:09:01.749 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:09:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:09:01.827 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:01.923 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:01.952 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:09:02.015 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:09:02.037 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:09:02.058 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:09:02.254 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:09:02.287 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:09:02.399 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:09:02.494 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:09:02.536 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:09:02.579 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:09:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:09:02.703 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:09:02.747 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:09:02.779 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:09:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:09:02.878 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:09:02.940 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:09:02.961 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:09:02.976 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:09:03.011 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:09:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:09:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:09:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:09:03.100 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:09:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:09:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:09:03.211 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:09:03.282 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:09:03.326 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:09:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:09:03.410 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:09:03.446 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:03.547 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:09:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:09:03.603 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:09:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:09:04.073 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:09:04.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:04.147 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-25T00:09:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:09:04.162 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:04.185 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:09:04.229 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:09:04.251 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:09:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:09:04.285 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:09:04.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:09:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:09:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:09:04.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:09:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:09:04.408 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:09:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:09:04.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:09:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:09:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:09:04.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:09:04.767 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:09:04.786 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:09:04.822 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:09:04.869 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:09:04.873 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:04.905 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:09:04.949 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:09:04.987 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:04.990 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:09:05.008 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:05.083 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:05.112 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:09:05.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:09:05.179 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:09:05.208 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:09:05.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:09:05.261 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:09:05.309 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:09:05.338 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:05.347 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:09:05.383 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:09:05.415 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:09:05.441 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:05.449 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:09:05.463 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:09:05.473 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:09:05.526 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:09:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:09:05.551 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:09:05.566 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:09:05.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:09:05.722 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:09:05.751 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:09:05.773 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:09:05.806 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:09:05.810 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:09:05.914 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:09:05.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:09:06.025 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:09:06.072 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:09:06.080 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:09:06.106 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:09:06.110 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:09:06.144 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:09:06.171 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:09:06.178 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:09:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:09:06.220 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:09:06.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:09:06.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:09:06.290 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:09:06.318 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:09:06.343 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:09:06.388 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:09:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:09:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:09:06.441 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:09:06.448 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:06.471 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:09:06.479 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:09:06.482 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:09:06.582 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:09:06.583 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:09:06.617 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:09:06.633 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:09:06.661 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:09:06.690 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:09:06.768 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:09:06.813 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:09:06.896 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:09:07.028 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:07.029 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:09:07.069 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:07.082 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:09:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:07.146 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:09:07.146 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:09:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:07.388 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:09:07.438 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:09:07.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:09:07.575 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:09:07.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:09:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:09:07.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:09:07.716 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:09:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:09:07.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:09:07.767 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:09:07.814 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:09:07.855 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:09:07.924 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:09:07.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:09:08.028 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:09:08.034 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:09:08.076 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:09:08.130 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:09:08.203 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:09:08.234 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:09:08.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:09:08.318 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:09:08.354 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:09:08.397 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:09:08.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:09:08.576 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:09:08.620 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:09:08.650 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:09:08.692 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:09:08.711 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:09:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:09:08.817 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:09:08.832 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:09:08.858 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:09:08.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:09:08.908 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:09:08.983 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:09:09.111 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:09:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:09:09.186 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:09:09.220 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:09:09.260 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:09:09.291 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:09:09.321 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:09:09.356 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:09:09.389 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:09:09.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:09:09.473 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:09:09.561 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:09:09.604 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:09:09.635 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:09:09.683 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:09:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:09:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:09:10.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:09:10.199 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:09:10.227 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:09:10.260 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:09:10.555 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:09:10.588 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:09:10.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:09:10.635 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:09:10.639 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:10.731 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:10.733 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:09:10.761 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:09:10.785 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:09:10.797 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:09:10.872 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:09:10.921 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:09:11.099 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:09:11.127 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:09:11.205 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:09:11.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:11.388 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:11.418 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:09:11.852 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:09:11.960 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:09:13.205 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:09:13.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:09:13.328 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:09:13.361 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:09:13.674 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:09:13.716 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:13.761 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:13.806 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:09:13.837 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:09:13.874 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:09:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:14.371 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:09:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:14.402 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:14.473 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:09:14.484 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:14.484 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:09:14.485 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:09:14.485 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:14.496 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:09:14.510 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:16.790 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:17.018 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:17.558 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:09:17.597 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:17.685 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:09:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:09:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:09:17.685 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:17.714 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:18.654 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:18.654 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-04-25T00:09:18.654 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-04-25T00:09:18.654 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-04-25T00:09:18.655 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-04-25T00:09:18.655 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-04-25T00:09:18.655 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-04-25T00:09:18.655 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-04-25T00:09:18.655 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:18.820 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:18.835 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:18.835 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:18.835 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:09:18.836 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:20.392 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:20.392 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:09:20.392 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:20.392 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:20.647 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:20.647 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:09:20.648 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:20.648 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:09:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:09:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:09:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:09:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:09:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:09:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:09:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:09:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:09:22.494 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:22.528 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:22.560 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:09:22.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:09:22.660 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:23.690 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:23.721 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:24.686 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:24.721 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:24.823 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:09:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:09:25.039 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:25.083 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:25.094 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:09:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:09:26.136 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:09:26.169 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:26.191 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:26.191 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:26.191 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:09:26.192 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:26.192 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:26.192 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:26.236 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:26.258 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:26.258 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:26.258 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:09:26.258 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:26.328 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:26.351 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:26.351 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:26.351 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:09:26.351 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:26.352 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:26.352 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-04-25T00:09:26.365 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-04-25T00:09:26.366 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-04-25T00:09:26.366 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:26.654 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:26.677 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:28.088 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:28.115 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:28.347 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:28.372 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:28.841 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:28.866 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:28.972 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:09:29.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:09:29.093 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:09:29.287 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:09:29.334 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:09:29.378 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:09:29.378 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-04-25T00:09:29.378 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-04-25T00:09:29.378 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-04-25T00:09:29.378 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-04-25T00:09:29.379 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-04-25T00:09:29.379 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-04-25T00:09:29.379 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-04-25T00:09:29.379 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:29.415 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:09:29.512 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:29.547 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:09:29.563 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:29.563 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:09:29.563 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:29.662 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:29.700 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:09:29.732 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:09:29.759 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:09:30.126 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:30.157 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:30.197 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:09:30.262 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:09:30.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:31.059 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:31.084 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:09:31.084 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:31.085 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:09:31.085 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:31.085 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:09:31.085 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:09:31.241 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:09:31.283 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:09:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:31.314 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:09:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:31.342 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:09:31.379 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:31.415 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:31.526 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:09:31.572 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:09:31.602 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:09:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:09:31.799 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:31.800 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:09:31.800 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:31.800 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:09:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:09:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:09:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:09:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:09:32.214 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:09:32.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:32.270 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:09:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:09:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:09:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:09:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:09:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:32.792 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:32.792 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:09:32.793 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:32.793 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:09:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:09:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:09:33.095 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:09:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:09:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:09:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:09:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:33.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:09:33.313 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:09:33.353 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:09:33.353 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:33.446 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:33.470 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:33.470 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:33.563 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:33.626 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:33.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:09:33.987 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:34.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:34.054 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:34.078 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:34.078 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:09:34.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:34.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:34.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:34.239 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:09:34.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:34.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:34.496 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:34.497 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:34.497 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:09:34.497 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:34.497 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:34.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:09:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:35.373 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:09:35.534 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:09:35.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:35.580 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:09:35.581 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:35.581 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:09:35.581 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:35.581 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:09:35.581 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:35.625 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:09:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:09:35.790 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:09:36.689 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:09:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:36.742 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:09:36.742 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:36.742 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:09:36.743 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:36.743 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:09:36.743 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:36.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:36.817 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:09:36.817 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:36.817 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:09:36.817 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:36.862 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:09:36.885 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:36.885 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:09:36.885 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:36.885 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:09:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:37.188 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:37.210 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:09:37.211 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:09:37.211 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:09:37.211 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:37.211 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:09:37.211 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:09:39.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:09:39.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:09:39.349 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:09:39.376 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:09:39.561 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:09:39.605 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:09:39.634 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:09:39.705 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:39.705 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:09:39.705 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:09:39.706 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:09:39.707 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:09:39.708 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:09:39.709 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:09:39.710 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:09:39.711 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:09:39.712 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:09:39.713 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:09:39.714 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:09:39.715 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:09:39.716 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:09:39.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:39.717 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:09:40.232 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:40.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:41.193 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:09:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:09:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:09:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:09:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:09:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:41.646 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:41.710 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:09:41.743 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:41.743 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:41.743 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-04-25T00:09:41.743 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.744 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.745 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:09:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.747 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:09:41.748 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:09:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:09:41.750 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:09:41.751 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:09:41.752 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:09:41.753 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.754 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:09:41.755 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-25T00:09:41.766 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:09:41.807 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:09:41.839 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:09:41.867 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:09:42.053 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:09:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:09:42.135 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:09:42.155 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:42.765 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:09:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:09:43.748 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:09:43.789 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:09:43.870 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:09:43.910 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:09:43.910 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:43.986 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:09:44.019 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:44.019 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:44.106 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:09:44.177 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:47.671 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:47.671 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:09:47.671 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:09:47.671 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:09:47.672 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:09:47.673 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:09:47.674 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:09:47.675 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:09:47.676 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:09:47.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:09:47.678 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:09:47.679 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:09:47.680 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:09:47.681 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:09:47.682 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:09:47.683 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:09:47.683 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:47.683 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:09:49.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:09:49.764 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.765 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:09:49.766 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:09:49.767 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:09:49.768 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:09:49.769 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:09:49.770 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:09:49.771 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:09:49.772 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:09:49.773 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:09:49.774 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:09:49.775 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:09:49.776 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:09:49.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:09:49.776 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-25T00:09:50.173 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:09:50.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:09:50.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:09:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:09:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:09:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:09:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:09:50.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:09:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:09:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:09:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:09:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:09:50.398 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:09:50.399 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:09:50.400 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:09:50.401 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:09:50.402 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:09:50.403 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:09:50.404 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:09:50.405 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:09:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:52.483 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.484 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.485 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:09:52.486 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.487 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:09:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.489 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:09:52.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:09:52.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.492 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:09:52.493 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:09:52.494 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:09:52.495 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:09:52.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-25T00:09:52.495 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-25T00:09:52.898 DEBUG:teuthology.parallel:result is None 2024-04-25T00:09:52.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:53.135 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:09:53.162 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.162 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.162 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:09:53.164 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:53.409 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:09:53.431 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.432 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.432 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:09:53.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:53.580 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:09:53.607 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.607 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:09:53.607 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:09:53.609 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:09:53.609 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:09:53.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:09:53.641 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:09:53.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:09:53.667 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:09:53.667 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:09:53.697 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:09:53.697 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:09:53.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:09:53.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:09:53.791 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:09:53.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:09:53.816 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:09:53.881 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:09:53.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:09:53.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:09:53.978 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:09:53.979 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:09:53.979 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:09:54.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:09:54.073 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:09:54.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:09:54.096 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:09:54.160 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:09:54.160 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:09:54.188 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:09:54.254 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:09:54.255 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:09:54.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:09:54.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:09:54.349 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:09:54.349 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:09:54.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:09:54.437 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:09:54.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:09:54.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:09:54.531 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T00:09:54.629 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T00:09:54.630 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:54.630 INFO:tasks.cephadm:Cluster fsid is 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:09:54.630 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T00:09:54.630 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.71', 'mon.c': '172.21.15.122'} 2024-04-25T00:09:54.631 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-04-25T00:09:54.631 INFO:tasks.cephadm:First mgr is x 2024-04-25T00:09:54.631 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T00:09:54.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-04-25T00:09:54.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-04-25T00:09:54.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-04-25T00:09:54.712 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T00:09:54.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:54.925 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T00:09:55.072 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:09:55.073 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T00:09:55.074 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:55.458 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:09 /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:55.458 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:55.996 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:09 /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:55.996 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:56.383 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:09 /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:56.383 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:56.403 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:56.420 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:09:56.539 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T00:09:56.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:09:56.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:09:56.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:09:58.080 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:09:58.117 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:09:58.254 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:10:46.758 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-04-25T00:10:46.759 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-25T00:11:22.536 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-25T00:11:22.536 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:11:22.536 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:11:22.536 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-04-25T00:11:22.536 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:11:22.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-25T00:11:22.537 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-25T00:11:23.267 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:11:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:11:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:11:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-04-25T00:11:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:11:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:11:23.268 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:11:23.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-04-25T00:11:23.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-04-25T00:11:23.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-04-25T00:11:23.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-04-25T00:11:23.426 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-04-25T00:11:23.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-04-25T00:11:23.480 INFO:tasks.cephadm:Writing seed config... 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T00:11:23.482 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T00:11:23.483 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T00:11:23.484 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T00:11:23.485 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T00:11:23.485 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T00:11:23.485 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-25T00:11:23.485 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:11:23.485 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T00:11:23.501 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 24bb2e7a-0298-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T00:11:23.501 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T00:11:23.543 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T00:11:23.586 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T00:11:23.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:11:23.782 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:11:23.782 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '24bb2e7a-0298-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-04-25T00:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T00:11:23.783 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-04-25T00:11:23.809 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:11:23.809 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-04-25T00:11:23.809 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-04-25T00:11:23.816 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:11:23.816 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:11:23.822 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:11:23.822 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:11:23.829 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-25T00:11:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-25T00:11:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:11:23.835 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-04-25T00:11:23.860 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:11:23.860 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T00:11:23.860 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-04-25T00:11:23.860 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-04-25T00:11:23.866 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:11:23.866 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:11:23.872 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:11:23.872 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:11:23.879 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-25T00:11:23.885 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-25T00:11:23.885 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:11:23.885 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-04-25T00:11:23.885 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:23.886 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139978078151584 on /run/cephadm/24bb2e7a-0298-11ef-bc93-c7b262605968.lock 2024-04-25T00:11:23.886 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139978078151584 acquired on /run/cephadm/24bb2e7a-0298-11ef-bc93-c7b262605968.lock 2024-04-25T00:11:23.886 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-04-25T00:11:23.886 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-04-25T00:11:23.886 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-25T00:11:23.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-04-25T00:11:23.890 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-04-25T00:11:23.892 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T00:11:23.892 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T00:11:23.892 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-25T00:11:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:11:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-25T00:11:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-25T00:11:23.899 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T00:11:23.899 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T00:11:23.899 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:11:24.320 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T00:11:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:11:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:11:25.234 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T00:11:26.132 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-04-25T00:11:26.132 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-04-25T00:11:26.951 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCunylm5SUUGhAATrayLpztFQSwAYe9X2wR8Q== 2024-04-25T00:11:27.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCvnylmoycrEBAAa4IOlru4jzzAvouznGSl3Q== 2024-04-25T00:11:28.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCwnylmXu/bBxAA76mgQL8Lc22SOUT00UEGLg== 2024-04-25T00:11:28.608 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-04-25T00:11:29.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:11:29.432 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-04-25T00:11:30.330 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-04-25T00:11:30.549 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T00:11:30.714 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T00:11:30.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target. 2024-04-25T00:11:30.915 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target. 2024-04-25T00:11:31.178 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:11:31.178 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T00:11:31.356 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target.wants/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@.service. 2024-04-25T00:11:31.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 systemd[1]: Starting Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:11:31.710 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 podman[28803]: 2024-04-25 00:11:31.613570694 +0000 UTC m=+0.111162025 container create f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:11:31.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 podman[28803]: 2024-04-25 00:11:31.520284847 +0000 UTC m=+0.017876189 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:11:31.890 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:11:31.890 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-04-25T00:11:31.898 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-04-25T00:11:31.991 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 podman[28803]: 2024-04-25 00:11:31.8081097 +0000 UTC m=+0.305701039 container init f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:11:31.991 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 podman[28803]: 2024-04-25 00:11:31.811332224 +0000 UTC m=+0.308923553 container start f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T00:11:31.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 ceph-mon[28821]: mkfs 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:31.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 ceph-mon[28821]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:11:31.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 bash[28803]: f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 2024-04-25T00:11:31.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:31 smithi045 systemd[1]: Started Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.55924s) 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T00:11:32.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.330+0000 7f7d7e767700 1 Processor -- start 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d7e767700 1 -- start start 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d7e767700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d7e767700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78108660 con 0x7f7d78107d00 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57722/0 (socket says 172.21.15.45:57722) 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d77fff700 1 -- 172.21.15.45:0/2885788163 learned_addr learned my addr 172.21.15.45:0/2885788163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.331+0000 7f7d77fff700 1 -- 172.21.15.45:0/2885788163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d781087e0 con 0x7f7d78107d00 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d77fff700 1 --2- 172.21.15.45:0/2885788163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0098a0 tx=0x7f7d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19b8c06efa0d265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d76ffd700 1 -- 172.21.15.45:0/2885788163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d6c004160 con 0x7f7d78107d00 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d76ffd700 1 -- 172.21.15.45:0/2885788163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c004300 con 0x7f7d78107d00 2024-04-25T00:11:32.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d76ffd700 1 -- 172.21.15.45:0/2885788163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d6c0036a0 con 0x7f7d78107d00 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 -- 172.21.15.45:0/2885788163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 msgr2=0x7f7d78108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 --2- 172.21.15.45:0/2885788163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0098a0 tx=0x7f7d6c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 -- 172.21.15.45:0/2885788163 shutdown_connections 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 --2- 172.21.15.45:0/2885788163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d78108120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 -- 172.21.15.45:0/2885788163 >> 172.21.15.45:0/2885788163 conn(0x7f7d781033a0 msgr2=0x7f7d781057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 -- 172.21.15.45:0/2885788163 shutdown_connections 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.332+0000 7f7d7e767700 1 -- 172.21.15.45:0/2885788163 wait complete. 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.333+0000 7f7d7e767700 1 Processor -- start 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.333+0000 7f7d7e767700 1 -- start start 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.333+0000 7f7d7e767700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.333+0000 7f7d7e767700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78109160 con 0x7f7d78107d00 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d77fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57732/0 (socket says 172.21.15.45:57732) 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d77fff700 1 -- 172.21.15.45:0/2435621930 learned_addr learned my addr 172.21.15.45:0/2435621930 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:32.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d77fff700 1 -- 172.21.15.45:0/2435621930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7819b860 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d77fff700 1 --2- 172.21.15.45:0/2435621930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d6c003db0 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c011630 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d6c011950 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7819baf0 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7819e7e0 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d6c011af0 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.334+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d6c01c450 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.335+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d78063890 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.338+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7d6c032020 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.411+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7d7802dcb0 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.411+0000 7f7d757fa700 1 -- 172.21.15.45:0/2435621930 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f7d6c004260 con 0x7f7d78107d00 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 msgr2=0x7f7d7819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 --2- 172.21.15.45:0/2435621930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c004750 comp rx=0 tx=0).stop 2024-04-25T00:11:32.947 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 shutdown_connections 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 --2- 172.21.15.45:0/2435621930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d78107d00 0x7f7d7819b320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 >> 172.21.15.45:0/2435621930 conn(0x7f7d781033a0 msgr2=0x7f7d78191f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 shutdown_connections 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:32.412+0000 7f7d7e767700 1 -- 172.21.15.45:0/2435621930 wait complete. 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-04-25T00:11:32.948 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T00:11:33.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:11:33.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: monmap epoch 1 2024-04-25T00:11:33.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: last_changed 2024-04-25T00:11:28.889760+0000 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: min_mon_release 19 (squid) 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: election_strategy: 1 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: fsmap 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: mgrmap e1: no daemons active 2024-04-25T00:11:33.063 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:32 smithi045 ceph-mon[28821]: from='client.? 172.21.15.45:0/2435621930' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T00:11:34.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:33 smithi045 ceph-mon[28821]: from='client.? 172.21.15.45:0/925467522' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:11:34.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:33 smithi045 ceph-mon[28821]: from='client.? 172.21.15.45:0/925467522' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T00:11:34.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:34.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:11:34.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.385+0000 7fa26537a700 1 Processor -- start 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.385+0000 7fa26537a700 1 -- start start 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.385+0000 7fa26537a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.385+0000 7fa26537a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa260106c70 con 0x7fa260106280 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57738/0 (socket says 172.21.15.45:57738) 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25effd700 1 -- 172.21.15.45:0/3561641776 learned_addr learned my addr 172.21.15.45:0/3561641776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25effd700 1 -- 172.21.15.45:0/3561641776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa260107480 con 0x7fa260106280 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25effd700 1 --2- 172.21.15.45:0/3561641776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa25800af90 tx=0x7fa25800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd5f224cee8de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25dffb700 1 -- 172.21.15.45:0/3561641776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258018070 con 0x7fa260106280 2024-04-25T00:11:34.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25dffb700 1 -- 172.21.15.45:0/3561641776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258003a20 con 0x7fa260106280 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.386+0000 7fa25dffb700 1 -- 172.21.15.45:0/3561641776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258003c10 con 0x7fa260106280 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 -- 172.21.15.45:0/3561641776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 msgr2=0x7fa2601066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 --2- 172.21.15.45:0/3561641776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa25800af90 tx=0x7fa25800c330 comp rx=0 tx=0).stop 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 -- 172.21.15.45:0/3561641776 shutdown_connections 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 --2- 172.21.15.45:0/3561641776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa2601066a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 -- 172.21.15.45:0/3561641776 >> 172.21.15.45:0/3561641776 conn(0x7fa2601018e0 msgr2=0x7fa260103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 -- 172.21.15.45:0/3561641776 shutdown_connections 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.387+0000 7fa26537a700 1 -- 172.21.15.45:0/3561641776 wait complete. 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa26537a700 1 Processor -- start 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa26537a700 1 -- start start 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa26537a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa26537a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600681a0 con 0x7fa260106280 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa25effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa25effd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57754/0 (socket says 172.21.15.45:57754) 2024-04-25T00:11:34.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa25effd700 1 -- 172.21.15.45:0/925467522 learned_addr learned my addr 172.21.15.45:0/925467522 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa25effd700 1 -- 172.21.15.45:0/925467522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26019b910 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.388+0000 7fa25effd700 1 --2- 172.21.15.45:0/925467522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa258016040 tx=0x7fa258004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258023020 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258004160 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26019bba0 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa258018040 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26019e890 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa258026020 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.389+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa258010050 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.390+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa260063890 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.392+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa25802d030 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.461+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa26019ea60 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.464+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fa25801edb0 con 0x7fa260106280 2024-04-25T00:11:34.161 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa257fff700 1 -- 172.21.15.45:0/925467522 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa258003a20 con 0x7fa260106280 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 msgr2=0x7fa26019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 --2- 172.21.15.45:0/925467522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa258016040 tx=0x7fa258004ad0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 shutdown_connections 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 --2- 172.21.15.45:0/925467522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa260106280 0x7fa26019b3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 >> 172.21.15.45:0/925467522 conn(0x7fa2601018e0 msgr2=0x7fa260191a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 shutdown_connections 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:33.465+0000 7fa26537a700 1 -- 172.21.15.45:0/925467522 wait complete. 2024-04-25T00:11:34.162 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-04-25T00:11:35.120 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:34 smithi045 ceph-mon[28821]: from='client.? 172.21.15.45:0/1556815086' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.482+0000 7fe3eeda8700 1 Processor -- start 2024-04-25T00:11:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3eeda8700 1 -- start start 2024-04-25T00:11:35.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3eeda8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3eeda8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8108660 con 0x7fe3e8107d00 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3ecb44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3ecb44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47230/0 (socket says 172.21.15.45:47230) 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3ecb44700 1 -- 172.21.15.45:0/4054051614 learned_addr learned my addr 172.21.15.45:0/4054051614 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3ecb44700 1 -- 172.21.15.45:0/4054051614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81087e0 con 0x7fe3e8107d00 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.483+0000 7fe3ecb44700 1 --2- 172.21.15.45:0/4054051614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0098a0 tx=0x7fe3dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66736746ab369f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3e77fe700 1 -- 172.21.15.45:0/4054051614 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc004160 con 0x7fe3e8107d00 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3e77fe700 1 -- 172.21.15.45:0/4054051614 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3dc004300 con 0x7fe3e8107d00 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3e77fe700 1 -- 172.21.15.45:0/4054051614 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc011b50 con 0x7fe3e8107d00 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 -- 172.21.15.45:0/4054051614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 msgr2=0x7fe3e8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 --2- 172.21.15.45:0/4054051614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0098a0 tx=0x7fe3dc009bb0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 -- 172.21.15.45:0/4054051614 shutdown_connections 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 --2- 172.21.15.45:0/4054051614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e8108120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 -- 172.21.15.45:0/4054051614 >> 172.21.15.45:0/4054051614 conn(0x7fe3e81033a0 msgr2=0x7fe3e81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 -- 172.21.15.45:0/4054051614 shutdown_connections 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.484+0000 7fe3eeda8700 1 -- 172.21.15.45:0/4054051614 wait complete. 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3eeda8700 1 Processor -- start 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3eeda8700 1 -- start start 2024-04-25T00:11:35.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3eeda8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3eeda8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e810c120 con 0x7fe3e8107d00 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3ecb44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3ecb44700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47234/0 (socket says 172.21.15.45:47234) 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.485+0000 7fe3ecb44700 1 -- 172.21.15.45:0/1556815086 learned_addr learned my addr 172.21.15.45:0/1556815086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3ecb44700 1 -- 172.21.15.45:0/1556815086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e807e0b0 con 0x7fe3e8107d00 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3ecb44700 1 --2- 172.21.15.45:0/1556815086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc011b60 tx=0x7fe3dc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc003b20 con 0x7fe3e8107d00 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3dc003cc0 con 0x7fe3e8107d00 2024-04-25T00:11:35.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e807c2a0 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc024840 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e807c7e0 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3dc022020 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.486+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3dc011e00 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.487+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e8063890 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.490+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe3dc043dd0 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.560+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe3e807ccc0 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.560+0000 7fe3e5ffb700 1 -- 172.21.15.45:0/1556815086 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fe3dc032030 con 0x7fe3e8107d00 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 msgr2=0x7fe3e807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 --2- 172.21.15.45:0/1556815086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc011b60 tx=0x7fe3dc011c40 comp rx=0 tx=0).stop 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 shutdown_connections 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 --2- 172.21.15.45:0/1556815086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8107d00 0x7fe3e807db70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:35.137 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 >> 172.21.15.45:0/1556815086 conn(0x7fe3e81033a0 msgr2=0x7fe3e8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:35.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 shutdown_connections 2024-04-25T00:11:35.138 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:34.561+0000 7fe3eeda8700 1 -- 172.21.15.45:0/1556815086 wait complete. 2024-04-25T00:11:35.138 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-04-25T00:11:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 systemd[1]: Stopping Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:11:35.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a[28817]: 2024-04-25T00:11:35.431+0000 7f22643bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:11:35.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a[28817]: 2024-04-25T00:11:35.431+0000 7f22643bb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T00:11:35.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 podman[29067]: 2024-04-25 00:11:35.636803952 +0000 UTC m=+0.314004536 container died f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:11:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 podman[29067]: 2024-04-25 00:11:35.965108895 +0000 UTC m=+0.642309487 container cleanup f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:11:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:35 smithi045 bash[29067]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:11:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 podman[29081]: 2024-04-25 00:11:36.09101369 +0000 UTC m=+0.453458893 container remove f7dc8ccd4c6a43884d5001286b4a92cd675a589b391cf5bd90e429255e2768f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:11:36.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T00:11:36.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 systemd[1]: Stopped Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:11:36.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 systemd[1]: Starting Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:11:36.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 podman[29175]: 2024-04-25 00:11:36.508513768 +0000 UTC m=+0.111511549 container create 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:11:36.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 podman[29175]: 2024-04-25 00:11:36.415513291 +0000 UTC m=+0.018511036 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:11:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T00:11:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 podman[29175]: 2024-04-25 00:11:36.70292884 +0000 UTC m=+0.305926586 container init 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T00:11:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 podman[29175]: 2024-04-25 00:11:36.70621459 +0000 UTC m=+0.309212326 container start 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:11:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: pidfile_write: ignore empty --pid-file 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: load: jerasure load: lrc 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Git sha 0 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: DB SUMMARY 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: DB Session ID: NID2MMKRWTHLFX9ZKT7V 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104718 ; 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:11:36.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.env: 0x5635126d8c60 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.info_log: 0x5635144b1aa0 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:11:36.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.statistics: (nil) 2024-04-25T00:11:36.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.use_fsync: 0 2024-04-25T00:11:36.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.db_log_dir: 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.wal_dir: 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:11:36.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.write_buffer_manager: 0x5635144841e0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:11:36.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.unordered_write: 0 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.row_cache: None 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.wal_filter: None 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:11:36.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.wal_compression: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:11:36.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_open_files: -1 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:11:36.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Compression algorithms supported: 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kZSTD supported: 0 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:11:36.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.merge_operator: 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_filter: None 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5635144b1ba0) 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:11:36.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x5635144b5090 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-04-25T00:11:36.910 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:11:36.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression: NoCompression 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.num_levels: 7 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:11:36.912 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:11:36.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:11:36.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:11:36.915 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.ttl: 2592000 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:11:36.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d357ad8a-0f5a-4cc8-8671-b6f4a3f7c04e 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003896735497, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003896736683, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99807, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97719, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90058, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003896, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d357ad8a-0f5a-4cc8-8671-b6f4a3f7c04e", "db_session_id": "NID2MMKRWTHLFX9ZKT7V", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003896736769, "job": 1, "event": "recovery_finished"} 2024-04-25T00:11:36.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5635145a6000 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: DB pointer 0x563514590000 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:11:36.918 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 99.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 99.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.919 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 29.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 29.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x5635144b5090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:11:36.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???) e1 preinit fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).mds e1 new map 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).mds e1 print_map 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T00:11:36.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 bash[29175]: 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 systemd[1]: Started Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: monmap epoch 1 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: last_changed 2024-04-25T00:11:28.889760+0000 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: min_mon_release 19 (squid) 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: election_strategy: 1 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: fsmap 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:11:36.922 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:36 smithi045 ceph-mon[29194]: mgrmap e1: no daemons active 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.321+0000 7f615ba6f700 1 Processor -- start 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.321+0000 7f615ba6f700 1 -- start start 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.321+0000 7f615ba6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.321+0000 7f615ba6f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f615407beb0 con 0x7f6154108690 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f615980b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f615980b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47246/0 (socket says 172.21.15.45:47246) 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f615980b700 1 -- 172.21.15.45:0/3813607073 learned_addr learned my addr 172.21.15.45:0/3813607073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f615980b700 1 -- 172.21.15.45:0/3813607073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615410afc0 con 0x7f6154108690 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f615980b700 1 --2- 172.21.15.45:0/3813607073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f614c009e80 tx=0x7f614c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25837cac20ceb77f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f6158809700 1 -- 172.21.15.45:0/3813607073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c017070 con 0x7f6154108690 2024-04-25T00:11:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f6158809700 1 -- 172.21.15.45:0/3813607073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f614c00b8c0 con 0x7f6154108690 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.322+0000 7f6158809700 1 -- 172.21.15.45:0/3813607073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c0045c0 con 0x7f6154108690 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 -- 172.21.15.45:0/3813607073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 msgr2=0x7f615410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 --2- 172.21.15.45:0/3813607073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f614c009e80 tx=0x7f614c00b330 comp rx=0 tx=0).stop 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 -- 172.21.15.45:0/3813607073 shutdown_connections 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 --2- 172.21.15.45:0/3813607073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615410aa80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 -- 172.21.15.45:0/3813607073 >> 172.21.15.45:0/3813607073 conn(0x7f6154102100 msgr2=0x7f6154104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 -- 172.21.15.45:0/3813607073 shutdown_connections 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.323+0000 7f615ba6f700 1 -- 172.21.15.45:0/3813607073 wait complete. 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615ba6f700 1 Processor -- start 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615ba6f700 1 -- start start 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615ba6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615ba6f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f615410bff0 con 0x7f6154108690 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615980b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:37.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615980b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47258/0 (socket says 172.21.15.45:47258) 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615980b700 1 -- 172.21.15.45:0/4289124312 learned_addr learned my addr 172.21.15.45:0/4289124312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615980b700 1 -- 172.21.15.45:0/4289124312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615419c0b0 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.324+0000 7f615980b700 1 --2- 172.21.15.45:0/4289124312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f614c0072a0 tx=0x7f614c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c020020 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f614c003b50 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615419c340 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c017040 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615419f030 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f614c023020 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.325+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f614c00f050 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.326+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6154063890 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.328+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f614c02a020 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.399+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f615419f200 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.400+0000 7f6152ffd700 1 -- 172.21.15.45:0/4289124312 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f614c003d00 con 0x7f6154108690 2024-04-25T00:11:37.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 msgr2=0x7f615419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 --2- 172.21.15.45:0/4289124312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f614c0072a0 tx=0x7f614c00bd50 comp rx=0 tx=0).stop 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 shutdown_connections 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 --2- 172.21.15.45:0/4289124312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6154108690 0x7f615419bb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 >> 172.21.15.45:0/4289124312 conn(0x7f6154102100 msgr2=0x7f6154102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 shutdown_connections 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:37.401+0000 7f615ba6f700 1 -- 172.21.15.45:0/4289124312 wait complete. 2024-04-25T00:11:37.928 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T00:11:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:11:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-04-25T00:11:37.943 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T00:11:37.944 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T00:11:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:11:38.214 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T00:11:38.369 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968.target.wants/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@.service. 2024-04-25T00:11:38.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:38 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4289124312' entity='client.admin' 2024-04-25T00:11:38.881 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:38 smithi045 podman[29410]: 2024-04-25 00:11:38.826666217 +0000 UTC m=+0.330462888 container start 69afc426db4913fe6747be993bdcc3eb17c4ca8fedd549436ba6e9f2869dd866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:11:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:11:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T00:11:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:11:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:11:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T00:11:38.904 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:11:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:11:38.913 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T00:11:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:11:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:11:38.920 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T00:11:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T00:11:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-04-25T00:11:38.921 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-04-25T00:11:39.136 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:38 smithi045 bash[29410]: 69afc426db4913fe6747be993bdcc3eb17c4ca8fedd549436ba6e9f2869dd866 2024-04-25T00:11:39.136 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:38 smithi045 systemd[1]: Started Ceph mgr.x for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:11:39.136 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:38 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:38.983+0000 7fb5bc732200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:11:39.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:39 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:39.180+0000 7fb5bc732200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:11:39.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:39 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:39.287+0000 7fb5bc732200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:11:39.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:39 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1563760769' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:11:39.731 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:39 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:39.641+0000 7fb5bc732200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:11:39.731 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:39 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:39.730+0000 7fb5bc732200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:11:40.011 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:40.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:40.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:11:40.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:40.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:40.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:11:31.854393+0000", 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.348+0000 7f045f5ac700 1 Processor -- start 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.348+0000 7f045f5ac700 1 -- start start 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.348+0000 7f045f5ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.348+0000 7f045f5ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f045807be50 con 0x7f0458108700 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f045d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f045d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47278/0 (socket says 172.21.15.45:47278) 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f045d348700 1 -- 172.21.15.45:0/1372878658 learned_addr learned my addr 172.21.15.45:0/1372878658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f045d348700 1 -- 172.21.15.45:0/1372878658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045810b030 con 0x7f0458108700 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f045d348700 1 --2- 172.21.15.45:0/1372878658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66a6f51f1be9021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f044ffff700 1 -- 172.21.15.45:0/1372878658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454004160 con 0x7f0458108700 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f044ffff700 1 -- 172.21.15.45:0/1372878658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0454004300 con 0x7f0458108700 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.349+0000 7f044ffff700 1 -- 172.21.15.45:0/1372878658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454011b50 con 0x7f0458108700 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 -- 172.21.15.45:0/1372878658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 msgr2=0x7f045810aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 --2- 172.21.15.45:0/1372878658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).stop 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 -- 172.21.15.45:0/1372878658 shutdown_connections 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 --2- 172.21.15.45:0/1372878658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045810aaf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 -- 172.21.15.45:0/1372878658 >> 172.21.15.45:0/1372878658 conn(0x7f04581020b0 msgr2=0x7f0458104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 -- 172.21.15.45:0/1372878658 shutdown_connections 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.350+0000 7f045f5ac700 1 -- 172.21.15.45:0/1372878658 wait complete. 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045f5ac700 1 Processor -- start 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045f5ac700 1 -- start start 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045f5ac700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:40.027 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045f5ac700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f045810c000 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045d348700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47292/0 (socket says 172.21.15.45:47292) 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.351+0000 7f045d348700 1 -- 172.21.15.45:0/1563760769 learned_addr learned my addr 172.21.15.45:0/1563760769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f045d348700 1 -- 172.21.15.45:0/1563760769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045819e3c0 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f045d348700 1 --2- 172.21.15.45:0/1563760769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0454009fb0 tx=0x7f0454011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04540038b0 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0454011e50 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045819e650 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454019420 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a1340 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0454019ab0 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.352+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0454024730 con 0x7f0458108700 2024-04-25T00:11:40.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.353+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0458063890 con 0x7f0458108700 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.357+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0454043dd0 con 0x7f0458108700 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.451+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f04581a1880 con 0x7f0458108700 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.452+0000 7f044e7fc700 1 -- 172.21.15.45:0/1563760769 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0454032030 con 0x7f0458108700 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 msgr2=0x7f045819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 --2- 172.21.15.45:0/1563760769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0454009fb0 tx=0x7f0454011bf0 comp rx=0 tx=0).stop 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 shutdown_connections 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 --2- 172.21.15.45:0/1563760769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0458108700 0x7f045819de80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 >> 172.21.15.45:0/1563760769 conn(0x7f04581020b0 msgr2=0x7f0458102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 shutdown_connections 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:39.453+0000 7f045f5ac700 1 -- 172.21.15.45:0/1563760769 wait complete. 2024-04-25T00:11:40.029 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-04-25T00:11:41.339 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:40 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:40.988+0000 7fb5bc732200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:11:41.339 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:41.163+0000 7fb5bc732200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:11:41.339 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:41.338+0000 7fb5bc732200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:11:41.899 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:41.637+0000 7fb5bc732200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:11:41.899 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:41.732+0000 7fb5bc732200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:11:42.261 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:41.941+0000 7fb5bc732200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:11:42.261 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:42 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:42.146+0000 7fb5bc732200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:11:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:42 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3485774994' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:11:42.899 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:42 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:42.791+0000 7fb5bc732200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:11:43.537 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:43 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:43.230+0000 7fb5bc732200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:11:43.537 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:43 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:43.329+0000 7fb5bc732200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:11:43.537 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:43 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:43.536+0000 7fb5bc732200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:11:44.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.085+0000 7fb5bc732200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:11:44.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.180+0000 7fb5bc732200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:11:44.786 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.431+0000 7fb5bc732200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:11:44.787 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.569+0000 7fb5bc732200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:11:44.787 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.667+0000 7fb5bc732200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:11:44.787 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.786+0000 7fb5bc732200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:11:44.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:44.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:11:44.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:44.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:11:31.854393+0000", 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.471+0000 7fe99d398700 1 Processor -- start 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.471+0000 7fe99d398700 1 -- start start 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.471+0000 7fe99d398700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.471+0000 7fe99d398700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9981085c0 con 0x7fe998107c60 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe996ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe996ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47300/0 (socket says 172.21.15.45:47300) 2024-04-25T00:11:44.856 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe996ffd700 1 -- 172.21.15.45:0/4232652480 learned_addr learned my addr 172.21.15.45:0/4232652480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe996ffd700 1 -- 172.21.15.45:0/4232652480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe998108740 con 0x7fe998107c60 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe996ffd700 1 --2- 172.21.15.45:0/4232652480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe988009e80 tx=0x7fe98800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44398f4b2d36283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe995ffb700 1 -- 172.21.15.45:0/4232652480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe988017070 con 0x7fe998107c60 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe995ffb700 1 -- 172.21.15.45:0/4232652480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe98800b8c0 con 0x7fe998107c60 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.472+0000 7fe995ffb700 1 -- 172.21.15.45:0/4232652480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9880045c0 con 0x7fe998107c60 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 -- 172.21.15.45:0/4232652480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 msgr2=0x7fe998108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 --2- 172.21.15.45:0/4232652480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe988009e80 tx=0x7fe98800b330 comp rx=0 tx=0).stop 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 -- 172.21.15.45:0/4232652480 shutdown_connections 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 --2- 172.21.15.45:0/4232652480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe998108080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 -- 172.21.15.45:0/4232652480 >> 172.21.15.45:0/4232652480 conn(0x7fe998103280 msgr2=0x7fe9981056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 -- 172.21.15.45:0/4232652480 shutdown_connections 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.473+0000 7fe99d398700 1 -- 172.21.15.45:0/4232652480 wait complete. 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe99d398700 1 Processor -- start 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe99d398700 1 -- start start 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe99d398700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe99d398700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe99810c0d0 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe996ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe996ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47316/0 (socket says 172.21.15.45:47316) 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.474+0000 7fe996ffd700 1 -- 172.21.15.45:0/3485774994 learned_addr learned my addr 172.21.15.45:0/3485774994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe996ffd700 1 -- 172.21.15.45:0/3485774994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99807e060 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe996ffd700 1 --2- 172.21.15.45:0/3485774994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe988004350 tx=0x7fe98800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe988020020 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99807c2a0 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe988003c10 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe988017040 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99807c7e0 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe988023020 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.475+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe98800f050 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.476+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe998063890 con 0x7fe998107c60 2024-04-25T00:11:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.478+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe98802a020 con 0x7fe998107c60 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.552+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe99807ce70 con 0x7fe998107c60 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.552+0000 7fe97ffff700 1 -- 172.21.15.45:0/3485774994 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe98801ccd0 con 0x7fe998107c60 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 msgr2=0x7fe99807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 --2- 172.21.15.45:0/3485774994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe988004350 tx=0x7fe98800bd50 comp rx=0 tx=0).stop 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 shutdown_connections 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 --2- 172.21.15.45:0/3485774994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe998107c60 0x7fe99807db20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 >> 172.21.15.45:0/3485774994 conn(0x7fe998103280 msgr2=0x7fe998103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 shutdown_connections 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:42.553+0000 7fe99d398700 1 -- 172.21.15.45:0/3485774994 wait complete. 2024-04-25T00:11:44.859 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-04-25T00:11:45.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:44 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:44.881+0000 7fb5bc732200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:11:46.005 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:45 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:45.730+0000 7fb5bc732200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:11:46.005 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:45 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:45.952+0000 7fb5bc732200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:11:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: Activating manager daemon x 2024-04-25T00:11:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: mgrmap e2: x(active, starting, since 0.00335989s) 2024-04-25T00:11:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:11:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:11:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: Manager daemon x is now available 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' 2024-04-25T00:11:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:46 smithi045 ceph-mon[29194]: from='mgr.14100 172.21.15.45:0/680572349' entity='mgr.x' 2024-04-25T00:11:48.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:47 smithi045 ceph-mon[29194]: mgrmap e3: x(active, since 1.00698s) 2024-04-25T00:11:48.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1404086458' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:48.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:11:48.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:11:48.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:11:48.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:11:31.854393+0000", 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41f193700 1 Processor -- start 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41f193700 1 -- start start 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41f193700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41f193700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb418108ab0 con 0x7fb4181080c0 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41cf2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41cf2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40086/0 (socket says 172.21.15.45:40086) 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.331+0000 7fb41cf2f700 1 -- 172.21.15.45:0/921401440 learned_addr learned my addr 172.21.15.45:0/921401440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.332+0000 7fb41cf2f700 1 -- 172.21.15.45:0/921401440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418108c30 con 0x7fb4181080c0 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.332+0000 7fb41cf2f700 1 --2- 172.21.15.45:0/921401440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098a0 tx=0x7fb40c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f57ebcd2d15cab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.332+0000 7fb4177fe700 1 -- 172.21.15.45:0/921401440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40c004160 con 0x7fb4181080c0 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.332+0000 7fb4177fe700 1 -- 172.21.15.45:0/921401440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb40c004300 con 0x7fb4181080c0 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.332+0000 7fb4177fe700 1 -- 172.21.15.45:0/921401440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40c011b50 con 0x7fb4181080c0 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- 172.21.15.45:0/921401440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 msgr2=0x7fb4181084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 --2- 172.21.15.45:0/921401440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098a0 tx=0x7fb40c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- 172.21.15.45:0/921401440 shutdown_connections 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 --2- 172.21.15.45:0/921401440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb4181084e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- 172.21.15.45:0/921401440 >> 172.21.15.45:0/921401440 conn(0x7fb418101850 msgr2=0x7fb418103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- 172.21.15.45:0/921401440 shutdown_connections 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- 172.21.15.45:0/921401440 wait complete. 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 Processor -- start 2024-04-25T00:11:48.385 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.333+0000 7fb41f193700 1 -- start start 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41f193700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41f193700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb418106a80 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41cf2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41cf2f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40102/0 (socket says 172.21.15.45:40102) 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41cf2f700 1 -- 172.21.15.45:0/1404086458 learned_addr learned my addr 172.21.15.45:0/1404086458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41cf2f700 1 -- 172.21.15.45:0/1404086458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819e300 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41cf2f700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009fb0 tx=0x7fb40c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40c0038b0 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb40c011e50 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819e5f0 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.334+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb40c0193c0 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.335+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a1280 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.335+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fb40c01a430 con 0x7fb4181080c0 2024-04-25T00:11:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.336+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418063890 con 0x7fb4181080c0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.336+0000 7fb415ffb700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 0x7fb40803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.336+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb40c04cf70 con 0x7fb4181080c0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.336+0000 7fb417fff700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 0x7fb40803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.336+0000 7fb417fff700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 0x7fb40803b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb404006fd0 tx=0x7fb404006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.341+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb40c003b00 con 0x7fb4181080c0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.637+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb418197320 con 0x7fb4181080c0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.637+0000 7fb415ffb700 1 -- 172.21.15.45:0/1404086458 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb418197320 con 0x7fb4181080c0 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 msgr2=0x7fb40803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 0x7fb40803b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb404006fd0 tx=0x7fb404006e40 comp rx=0 tx=0).stop 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 msgr2=0x7fb41819ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009fb0 tx=0x7fb40c011bf0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 shutdown_connections 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7fb408038ef0 0x7fb40803b3b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 --2- 172.21.15.45:0/1404086458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4181080c0 0x7fb41819ddc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:48.387 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 >> 172.21.15.45:0/1404086458 conn(0x7fb418101850 msgr2=0x7fb418194400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:48.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.639+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 shutdown_connections 2024-04-25T00:11:48.388 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:47.640+0000 7fb41f193700 1 -- 172.21.15.45:0/1404086458 wait complete. 2024-04-25T00:11:48.388 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-04-25T00:11:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:48 smithi045 ceph-mon[29194]: mgrmap e4: x(active, since 2s) 2024-04-25T00:11:49.575 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:11:49.576 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.794+0000 7f6878adc700 1 Processor -- start 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.794+0000 7f6878adc700 1 -- start start 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.794+0000 7f6878adc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.794+0000 7f6878adc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68741068b0 con 0x7f6874105ec0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40104/0 (socket says 172.21.15.45:40104) 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687259c700 1 -- 172.21.15.45:0/4161413364 learned_addr learned my addr 172.21.15.45:0/4161413364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687259c700 1 -- 172.21.15.45:0/4161413364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6874108ea0 con 0x7f6874105ec0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687259c700 1 --2- 172.21.15.45:0/4161413364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f686c009e80 tx=0x7f686c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d5be2ef81ec894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687159a700 1 -- 172.21.15.45:0/4161413364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f686c017070 con 0x7f6874105ec0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687159a700 1 -- 172.21.15.45:0/4161413364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f686c00b8c0 con 0x7f6874105ec0 2024-04-25T00:11:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.795+0000 7f687159a700 1 -- 172.21.15.45:0/4161413364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f686c0045c0 con 0x7f6874105ec0 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 -- 172.21.15.45:0/4161413364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 msgr2=0x7f68741062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 --2- 172.21.15.45:0/4161413364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f686c009e80 tx=0x7f686c00b330 comp rx=0 tx=0).stop 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 -- 172.21.15.45:0/4161413364 shutdown_connections 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 --2- 172.21.15.45:0/4161413364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f68741062e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 -- 172.21.15.45:0/4161413364 >> 172.21.15.45:0/4161413364 conn(0x7f68741018e0 msgr2=0x7f6874103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 -- 172.21.15.45:0/4161413364 shutdown_connections 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.796+0000 7f6878adc700 1 -- 172.21.15.45:0/4161413364 wait complete. 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f6878adc700 1 Processor -- start 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f6878adc700 1 -- start start 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f6878adc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f6878adc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f687410c250 con 0x7f6874105ec0 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f687259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f687259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40116/0 (socket says 172.21.15.45:40116) 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f687259c700 1 -- 172.21.15.45:0/864791311 learned_addr learned my addr 172.21.15.45:0/864791311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:49.578 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.797+0000 7f687259c700 1 -- 172.21.15.45:0/864791311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687419e2e0 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f687259c700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f686c0072a0 tx=0x7f686c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f686c020020 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687419e570 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f686c003b50 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f686c017040 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.798+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68741a1260 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.799+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f686c023020 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.799+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6874063890 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.799+0000 7f686b7fe700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 0x7f686003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.799+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f686c00f050 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.799+0000 7f6871d9b700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 0x7f686003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.800+0000 7f6871d9b700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 0x7f686003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f685c006fd0 tx=0x7f685c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:48.804+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f686c02a020 con 0x7f6874105ec0 2024-04-25T00:11:49.579 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.012+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f68741a2ad0 con 0x7f6874105ec0 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.013+0000 7f686b7fe700 1 -- 172.21.15.45:0/864791311 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f686c01b730 con 0x7f6874105ec0 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 msgr2=0x7f686003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 0x7f686003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f685c006fd0 tx=0x7f685c006e40 comp rx=0 tx=0).stop 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 msgr2=0x7f687419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f686c0072a0 tx=0x7f686c00bd50 comp rx=0 tx=0).stop 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 shutdown_connections 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f6860038fb0 0x7f686003b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 --2- 172.21.15.45:0/864791311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6874105ec0 0x7f687419dda0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 >> 172.21.15.45:0/864791311 conn(0x7f68741018e0 msgr2=0x7f68741023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.015+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 shutdown_connections 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.016+0000 7f6878adc700 1 -- 172.21.15.45:0/864791311 wait complete. 2024-04-25T00:11:49.580 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-04-25T00:11:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:49 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/864791311' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:11:51.255 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:50 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1245317596' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T00:11:51.256 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setuser ceph since I am not root 2024-04-25T00:11:51.256 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setgroup ceph since I am not root 2024-04-25T00:11:51.256 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:51.145+0000 7fec6992a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:11:51.484 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.985+0000 7f109afbb700 1 Processor -- start 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.985+0000 7f109afbb700 1 -- start start 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.985+0000 7f109afbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.985+0000 7f109afbb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10941068b0 con 0x7f1094105ec0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f1098d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f1098d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40124/0 (socket says 172.21.15.45:40124) 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f1098d57700 1 -- 172.21.15.45:0/3416231874 learned_addr learned my addr 172.21.15.45:0/3416231874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f1098d57700 1 -- 172.21.15.45:0/3416231874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094108ea0 con 0x7f1094105ec0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f1098d57700 1 --2- 172.21.15.45:0/3416231874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f108c009e80 tx=0x7f108c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a650d75571a67c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f10937fe700 1 -- 172.21.15.45:0/3416231874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f108c017070 con 0x7f1094105ec0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f10937fe700 1 -- 172.21.15.45:0/3416231874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f108c00b8c0 con 0x7f1094105ec0 2024-04-25T00:11:51.485 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.986+0000 7f10937fe700 1 -- 172.21.15.45:0/3416231874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f108c0045c0 con 0x7f1094105ec0 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 -- 172.21.15.45:0/3416231874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 msgr2=0x7f10941062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 --2- 172.21.15.45:0/3416231874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f108c009e80 tx=0x7f108c00b330 comp rx=0 tx=0).stop 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 -- 172.21.15.45:0/3416231874 shutdown_connections 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 --2- 172.21.15.45:0/3416231874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f10941062e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 -- 172.21.15.45:0/3416231874 >> 172.21.15.45:0/3416231874 conn(0x7f10941018e0 msgr2=0x7f1094103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 -- 172.21.15.45:0/3416231874 shutdown_connections 2024-04-25T00:11:51.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.987+0000 7f109afbb700 1 -- 172.21.15.45:0/3416231874 wait complete. 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f109afbb700 1 Processor -- start 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f109afbb700 1 -- start start 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f109afbb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f109afbb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f109410c250 con 0x7f1094105ec0 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1098d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1098d57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40126/0 (socket says 172.21.15.45:40126) 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1098d57700 1 -- 172.21.15.45:0/1245317596 learned_addr learned my addr 172.21.15.45:0/1245317596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1098d57700 1 -- 172.21.15.45:0/1245317596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094108c60 con 0x7f1094105ec0 2024-04-25T00:11:51.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1098d57700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f108c0072a0 tx=0x7f108c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.988+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f108c020020 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.989+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f108c003b50 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.989+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1094106e50 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.989+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f108c017040 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.989+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1094107390 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f108c023020 con 0x7f1094105ec0 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f1091ffb700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 0x7f107c03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:51.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f108c00f050 con 0x7f1094105ec0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1094063890 con 0x7f1094105ec0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f1093fff700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 0x7f107c03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.990+0000 7f1093fff700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 0x7f107c03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1088006fd0 tx=0x7f1088006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:49.995+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f108c02a020 con 0x7f1094105ec0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.265+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1094107560 con 0x7f1094105ec0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.964+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f108c01b730 con 0x7f1094105ec0 2024-04-25T00:11:51.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.966+0000 7f1091ffb700 1 -- 172.21.15.45:0/1245317596 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f108c023d80 con 0x7f1094105ec0 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.968+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 msgr2=0x7f107c03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.968+0000 7f109afbb700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 0x7f107c03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1088006fd0 tx=0x7f1088006e40 comp rx=0 tx=0).stop 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.968+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 msgr2=0x7f1094108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.968+0000 7f109afbb700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f108c0072a0 tx=0x7f108c00bd50 comp rx=0 tx=0).stop 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.968+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 shutdown_connections 2024-04-25T00:11:51.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.969+0000 7f109afbb700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f107c038fb0 0x7f107c03b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.969+0000 7f109afbb700 1 --2- 172.21.15.45:0/1245317596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1094105ec0 0x7f1094108720 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:51.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.969+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 >> 172.21.15.45:0/1245317596 conn(0x7f10941018e0 msgr2=0x7f1094102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:51.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.969+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 shutdown_connections 2024-04-25T00:11:51.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:50.969+0000 7f109afbb700 1 -- 172.21.15.45:0/1245317596 wait complete. 2024-04-25T00:11:51.601 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:51.336+0000 7fec6992a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:11:51.601 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:51.442+0000 7fec6992a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:11:51.858 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:51.769+0000 7fec6992a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:11:51.859 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:51.858+0000 7fec6992a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:11:52.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1245317596' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T00:11:52.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:51 smithi045 ceph-mon[29194]: mgrmap e5: x(active, since 5s) 2024-04-25T00:11:53.456 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/242317590' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:11:53.456 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:53.155+0000 7fec6992a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:11:53.456 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:53.340+0000 7fec6992a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:11:53.797 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:53.529+0000 7fec6992a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:11:54.135 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:53.828+0000 7fec6992a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:11:54.135 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:53.924+0000 7fec6992a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:11:54.136 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:54.134+0000 7fec6992a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:11:54.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:54.334+0000 7fec6992a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:11:55.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:54.960+0000 7fec6992a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:11:55.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:55.399+0000 7fec6992a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:11:55.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:55.498+0000 7fec6992a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:11:55.708 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:55.707+0000 7fec6992a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:11:56.510 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.254+0000 7fec6992a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:11:56.510 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.348+0000 7fec6992a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:11:56.837 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.600+0000 7fec6992a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:11:56.838 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.739+0000 7fec6992a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:11:56.838 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.837+0000 7fec6992a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:11:57.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:56.956+0000 7fec6992a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:11:57.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:57.051+0000 7fec6992a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:11:58.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:57.897+0000 7fec6992a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:11:58.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:11:58.118+0000 7fec6992a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:11:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: Active manager daemon x restarted 2024-04-25T00:11:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: Activating manager daemon x 2024-04-25T00:11:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T00:11:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: mgrmap e6: x(active, starting, since 0.00333046s) 2024-04-25T00:11:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: Manager daemon x is now available 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:58 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:11:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:11:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T00:11:59.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.946+0000 7f090f66b700 1 Processor -- start 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.946+0000 7f090f66b700 1 -- start start 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.946+0000 7f090f66b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.946+0000 7f090f66b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09081085e0 con 0x7f0908107c80 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.946+0000 7f090d407700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f090d407700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40158/0 (socket says 172.21.15.45:40158) 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f090d407700 1 -- 172.21.15.45:0/467794968 learned_addr learned my addr 172.21.15.45:0/467794968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f090d407700 1 -- 172.21.15.45:0/467794968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0908108760 con 0x7f0908107c80 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f090d407700 1 --2- 172.21.15.45:0/467794968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09040098a0 tx=0x7f0904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282ef5f6a570e945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f08fffff700 1 -- 172.21.15.45:0/467794968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904004160 con 0x7f0908107c80 2024-04-25T00:11:59.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f08fffff700 1 -- 172.21.15.45:0/467794968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0904004300 con 0x7f0908107c80 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.947+0000 7f08fffff700 1 -- 172.21.15.45:0/467794968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904011b50 con 0x7f0908107c80 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 -- 172.21.15.45:0/467794968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 msgr2=0x7f09081080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 --2- 172.21.15.45:0/467794968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09040098a0 tx=0x7f0904009bb0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 -- 172.21.15.45:0/467794968 shutdown_connections 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 --2- 172.21.15.45:0/467794968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f09081080a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 -- 172.21.15.45:0/467794968 >> 172.21.15.45:0/467794968 conn(0x7f09081032a0 msgr2=0x7f0908105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 -- 172.21.15.45:0/467794968 shutdown_connections 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.948+0000 7f090f66b700 1 -- 172.21.15.45:0/467794968 wait complete. 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090f66b700 1 Processor -- start 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090f66b700 1 -- start start 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090f66b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090f66b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f090810c0f0 con 0x7f0908107c80 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090d407700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:59.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090d407700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40170/0 (socket says 172.21.15.45:40170) 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.949+0000 7f090d407700 1 -- 172.21.15.45:0/242317590 learned_addr learned my addr 172.21.15.45:0/242317590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f090d407700 1 -- 172.21.15.45:0/242317590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819c100 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f090d407700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0904011b60 tx=0x7f0904011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904003b20 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0904003cc0 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819c390 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904024840 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.950+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090819f080 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0904022020 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f08fe7fc700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 0x7f08f403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0904011e00 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f090cc06700 1 -- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 msgr2=0x7f08f403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/200010672 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f090cc06700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 0x7f08f403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.951+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908063890 con 0x7f0908107c80 2024-04-25T00:11:59.044 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:52.957+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f09040172b0 con 0x7f0908107c80 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.152+0000 7f090cc06700 1 -- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 msgr2=0x7f08f403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/200010672 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.152+0000 7f090cc06700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 0x7f08f403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.248+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f090819f250 con 0x7f0908107c80 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.249+0000 7f08fe7fc700 1 -- 172.21.15.45:0/242317590 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f090400ba30 con 0x7f0908107c80 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 msgr2=0x7f08f403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 0x7f08f403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 msgr2=0x7f090819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0904011b60 tx=0x7f0904011c40 comp rx=0 tx=0).stop 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 shutdown_connections 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:6800/200010672,v1:172.21.15.45:6801/200010672] conn(0x7f08f4039050 0x7f08f403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 --2- 172.21.15.45:0/242317590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908107c80 0x7f090819bbc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 >> 172.21.15.45:0/242317590 conn(0x7f09081032a0 msgr2=0x7f0908103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 shutdown_connections 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:53.251+0000 7f090f66b700 1 -- 172.21.15.45:0/242317590 wait complete. 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-25T00:11:59.045 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-04-25T00:11:59.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:59 smithi045 ceph-mon[29194]: Found migration_current of "None". Setting to last migration. 2024-04-25T00:11:59.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:59 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:11:59.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:59 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:11:59.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:59 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:11:59.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:11:59 smithi045 ceph-mon[29194]: mgrmap e7: x(active, since 1.00588s) 2024-04-25T00:12:00.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: [25/Apr/2024:00:11:59] ENGINE Bus STARTING 2024-04-25T00:12:00.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: [25/Apr/2024:00:11:59] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T00:12:00.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: [25/Apr/2024:00:11:59] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T00:12:00.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: [25/Apr/2024:00:11:59] ENGINE Bus STARTED 2024-04-25T00:12:00.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:00.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:12:00.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:00 smithi045 ceph-mon[29194]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:12:00.261 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe68a0a0700 1 Processor -- start 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe68a0a0700 1 -- start start 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe68a0a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe68a0a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe68407beb0 con 0x7fe684108710 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe6837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.262 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe6837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33758/0 (socket says 172.21.15.45:33758) 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.684+0000 7fe6837fe700 1 -- 172.21.15.45:0/2533312343 learned_addr learned my addr 172.21.15.45:0/2533312343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.685+0000 7fe6837fe700 1 -- 172.21.15.45:0/2533312343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68410b040 con 0x7fe684108710 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.685+0000 7fe6837fe700 1 --2- 172.21.15.45:0/2533312343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe6740098a0 tx=0x7fe674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a0f742e71076f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.685+0000 7fe6827fc700 1 -- 172.21.15.45:0/2533312343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe674004160 con 0x7fe684108710 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.685+0000 7fe6827fc700 1 -- 172.21.15.45:0/2533312343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe674004300 con 0x7fe684108710 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.685+0000 7fe6827fc700 1 -- 172.21.15.45:0/2533312343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe674011b50 con 0x7fe684108710 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 -- 172.21.15.45:0/2533312343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 msgr2=0x7fe68410ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/2533312343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe6740098a0 tx=0x7fe674009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 -- 172.21.15.45:0/2533312343 shutdown_connections 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/2533312343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68410ab00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.263 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 -- 172.21.15.45:0/2533312343 >> 172.21.15.45:0/2533312343 conn(0x7fe684102180 msgr2=0x7fe6841045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 -- 172.21.15.45:0/2533312343 shutdown_connections 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 -- 172.21.15.45:0/2533312343 wait complete. 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.686+0000 7fe68a0a0700 1 Processor -- start 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe68a0a0700 1 -- start start 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe68a0a0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe68a0a0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe68410c070 con 0x7fe684108710 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe6837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe6837fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33766/0 (socket says 172.21.15.45:33766) 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe6837fe700 1 -- 172.21.15.45:0/3201363623 learned_addr learned my addr 172.21.15.45:0/3201363623 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe6837fe700 1 -- 172.21.15.45:0/3201363623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68419c150 con 0x7fe684108710 2024-04-25T00:12:00.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe6837fe700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe674011670 tx=0x7fe674011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6740039c0 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68419c3e0 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.687+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe674011e50 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.688+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe67401a5c0 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.688+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68419f0d0 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.688+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fe67401a8c0 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe680ff9700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 0x7fe66c03b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe66c03bac0 con 0x7fe66c038ef0 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe67404d400 con 0x7fe684108710 2024-04-25T00:12:00.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe682ffd700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 0x7fe66c03b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe682ffd700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 0x7fe66c03b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.689+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fe66c03bac0 con 0x7fe66c038ef0 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe684063890 con 0x7fe66c038ef0 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe680ff9700 1 -- 172.21.15.45:0/3201363623 <== mgr.14118 v2:172.21.15.45:6800/1299966585 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe684063890 con 0x7fe66c038ef0 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 msgr2=0x7fe66c03b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 0x7fe66c03b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 msgr2=0x7fe68419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe674011670 tx=0x7fe674011b60 comp rx=0 tx=0).stop 2024-04-25T00:12:00.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 shutdown_connections 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe66c038ef0 0x7fe66c03b3b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 --2- 172.21.15.45:0/3201363623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe684108710 0x7fe68419bc10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 >> 172.21.15.45:0/3201363623 conn(0x7fe684102180 msgr2=0x7fe684102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 shutdown_connections 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:11:59.696+0000 7fe68a0a0700 1 -- 172.21.15.45:0/3201363623 wait complete. 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-04-25T00:12:00.267 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-04-25T00:12:01.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:01 smithi045 ceph-mon[29194]: mgrmap e8: x(active, since 2s) 2024-04-25T00:12:01.428 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:01 smithi045 ceph-mon[29194]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:01.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:01 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:01.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:01 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.627+0000 7f973e44c700 1 Processor -- start 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.627+0000 7f973e44c700 1 -- start start 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f973e44c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f973e44c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9738108660 con 0x7f9738107d00 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9737fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9737fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33770/0 (socket says 172.21.15.45:33770) 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9737fff700 1 -- 172.21.15.45:0/2762690681 learned_addr learned my addr 172.21.15.45:0/2762690681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9737fff700 1 -- 172.21.15.45:0/2762690681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97381087e0 con 0x7f9738107d00 2024-04-25T00:12:01.435 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9737fff700 1 --2- 172.21.15.45:0/2762690681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97200098a0 tx=0x7f9720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80eeac2c671a9883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.628+0000 7f9736ffd700 1 -- 172.21.15.45:0/2762690681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9720004160 con 0x7f9738107d00 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f9736ffd700 1 -- 172.21.15.45:0/2762690681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9720004300 con 0x7f9738107d00 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f9736ffd700 1 -- 172.21.15.45:0/2762690681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9720011b50 con 0x7f9738107d00 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 -- 172.21.15.45:0/2762690681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 msgr2=0x7f9738108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 --2- 172.21.15.45:0/2762690681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f97200098a0 tx=0x7f9720009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 -- 172.21.15.45:0/2762690681 shutdown_connections 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 --2- 172.21.15.45:0/2762690681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f9738108120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 -- 172.21.15.45:0/2762690681 >> 172.21.15.45:0/2762690681 conn(0x7f97381033a0 msgr2=0x7f97381057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 -- 172.21.15.45:0/2762690681 shutdown_connections 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.629+0000 7f973e44c700 1 -- 172.21.15.45:0/2762690681 wait complete. 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f973e44c700 1 Processor -- start 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f973e44c700 1 -- start start 2024-04-25T00:12:01.436 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f973e44c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f973e44c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f973810c120 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f9737fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f9737fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33786/0 (socket says 172.21.15.45:33786) 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.630+0000 7f9737fff700 1 -- 172.21.15.45:0/1735469434 learned_addr learned my addr 172.21.15.45:0/1735469434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f9737fff700 1 -- 172.21.15.45:0/1735469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973807e0b0 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f9737fff700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9720011670 tx=0x7f9720011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97200039c0 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9720011e50 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973807c2a0 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f972001a5c0 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.631+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973807c7e0 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.632+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f972001a870 con 0x7f9738107d00 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.632+0000 7f97357fa700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 0x7f972403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:01.437 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.632+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f972004d450 con 0x7f9738107d00 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.632+0000 7f97377fe700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 0x7f972403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.632+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9738063890 con 0x7f9738107d00 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.633+0000 7f97377fe700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 0x7f972403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9728006fd0 tx=0x7f9728006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.637+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f972002cc00 con 0x7f9738107d00 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.860+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f973807d110 con 0x7f9724039050 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.865+0000 7f97357fa700 1 -- 172.21.15.45:0/1735469434 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f973807d110 con 0x7f9724039050 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.867+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 msgr2=0x7f972403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.867+0000 7f973e44c700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 0x7f972403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9728006fd0 tx=0x7f9728006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.867+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 msgr2=0x7f973807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.867+0000 7f973e44c700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9720011670 tx=0x7f9720011b60 comp rx=0 tx=0).stop 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 shutdown_connections 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f9724039050 0x7f972403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 --2- 172.21.15.45:0/1735469434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9738107d00 0x7f973807db70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:01.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 >> 172.21.15.45:0/1735469434 conn(0x7f97381033a0 msgr2=0x7f9738103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:01.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 shutdown_connections 2024-04-25T00:12:01.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:00.868+0000 7f973e44c700 1 -- 172.21.15.45:0/1735469434 wait complete. 2024-04-25T00:12:02.658 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T00:12:02.658 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce711e700 1 Processor -- start 2024-04-25T00:12:02.658 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce711e700 1 -- start start 2024-04-25T00:12:02.658 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce711e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce711e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0108660 con 0x7f3ce0107d00 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce4eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce4eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33800/0 (socket says 172.21.15.45:33800) 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.867+0000 7f3ce4eba700 1 -- 172.21.15.45:0/3710275437 learned_addr learned my addr 172.21.15.45:0/3710275437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.868+0000 7f3ce4eba700 1 -- 172.21.15.45:0/3710275437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce01087e0 con 0x7f3ce0107d00 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.868+0000 7f3ce4eba700 1 --2- 172.21.15.45:0/3710275437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bff2bd4cb58974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.868+0000 7f3cdf7fe700 1 -- 172.21.15.45:0/3710275437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4004160 con 0x7f3ce0107d00 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.868+0000 7f3cdf7fe700 1 -- 172.21.15.45:0/3710275437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cd4004300 con 0x7f3ce0107d00 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.868+0000 7f3cdf7fe700 1 -- 172.21.15.45:0/3710275437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4011b50 con 0x7f3ce0107d00 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- 172.21.15.45:0/3710275437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 msgr2=0x7f3ce0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 --2- 172.21.15.45:0/3710275437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- 172.21.15.45:0/3710275437 shutdown_connections 2024-04-25T00:12:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 --2- 172.21.15.45:0/3710275437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce0108120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- 172.21.15.45:0/3710275437 >> 172.21.15.45:0/3710275437 conn(0x7f3ce01033a0 msgr2=0x7f3ce01057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- 172.21.15.45:0/3710275437 shutdown_connections 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- 172.21.15.45:0/3710275437 wait complete. 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 Processor -- start 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.869+0000 7f3ce711e700 1 -- start start 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce711e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce711e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce010c120 con 0x7f3ce0107d00 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce4eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce4eba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33812/0 (socket says 172.21.15.45:33812) 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce4eba700 1 -- 172.21.15.45:0/1444446623 learned_addr learned my addr 172.21.15.45:0/1444446623 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce4eba700 1 -- 172.21.15.45:0/1444446623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce019c110 con 0x7f3ce0107d00 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce4eba700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4011b60 tx=0x7f3cd4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4003b20 con 0x7f3ce0107d00 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cd4003cc0 con 0x7f3ce0107d00 2024-04-25T00:12:02.660 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce019c3a0 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.870+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4024840 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.871+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce019f090 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.871+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3cd4022020 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.872+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0063890 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.872+0000 7f3cddffb700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 0x7f3cd003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.872+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3cd4011e00 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.872+0000 7f3cdffff700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 0x7f3cd003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.872+0000 7f3cdffff700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 0x7f3cd003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:01.877+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3cd400b3e0 con 0x7f3ce0107d00 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.084+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3ce0104500 con 0x7f3cd00390a0 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.085+0000 7f3cddffb700 1 -- 172.21.15.45:0/1444446623 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3ce0104500 con 0x7f3cd00390a0 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 msgr2=0x7f3cd003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 0x7f3cd003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:02.661 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 msgr2=0x7f3ce019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4011b60 tx=0x7f3cd4011c40 comp rx=0 tx=0).stop 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 shutdown_connections 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3cd00390a0 0x7f3cd003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 --2- 172.21.15.45:0/1444446623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ce0107d00 0x7f3ce019bbd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 >> 172.21.15.45:0/1444446623 conn(0x7f3ce01033a0 msgr2=0x7f3ce0103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 shutdown_connections 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:02.087+0000 7f3ce711e700 1 -- 172.21.15.45:0/1444446623 wait complete. 2024-04-25T00:12:02.662 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-04-25T00:12:03.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-mon[29194]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:04.123 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: Generating public/private rsa key pair. 2024-04-25T00:12:04.123 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: Your identification has been saved in /tmp/tmpp816hf8u/key. 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: Your public key has been saved in /tmp/tmpp816hf8u/key.pub. 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: The key fingerprint is: 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: SHA256:RRvE9GjMFm0yPRXFmRtWwMnD34bnUh+mVK5S96fVq14 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: The key's randomart image is: 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: +---[RSA 3072]----+ 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | +*o =+**| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | ++*= **.| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | O+..+++| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | + o.**| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | S o ===| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | . o. B| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | . Eo| 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | o. | 2024-04-25T00:12:04.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: | .o. | 2024-04-25T00:12:04.125 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: +----[SHA256]-----+ 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.144+0000 7f32af919700 1 Processor -- start 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.144+0000 7f32af919700 1 -- start start 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32af919700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32af919700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32a807c8d0 con 0x7f32a807de50 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32ad6b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32ad6b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33814/0 (socket says 172.21.15.45:33814) 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32ad6b5700 1 -- 172.21.15.45:0/155391738 learned_addr learned my addr 172.21.15.45:0/155391738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:04.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32ad6b5700 1 -- 172.21.15.45:0/155391738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a807ca50 con 0x7f32a807de50 2024-04-25T00:12:04.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32ad6b5700 1 --2- 172.21.15.45:0/155391738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f329800b180 tx=0x7f329800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c005924a5b9cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.145+0000 7f32a7fff700 1 -- 172.21.15.45:0/155391738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298017070 con 0x7f32a807de50 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32a7fff700 1 -- 172.21.15.45:0/155391738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3298003a20 con 0x7f32a807de50 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32a7fff700 1 -- 172.21.15.45:0/155391738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298004820 con 0x7f32a807de50 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 -- 172.21.15.45:0/155391738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 msgr2=0x7f32a807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 --2- 172.21.15.45:0/155391738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f329800b180 tx=0x7f329800b490 comp rx=0 tx=0).stop 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 -- 172.21.15.45:0/155391738 shutdown_connections 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 --2- 172.21.15.45:0/155391738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a807c300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 -- 172.21.15.45:0/155391738 >> 172.21.15.45:0/155391738 conn(0x7f32a8103240 msgr2=0x7f32a8105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 -- 172.21.15.45:0/155391738 shutdown_connections 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.146+0000 7f32af919700 1 -- 172.21.15.45:0/155391738 wait complete. 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32af919700 1 Processor -- start 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32af919700 1 -- start start 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32af919700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.133 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32af919700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810e250 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32ad6b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32ad6b5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33818/0 (socket says 172.21.15.45:33818) 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32ad6b5700 1 -- 172.21.15.45:0/1099928832 learned_addr learned my addr 172.21.15.45:0/1099928832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.147+0000 7f32ad6b5700 1 -- 172.21.15.45:0/1099928832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a819e330 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32ad6b5700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32980045b0 tx=0x7f3298003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298020020 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329800be50 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298017040 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a819e5c0 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.148+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81a12b0 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3298023020 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a8063890 con 0x7f32a807de50 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32a67fc700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 0x7f329003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:04.134 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f329800f050 con 0x7f32a807de50 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32aceb4700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 0x7f329003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.149+0000 7f32aceb4700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 0x7f329003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.154+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f329802a020 con 0x7f32a807de50 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.362+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f32a8104480 con 0x7f3290039000 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.666+0000 7f32a67fc700 1 -- 172.21.15.45:0/1099928832 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32a8104480 con 0x7f3290039000 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 msgr2=0x7f329003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 0x7f329003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 msgr2=0x7f32a819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32980045b0 tx=0x7f3298003eb0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 shutdown_connections 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f3290039000 0x7f329003b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 --2- 172.21.15.45:0/1099928832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32a807de50 0x7f32a819ddf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 >> 172.21.15.45:0/1099928832 conn(0x7f32a8103240 msgr2=0x7f32a8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:04.135 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 shutdown_connections 2024-04-25T00:12:04.136 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:03.668+0000 7f32af919700 1 -- 172.21.15.45:0/1099928832 wait complete. 2024-04-25T00:12:05.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:04 smithi045 ceph-mon[29194]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:05.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:04 smithi045 ceph-mon[29194]: Generating ssh key... 2024-04-25T00:12:05.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:04 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:05.122 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:04 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:05.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:12:05.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7ddf77700 1 Processor -- start 2024-04-25T00:12:05.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7ddf77700 1 -- start start 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7ddf77700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7ddf77700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d807be50 con 0x7ff7d81086f0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36200/0 (socket says 172.21.15.45:36200) 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7d77fe700 1 -- 172.21.15.45:0/2172825088 learned_addr learned my addr 172.21.15.45:0/2172825088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7d77fe700 1 -- 172.21.15.45:0/2172825088 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d810b020 con 0x7ff7d81086f0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.582+0000 7ff7d77fe700 1 --2- 172.21.15.45:0/2172825088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098a0 tx=0x7ff7c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3c78b9594be21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7d67fc700 1 -- 172.21.15.45:0/2172825088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8004160 con 0x7ff7d81086f0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7d67fc700 1 -- 172.21.15.45:0/2172825088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7c8004300 con 0x7ff7d81086f0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7d67fc700 1 -- 172.21.15.45:0/2172825088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8011b50 con 0x7ff7d81086f0 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 -- 172.21.15.45:0/2172825088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 msgr2=0x7ff7d810aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/2172825088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098a0 tx=0x7ff7c8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 -- 172.21.15.45:0/2172825088 shutdown_connections 2024-04-25T00:12:05.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/2172825088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d810aae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 -- 172.21.15.45:0/2172825088 >> 172.21.15.45:0/2172825088 conn(0x7ff7d8102100 msgr2=0x7ff7d8104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 -- 172.21.15.45:0/2172825088 shutdown_connections 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.583+0000 7ff7ddf77700 1 -- 172.21.15.45:0/2172825088 wait complete. 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7ddf77700 1 Processor -- start 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7ddf77700 1 -- start start 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7ddf77700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7ddf77700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d810bff0 con 0x7ff7d81086f0 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7d77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36214/0 (socket says 172.21.15.45:36214) 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.584+0000 7ff7d77fe700 1 -- 172.21.15.45:0/767965047 learned_addr learned my addr 172.21.15.45:0/767965047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7d77fe700 1 -- 172.21.15.45:0/767965047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d819c0f0 con 0x7ff7d81086f0 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7d77fe700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8011670 tx=0x7ff7c8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c80039c0 con 0x7ff7d81086f0 2024-04-25T00:12:05.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7c8011e50 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d819c380 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c801a5c0 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.585+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d819f070 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.586+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff7c8024430 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.586+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d8063890 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.586+0000 7ff7d4ff9700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 0x7ff7c403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.586+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff7c804d520 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.586+0000 7ff7d6ffd700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 0x7ff7c403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.587+0000 7ff7d6ffd700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 0x7ff7c403b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.591+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7c802d910 con 0x7ff7d81086f0 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.798+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ff7d81032d0 con 0x7ff7c4039000 2024-04-25T00:12:05.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.799+0000 7ff7d4ff9700 1 -- 172.21.15.45:0/767965047 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ff7d81032d0 con 0x7ff7c4039000 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 msgr2=0x7ff7c403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 0x7ff7c403b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 msgr2=0x7ff7d819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8011670 tx=0x7ff7c8011b60 comp rx=0 tx=0).stop 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 shutdown_connections 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7ff7c4039000 0x7ff7c403b4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 --2- 172.21.15.45:0/767965047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7d81086f0 0x7ff7d819bbb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 >> 172.21.15.45:0/767965047 conn(0x7ff7d8102100 msgr2=0x7ff7d8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 shutdown_connections 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:04.801+0000 7ff7ddf77700 1 -- 172.21.15.45:0/767965047 wait complete. 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T00:12:05.375 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-04-25T00:12:05.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:05 smithi045 ceph-mon[29194]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:06 smithi045 ceph-mon[29194]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:08.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:07 smithi045 ceph-mon[29194]: Deploying cephadm binary to smithi045 2024-04-25T00:12:10.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feeff552700 1 Processor -- start 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feeff552700 1 -- start start 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feeff552700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feeff552700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feef807e140 con 0x7feef807dd20 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feefd2ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feefd2ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36228/0 (socket says 172.21.15.45:36228) 2024-04-25T00:12:10.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.787+0000 7feefd2ee700 1 -- 172.21.15.45:0/3624422737 learned_addr learned my addr 172.21.15.45:0/3624422737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:10.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feefd2ee700 1 -- 172.21.15.45:0/3624422737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef807c820 con 0x7feef807dd20 2024-04-25T00:12:10.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feefd2ee700 1 --2- 172.21.15.45:0/3624422737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feef40098a0 tx=0x7feef4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e73569cd2d180d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feeeffff700 1 -- 172.21.15.45:0/3624422737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef4004160 con 0x7feef807dd20 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feeeffff700 1 -- 172.21.15.45:0/3624422737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feef4004300 con 0x7feef807dd20 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feeeffff700 1 -- 172.21.15.45:0/3624422737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef4011b50 con 0x7feef807dd20 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feeff552700 1 -- 172.21.15.45:0/3624422737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 msgr2=0x7feef807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.788+0000 7feeff552700 1 --2- 172.21.15.45:0/3624422737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feef40098a0 tx=0x7feef4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 -- 172.21.15.45:0/3624422737 shutdown_connections 2024-04-25T00:12:10.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 --2- 172.21.15.45:0/3624422737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef807c2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 -- 172.21.15.45:0/3624422737 >> 172.21.15.45:0/3624422737 conn(0x7feef81033a0 msgr2=0x7feef81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 -- 172.21.15.45:0/3624422737 shutdown_connections 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 -- 172.21.15.45:0/3624422737 wait complete. 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 Processor -- start 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.789+0000 7feeff552700 1 -- start start 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeff552700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeff552700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feef810e190 con 0x7feef807dd20 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feefd2ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feefd2ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36238/0 (socket says 172.21.15.45:36238) 2024-04-25T00:12:10.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feefd2ee700 1 -- 172.21.15.45:0/1348210309 learned_addr learned my addr 172.21.15.45:0/1348210309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feefd2ee700 1 -- 172.21.15.45:0/1348210309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef819e320 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feefd2ee700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feef4011670 tx=0x7feef4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef40039c0 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feef4011e50 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef819e5b0 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef401a5c0 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.790+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef81a12a0 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.791+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7feef4024430 con 0x7feef807dd20 2024-04-25T00:12:10.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.791+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef8063890 con 0x7feef807dd20 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.791+0000 7feeee7fc700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 0x7feee403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.791+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feef404d380 con 0x7feef807dd20 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.792+0000 7feefcaed700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 0x7feee403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.792+0000 7feefcaed700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 0x7feee403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:05.797+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feef402d910 con 0x7feef807dd20 2024-04-25T00:12:10.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:06.007+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7feef81045a0 con 0x7feee4039050 2024-04-25T00:12:10.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.269+0000 7feeee7fc700 1 -- 172.21.15.45:0/1348210309 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7feef81045a0 con 0x7feee4039050 2024-04-25T00:12:10.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 msgr2=0x7feee403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 0x7feee403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee8006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:10.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 msgr2=0x7feef819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:10.832 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feef4011670 tx=0x7feef4011b60 comp rx=0 tx=0).stop 2024-04-25T00:12:10.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 shutdown_connections 2024-04-25T00:12:10.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7feee4039050 0x7feee403b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 --2- 172.21.15.45:0/1348210309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef807dd20 0x7feef819dde0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:10.833 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 >> 172.21.15.45:0/1348210309 conn(0x7feef81033a0 msgr2=0x7feef8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:10.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 shutdown_connections 2024-04-25T00:12:10.834 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:10.271+0000 7feeff552700 1 -- 172.21.15.45:0/1348210309 wait complete. 2024-04-25T00:12:10.834 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-04-25T00:12:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:11 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:11 smithi045 ceph-mon[29194]: Added host smithi045 2024-04-25T00:12:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:11 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:12.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T00:12:12.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.221+0000 7fe6dd6c3700 1 Processor -- start 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.221+0000 7fe6dd6c3700 1 -- start start 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.221+0000 7fe6dd6c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.221+0000 7fe6dd6c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8107570 con 0x7fe6d8106b80 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36244/0 (socket says 172.21.15.45:36244) 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d6ffd700 1 -- 172.21.15.45:0/2034141022 learned_addr learned my addr 172.21.15.45:0/2034141022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d6ffd700 1 -- 172.21.15.45:0/2034141022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8107d80 con 0x7fe6d8106b80 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d6ffd700 1 --2- 172.21.15.45:0/2034141022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80098a0 tx=0x7fe6c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19128701ba7167f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.222+0000 7fe6d5ffb700 1 -- 172.21.15.45:0/2034141022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c8004160 con 0x7fe6d8106b80 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6d5ffb700 1 -- 172.21.15.45:0/2034141022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6c8004300 con 0x7fe6d8106b80 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6d5ffb700 1 -- 172.21.15.45:0/2034141022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c8011b50 con 0x7fe6d8106b80 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2034141022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 msgr2=0x7fe6d8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2034141022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80098a0 tx=0x7fe6c8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2034141022 shutdown_connections 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2034141022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d8106fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2034141022 >> 172.21.15.45:0/2034141022 conn(0x7fe6d8102100 msgr2=0x7fe6d8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2034141022 shutdown_connections 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.223+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2034141022 wait complete. 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6dd6c3700 1 Processor -- start 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6dd6c3700 1 -- start start 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6dd6c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6dd6c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d810b130 con 0x7fe6d8106b80 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6d6ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36256/0 (socket says 172.21.15.45:36256) 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.224+0000 7fe6d6ffd700 1 -- 172.21.15.45:0/2210205034 learned_addr learned my addr 172.21.15.45:0/2210205034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:12.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6d6ffd700 1 -- 172.21.15.45:0/2210205034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d819c230 con 0x7fe6d8106b80 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6d6ffd700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009fb0 tx=0x7fe6c8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c80038b0 con 0x7fe6d8106b80 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6c8011e50 con 0x7fe6d8106b80 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d819c4c0 con 0x7fe6d8106b80 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6c8019550 con 0x7fe6d8106b80 2024-04-25T00:12:12.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.225+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d819f1a0 con 0x7fe6d8106b80 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.226+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe6c8019850 con 0x7fe6d8106b80 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.226+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d8063890 con 0x7fe6d8106b80 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.226+0000 7fe6bffff700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 0x7fe6c003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.226+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe6c804d4e0 con 0x7fe6d8106b80 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.226+0000 7fe6d67fc700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 0x7fe6c003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:12.186 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.227+0000 7fe6d67fc700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 0x7fe6c003b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006fd0 tx=0x7fe6cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.231+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe6c8029d50 con 0x7fe6d8106b80 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.462+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe6d81033e0 con 0x7fe6c0039000 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.466+0000 7fe6bffff700 1 -- 172.21.15.45:0/2210205034 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe6d81033e0 con 0x7fe6c0039000 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 msgr2=0x7fe6c003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 0x7fe6c003b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006fd0 tx=0x7fe6cc006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 msgr2=0x7fe6d819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009fb0 tx=0x7fe6c8011bf0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 shutdown_connections 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fe6c0039000 0x7fe6c003b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 --2- 172.21.15.45:0/2210205034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6d8106b80 0x7fe6d819bcf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:12.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.468+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 >> 172.21.15.45:0/2210205034 conn(0x7fe6d8102100 msgr2=0x7fe6d8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:12.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.469+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 shutdown_connections 2024-04-25T00:12:12.188 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:11.469+0000 7fe6dd6c3700 1 -- 172.21.15.45:0/2210205034 wait complete. 2024-04-25T00:12:12.188 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-04-25T00:12:12.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:12 smithi045 ceph-mon[29194]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:12.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:12 smithi045 ceph-mon[29194]: Saving service mon spec with placement count:5 2024-04-25T00:12:12.744 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:12 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:14.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:13 smithi045 ceph-mon[29194]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:14.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:13 smithi045 ceph-mon[29194]: Saving service mgr spec with placement count:2 2024-04-25T00:12:14.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:13 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:14.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:13 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa563d1e700 1 Processor -- start 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa563d1e700 1 -- start start 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa563d1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa563d1e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c108660 con 0x7fa55c107d00 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa561aba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa561aba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36262/0 (socket says 172.21.15.45:36262) 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.566+0000 7fa561aba700 1 -- 172.21.15.45:0/2516846373 learned_addr learned my addr 172.21.15.45:0/2516846373 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.567+0000 7fa561aba700 1 -- 172.21.15.45:0/2516846373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c1087e0 con 0x7fa55c107d00 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.567+0000 7fa561aba700 1 --2- 172.21.15.45:0/2516846373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5580098a0 tx=0x7fa558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829019f87af3fccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.104 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.567+0000 7fa560ab8700 1 -- 172.21.15.45:0/2516846373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa558004160 con 0x7fa55c107d00 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.567+0000 7fa560ab8700 1 -- 172.21.15.45:0/2516846373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa558004300 con 0x7fa55c107d00 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.567+0000 7fa560ab8700 1 -- 172.21.15.45:0/2516846373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa558011b50 con 0x7fa55c107d00 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- 172.21.15.45:0/2516846373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 msgr2=0x7fa55c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 --2- 172.21.15.45:0/2516846373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5580098a0 tx=0x7fa558009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- 172.21.15.45:0/2516846373 shutdown_connections 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 --2- 172.21.15.45:0/2516846373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c108120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- 172.21.15.45:0/2516846373 >> 172.21.15.45:0/2516846373 conn(0x7fa55c1033a0 msgr2=0x7fa55c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- 172.21.15.45:0/2516846373 shutdown_connections 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- 172.21.15.45:0/2516846373 wait complete. 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 Processor -- start 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.568+0000 7fa563d1e700 1 -- start start 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa563d1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa563d1e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10c120 con 0x7fa55c107d00 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa561aba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa561aba700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36274/0 (socket says 172.21.15.45:36274) 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa561aba700 1 -- 172.21.15.45:0/1731678504 learned_addr learned my addr 172.21.15.45:0/1731678504 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa561aba700 1 -- 172.21.15.45:0/1731678504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c07e0b0 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa561aba700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa558011b60 tx=0x7fa558011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa558003b20 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa558003cc0 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c07c2a0 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa558024840 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.569+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c07c7e0 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.570+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa558022020 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.570+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c063890 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.570+0000 7fa552ffd700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 0x7fa54803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.570+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa558011e00 con 0x7fa55c107d00 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.570+0000 7fa5612b9700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 0x7fa54803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.571+0000 7fa5612b9700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 0x7fa54803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006fd0 tx=0x7fa54c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.575+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa55802d3f0 con 0x7fa55c107d00 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.798+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa55c104550 con 0x7fa548039050 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.803+0000 7fa552ffd700 1 -- 172.21.15.45:0/1731678504 <== mgr.14118 v2:172.21.15.45:6800/1299966585 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa55c104550 con 0x7fa548039050 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 msgr2=0x7fa54803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 0x7fa54803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006fd0 tx=0x7fa54c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 msgr2=0x7fa55c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa558011b60 tx=0x7fa558011c40 comp rx=0 tx=0).stop 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 shutdown_connections 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa548039050 0x7fa54803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 --2- 172.21.15.45:0/1731678504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa55c107d00 0x7fa55c07db70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 >> 172.21.15.45:0/1731678504 conn(0x7fa55c1033a0 msgr2=0x7fa55c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 shutdown_connections 2024-04-25T00:12:14.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:12.805+0000 7fa563d1e700 1 -- 172.21.15.45:0/1731678504 wait complete. 2024-04-25T00:12:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc780e94700 1 Processor -- start 2024-04-25T00:12:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc780e94700 1 -- start start 2024-04-25T00:12:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc780e94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc780e94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c07c860 con 0x7fc77c07ddb0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc77a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc77a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35200/0 (socket says 172.21.15.45:35200) 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc77a59c700 1 -- 172.21.15.45:0/3529452174 learned_addr learned my addr 172.21.15.45:0/3529452174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.570+0000 7fc77a59c700 1 -- 172.21.15.45:0/3529452174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c07c9e0 con 0x7fc77c07ddb0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc77a59c700 1 --2- 172.21.15.45:0/3529452174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a74814223d47ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc77959a700 1 -- 172.21.15.45:0/3529452174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c004160 con 0x7fc77c07ddb0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc77959a700 1 -- 172.21.15.45:0/3529452174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c004300 con 0x7fc77c07ddb0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc77959a700 1 -- 172.21.15.45:0/3529452174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c011b20 con 0x7fc77c07ddb0 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc780e94700 1 -- 172.21.15.45:0/3529452174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 msgr2=0x7fc77c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc780e94700 1 --2- 172.21.15.45:0/3529452174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc780e94700 1 -- 172.21.15.45:0/3529452174 shutdown_connections 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc780e94700 1 --2- 172.21.15.45:0/3529452174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c07c320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.379 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.571+0000 7fc780e94700 1 -- 172.21.15.45:0/3529452174 >> 172.21.15.45:0/3529452174 conn(0x7fc77c1032b0 msgr2=0x7fc77c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.380 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.572+0000 7fc780e94700 1 -- 172.21.15.45:0/3529452174 shutdown_connections 2024-04-25T00:12:15.380 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.572+0000 7fc780e94700 1 -- 172.21.15.45:0/3529452174 wait complete. 2024-04-25T00:12:15.380 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.572+0000 7fc780e94700 1 Processor -- start 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.572+0000 7fc780e94700 1 -- start start 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc780e94700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc780e94700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c10e160 con 0x7fc77c07ddb0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc77a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc77a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35206/0 (socket says 172.21.15.45:35206) 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc77a59c700 1 -- 172.21.15.45:0/1279942843 learned_addr learned my addr 172.21.15.45:0/1279942843 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc77a59c700 1 -- 172.21.15.45:0/1279942843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c19e2d0 con 0x7fc77c07ddb0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc77a59c700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0117e0 tx=0x7fc76c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c003bf0 con 0x7fc77c07ddb0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c19e560 con 0x7fc77c07ddb0 2024-04-25T00:12:15.381 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c01a910 con 0x7fc77c07ddb0 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c024970 con 0x7fc77c07ddb0 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.573+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c1a1210 con 0x7fc77c07ddb0 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.574+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc76c022020 con 0x7fc77c07ddb0 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.574+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c063890 con 0x7fc77c07ddb0 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.574+0000 7fc76b7fe700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 0x7fc76403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:15.382 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.575+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc76c04d400 con 0x7fc77c07ddb0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.575+0000 7fc779d9b700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 0x7fc76403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.575+0000 7fc779d9b700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 0x7fc76403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc770006fd0 tx=0x7fc770006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.579+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc76c011e20 con 0x7fc77c07ddb0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.796+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc77c1a17d0 con 0x7fc77c07ddb0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.798+0000 7fc76b7fe700 1 -- 172.21.15.45:0/1279942843 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc76c019c70 con 0x7fc77c07ddb0 2024-04-25T00:12:15.383 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.800+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 msgr2=0x7fc76403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.800+0000 7fc780e94700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 0x7fc76403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc770006fd0 tx=0x7fc770006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 msgr2=0x7fc77c19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0117e0 tx=0x7fc76c011c10 comp rx=0 tx=0).stop 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 shutdown_connections 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fc764039000 0x7fc76403b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 --2- 172.21.15.45:0/1279942843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc77c07ddb0 0x7fc77c19dd90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 >> 172.21.15.45:0/1279942843 conn(0x7fc77c1032b0 msgr2=0x7fc77c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 shutdown_connections 2024-04-25T00:12:15.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:14.801+0000 7fc780e94700 1 -- 172.21.15.45:0/1279942843 wait complete. 2024-04-25T00:12:16.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:15 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:16.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:15 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1279942843' entity='client.admin' 2024-04-25T00:12:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:17 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2183389306' entity='client.admin' 2024-04-25T00:12:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:17 smithi045 ceph-mon[29194]: from='mgr.14118 172.21.15.45:0/77254032' entity='mgr.x' 2024-04-25T00:12:17.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.793+0000 7fac5a221700 1 Processor -- start 2024-04-25T00:12:17.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.793+0000 7fac5a221700 1 -- start start 2024-04-25T00:12:17.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.793+0000 7fac5a221700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.793+0000 7fac5a221700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac54108b70 con 0x7fac54108180 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35208/0 (socket says 172.21.15.45:35208) 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac537fe700 1 -- 172.21.15.45:0/3066746381 learned_addr learned my addr 172.21.15.45:0/3066746381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac537fe700 1 -- 172.21.15.45:0/3066746381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac54105c80 con 0x7fac54108180 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac537fe700 1 --2- 172.21.15.45:0/3066746381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac44009e80 tx=0x7fac4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a4425e6373f1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac527fc700 1 -- 172.21.15.45:0/3066746381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac44017070 con 0x7fac54108180 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac527fc700 1 -- 172.21.15.45:0/3066746381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac4400b8c0 con 0x7fac54108180 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.794+0000 7fac527fc700 1 -- 172.21.15.45:0/3066746381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac440045c0 con 0x7fac54108180 2024-04-25T00:12:17.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 -- 172.21.15.45:0/3066746381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 msgr2=0x7fac541085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 --2- 172.21.15.45:0/3066746381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac44009e80 tx=0x7fac4400b330 comp rx=0 tx=0).stop 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 -- 172.21.15.45:0/3066746381 shutdown_connections 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 --2- 172.21.15.45:0/3066746381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac541085a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 -- 172.21.15.45:0/3066746381 >> 172.21.15.45:0/3066746381 conn(0x7fac541018e0 msgr2=0x7fac54103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 -- 172.21.15.45:0/3066746381 shutdown_connections 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 -- 172.21.15.45:0/3066746381 wait complete. 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.795+0000 7fac5a221700 1 Processor -- start 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac5a221700 1 -- start start 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac5a221700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac5a221700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac54106d40 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac537fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35218/0 (socket says 172.21.15.45:35218) 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac537fe700 1 -- 172.21.15.45:0/2183389306 learned_addr learned my addr 172.21.15.45:0/2183389306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac537fe700 1 -- 172.21.15.45:0/2183389306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5419e2d0 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac537fe700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b7b0 tx=0x7fac4400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac44020020 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac44003d90 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.796+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5419e560 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.797+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac44017040 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.797+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a1210 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.797+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fac44023020 con 0x7fac54108180 2024-04-25T00:12:17.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.797+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54063890 con 0x7fac54108180 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.798+0000 7fac50ff9700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 0x7fac4003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.798+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac4400f050 con 0x7fac54108180 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.798+0000 7fac52ffd700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 0x7fac4003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.798+0000 7fac52ffd700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 0x7fac4003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006fd0 tx=0x7fac3c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:15.803+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fac4402a020 con 0x7fac54108180 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.029+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fac54197370 con 0x7fac54108180 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.032+0000 7fac50ff9700 1 -- 172.21.15.45:0/2183389306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fac54197370 con 0x7fac54108180 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 msgr2=0x7fac4003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 0x7fac4003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006fd0 tx=0x7fac3c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:17.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 msgr2=0x7fac5419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b7b0 tx=0x7fac4400bd50 comp rx=0 tx=0).stop 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 shutdown_connections 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fac40039050 0x7fac4003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 --2- 172.21.15.45:0/2183389306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac54108180 0x7fac5419dd90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 >> 172.21.15.45:0/2183389306 conn(0x7fac541018e0 msgr2=0x7fac54194430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 shutdown_connections 2024-04-25T00:12:17.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:16.034+0000 7fac5a221700 1 -- 172.21.15.45:0/2183389306 wait complete. 2024-04-25T00:12:17.186 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-04-25T00:12:18.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1318994117' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T00:12:18.219 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setuser ceph since I am not root 2024-04-25T00:12:18.219 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setgroup ceph since I am not root 2024-04-25T00:12:18.219 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:18.218+0000 7eff1d091200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:12:18.515 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:18.409+0000 7eff1d091200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:12:18.516 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:18.515+0000 7eff1d091200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:12:18.549 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e39af700 1 Processor -- start 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e39af700 1 -- start start 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e39af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e39af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc107550 con 0x7fa2dc106b60 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e174b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e174b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35234/0 (socket says 172.21.15.45:35234) 2024-04-25T00:12:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.622+0000 7fa2e174b700 1 -- 172.21.15.45:0/3792251531 learned_addr learned my addr 172.21.15.45:0/3792251531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.623+0000 7fa2e174b700 1 -- 172.21.15.45:0/3792251531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc107d60 con 0x7fa2dc106b60 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.623+0000 7fa2e174b700 1 --2- 172.21.15.45:0/3792251531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80098a0 tx=0x7fa2d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b926b7e65665a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.623+0000 7fa2d3fff700 1 -- 172.21.15.45:0/3792251531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2d8004160 con 0x7fa2dc106b60 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.623+0000 7fa2d3fff700 1 -- 172.21.15.45:0/3792251531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d8004300 con 0x7fa2dc106b60 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.623+0000 7fa2d3fff700 1 -- 172.21.15.45:0/3792251531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2d8011b20 con 0x7fa2dc106b60 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 -- 172.21.15.45:0/3792251531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 msgr2=0x7fa2dc106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 --2- 172.21.15.45:0/3792251531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80098a0 tx=0x7fa2d8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 -- 172.21.15.45:0/3792251531 shutdown_connections 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 --2- 172.21.15.45:0/3792251531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc106f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 -- 172.21.15.45:0/3792251531 >> 172.21.15.45:0/3792251531 conn(0x7fa2dc102120 msgr2=0x7fa2dc104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 -- 172.21.15.45:0/3792251531 shutdown_connections 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.624+0000 7fa2e39af700 1 -- 172.21.15.45:0/3792251531 wait complete. 2024-04-25T00:12:18.552 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e39af700 1 Processor -- start 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e39af700 1 -- start start 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e39af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e39af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc10b110 con 0x7fa2dc106b60 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e174b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e174b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35246/0 (socket says 172.21.15.45:35246) 2024-04-25T00:12:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e174b700 1 -- 172.21.15.45:0/1318994117 learned_addr learned my addr 172.21.15.45:0/1318994117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e174b700 1 -- 172.21.15.45:0/1318994117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc19c170 con 0x7fa2dc106b60 2024-04-25T00:12:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.625+0000 7fa2e174b700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80117e0 tx=0x7fa2d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.626+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2d8003bf0 con 0x7fa2dc106b60 2024-04-25T00:12:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.626+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc19c400 con 0x7fa2dc106b60 2024-04-25T00:12:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.626+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d801a910 con 0x7fa2dc106b60 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.626+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc19c860 con 0x7fa2dc106b60 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.626+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2d80249a0 con 0x7fa2dc106b60 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.627+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa2d8022020 con 0x7fa2dc106b60 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.627+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc063890 con 0x7fa2dc106b60 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.627+0000 7fa2d27fc700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 0x7fa2c803f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:18.555 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.627+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa2d804d360 con 0x7fa2dc106b60 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.627+0000 7fa2e0f4a700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 0x7fa2c803f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.628+0000 7fa2e0f4a700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 0x7fa2c803f8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006fd0 tx=0x7fa2cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.632+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa2d8011e20 con 0x7fa2dc106b60 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:17.900+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa2dc062e40 con 0x7fa2dc106b60 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.034+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa2d80172b0 con 0x7fa2dc106b60 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.036+0000 7fa2d27fc700 1 -- 172.21.15.45:0/1318994117 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa2d802d910 con 0x7fa2dc106b60 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.037+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 msgr2=0x7fa2c803f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.037+0000 7fa2e39af700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 0x7fa2c803f8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006fd0 tx=0x7fa2cc006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:18.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 msgr2=0x7fa2dc19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:18.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80117e0 tx=0x7fa2d8011c10 comp rx=0 tx=0).stop 2024-04-25T00:12:18.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 shutdown_connections 2024-04-25T00:12:18.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fa2c803d410 0x7fa2c803f8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 --2- 172.21.15.45:0/1318994117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2dc106b60 0x7fa2dc19bc30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:18.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 >> 172.21.15.45:0/1318994117 conn(0x7fa2dc102120 msgr2=0x7fa2dc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:18.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 shutdown_connections 2024-04-25T00:12:18.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.038+0000 7fa2e39af700 1 -- 172.21.15.45:0/1318994117 wait complete. 2024-04-25T00:12:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:19 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1318994117' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T00:12:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:19 smithi045 ceph-mon[29194]: mgrmap e9: x(active, since 19s) 2024-04-25T00:12:19.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:18.842+0000 7eff1d091200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:12:19.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:18.932+0000 7eff1d091200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:12:19.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:12:19.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.968+0000 7fec5f434700 1 Processor -- start 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5f434700 1 -- start start 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5f434700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5f434700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec5807be30 con 0x7fec58108710 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5d1d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5d1d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35274/0 (socket says 172.21.15.45:35274) 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5d1d0700 1 -- 172.21.15.45:0/1401897890 learned_addr learned my addr 172.21.15.45:0/1401897890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.969+0000 7fec5d1d0700 1 -- 172.21.15.45:0/1401897890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5810b040 con 0x7fec58108710 2024-04-25T00:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5d1d0700 1 --2- 172.21.15.45:0/1401897890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec540098a0 tx=0x7fec54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad738e3a453189df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec4ffff700 1 -- 172.21.15.45:0/1401897890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec54004160 con 0x7fec58108710 2024-04-25T00:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec4ffff700 1 -- 172.21.15.45:0/1401897890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec54004300 con 0x7fec58108710 2024-04-25T00:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec4ffff700 1 -- 172.21.15.45:0/1401897890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec54011b20 con 0x7fec58108710 2024-04-25T00:12:19.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5f434700 1 -- 172.21.15.45:0/1401897890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 msgr2=0x7fec5810ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5f434700 1 --2- 172.21.15.45:0/1401897890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec540098a0 tx=0x7fec54009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5f434700 1 -- 172.21.15.45:0/1401897890 shutdown_connections 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5f434700 1 --2- 172.21.15.45:0/1401897890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5810ab00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.970+0000 7fec5f434700 1 -- 172.21.15.45:0/1401897890 >> 172.21.15.45:0/1401897890 conn(0x7fec58102100 msgr2=0x7fec58104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 -- 172.21.15.45:0/1401897890 shutdown_connections 2024-04-25T00:12:19.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 -- 172.21.15.45:0/1401897890 wait complete. 2024-04-25T00:12:19.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 Processor -- start 2024-04-25T00:12:19.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 -- start start 2024-04-25T00:12:19.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5f434700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810c040 con 0x7fec58108710 2024-04-25T00:12:19.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5d1d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:19.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5d1d0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35290/0 (socket says 172.21.15.45:35290) 2024-04-25T00:12:19.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.971+0000 7fec5d1d0700 1 -- 172.21.15.45:0/1139256760 learned_addr learned my addr 172.21.15.45:0/1139256760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:19.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec5d1d0700 1 -- 172.21.15.45:0/1139256760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5819c0a0 con 0x7fec58108710 2024-04-25T00:12:19.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec5d1d0700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec540117e0 tx=0x7fec54011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec54003ba0 con 0x7fec58108710 2024-04-25T00:12:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5819c330 con 0x7fec58108710 2024-04-25T00:12:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5401a910 con 0x7fec58108710 2024-04-25T00:12:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec540248f0 con 0x7fec58108710 2024-04-25T00:12:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.972+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5819c790 con 0x7fec58108710 2024-04-25T00:12:19.799 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58063890 con 0x7fec58108710 2024-04-25T00:12:19.799 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fec54022020 con 0x7fec58108710 2024-04-25T00:12:19.799 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec4e7fc700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 0x7fec4403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fec5404da40 con 0x7fec58108710 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec5c9cf700 1 -- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 msgr2=0x7fec4403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.973+0000 7fec5c9cf700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 0x7fec4403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:18.978+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec5401ab30 con 0x7fec58108710 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.174+0000 7fec5c9cf700 1 -- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 msgr2=0x7fec4403b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.174+0000 7fec5c9cf700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 0x7fec4403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:12:19.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.249+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fec5819f140 con 0x7fec58108710 2024-04-25T00:12:19.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.249+0000 7fec4e7fc700 1 -- 172.21.15.45:0/1139256760 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fec5404de20 con 0x7fec58108710 2024-04-25T00:12:19.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 msgr2=0x7fec4403b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:12:19.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 0x7fec4403b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 msgr2=0x7fec5819bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fec540117e0 tx=0x7fec54011ba0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 shutdown_connections 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7fec440390a0 0x7fec4403b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 --2- 172.21.15.45:0/1139256760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec58108710 0x7fec5819bb60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 >> 172.21.15.45:0/1139256760 conn(0x7fec58102100 msgr2=0x7fec58102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:19.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 shutdown_connections 2024-04-25T00:12:19.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:19.251+0000 7fec5f434700 1 -- 172.21.15.45:0/1139256760 wait complete. 2024-04-25T00:12:19.803 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-25T00:12:19.803 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-04-25T00:12:20.196 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1139256760' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:12:20.450 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:20.196+0000 7eff1d091200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:12:20.451 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:20.376+0000 7eff1d091200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:12:20.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:20.553+0000 7eff1d091200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:12:21.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:20.854+0000 7eff1d091200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:12:21.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:20.950+0000 7eff1d091200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:12:21.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:21.161+0000 7eff1d091200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:12:21.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:21.360+0000 7eff1d091200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:12:22.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:21.985+0000 7eff1d091200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:12:22.732 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:22.423+0000 7eff1d091200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:12:22.732 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:22.523+0000 7eff1d091200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:12:22.732 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:22.731+0000 7eff1d091200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:12:23.647 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:23.291+0000 7eff1d091200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:12:23.647 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:23.389+0000 7eff1d091200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:12:23.899 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:23.647+0000 7eff1d091200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:12:23.922 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:23.787+0000 7eff1d091200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:12:23.922 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:23.887+0000 7eff1d091200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:12:24.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:24.009+0000 7eff1d091200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:12:24.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:24.105+0000 7eff1d091200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:12:25.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:24.949+0000 7eff1d091200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:12:25.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:12:25.173+0000 7eff1d091200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:12:25.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: Active manager daemon x restarted 2024-04-25T00:12:25.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: Activating manager daemon x 2024-04-25T00:12:25.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T00:12:25.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: mgrmap e10: x(active, starting, since 0.00343466s) 2024-04-25T00:12:25.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: Manager daemon x is now available 2024-04-25T00:12:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:26.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:12:26.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:12:26.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: [25/Apr/2024:00:12:25] ENGINE Bus STARTING 2024-04-25T00:12:26.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: [25/Apr/2024:00:12:25] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T00:12:26.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: [25/Apr/2024:00:12:26] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T00:12:26.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: [25/Apr/2024:00:12:26] ENGINE Bus STARTED 2024-04-25T00:12:26.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:26 smithi045 ceph-mon[29194]: mgrmap e11: x(active, since 1.00608s) 2024-04-25T00:12:26.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:12:26.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T00:12:26.974 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.244+0000 7f8b2c2be700 1 Processor -- start 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.244+0000 7f8b2c2be700 1 -- start start 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2c2be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2c2be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24107570 con 0x7f8b24106b80 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2a05a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2a05a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35298/0 (socket says 172.21.15.45:35298) 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2a05a700 1 -- 172.21.15.45:0/3875989988 learned_addr learned my addr 172.21.15.45:0/3875989988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2a05a700 1 -- 172.21.15.45:0/3875989988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b24107d80 con 0x7f8b24106b80 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.245+0000 7f8b2a05a700 1 --2- 172.21.15.45:0/3875989988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b200098a0 tx=0x7f8b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cf47f0bdbd86f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b29058700 1 -- 172.21.15.45:0/3875989988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b20004160 con 0x7f8b24106b80 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b29058700 1 -- 172.21.15.45:0/3875989988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b20004300 con 0x7f8b24106b80 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b29058700 1 -- 172.21.15.45:0/3875989988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b20011b20 con 0x7f8b24106b80 2024-04-25T00:12:26.975 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 -- 172.21.15.45:0/3875989988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 msgr2=0x7f8b24106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/3875989988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b200098a0 tx=0x7f8b20009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 -- 172.21.15.45:0/3875989988 shutdown_connections 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/3875989988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b24106fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 -- 172.21.15.45:0/3875989988 >> 172.21.15.45:0/3875989988 conn(0x7f8b24102100 msgr2=0x7f8b24104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 -- 172.21.15.45:0/3875989988 shutdown_connections 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.246+0000 7f8b2c2be700 1 -- 172.21.15.45:0/3875989988 wait complete. 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2c2be700 1 Processor -- start 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2c2be700 1 -- start start 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2c2be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2c2be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2410b130 con 0x7f8b24106b80 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2a05a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2a05a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35302/0 (socket says 172.21.15.45:35302) 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.247+0000 7f8b2a05a700 1 -- 172.21.15.45:0/1471115826 learned_addr learned my addr 172.21.15.45:0/1471115826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:26.976 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b2a05a700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2419c1a0 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b2a05a700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b200116a0 tx=0x7f8b20011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b20003a40 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2419c430 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b20003be0 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b2001a720 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.248+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2419c890 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8b20024aa0 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b1b7fe700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b1003bc20 con 0x7f8b10039050 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8b2004d700 con 0x7f8b24106b80 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b29859700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.249+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.450+0000 7f8b29859700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.450+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:12:26.977 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.851+0000 7f8b29859700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:20.851+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:21.652+0000 7f8b29859700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:21.652+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:23.253+0000 7f8b29859700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1299966585 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:23.253+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:25.176+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f8b20024430 con 0x7f8b24106b80 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:25.176+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 msgr2=0x7f8b1003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:25.176+0000 7f8b1b7fe700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.178+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f8b2004ef70 con 0x7f8b24106b80 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.178+0000 7f8b1b7fe700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 0x7f8b1003ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.178+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b20020ec0 con 0x7f8b1003cb10 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.179+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 0x7f8b1003ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.179+0000 7f8b29859700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 0x7f8b1003ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8b14003a10 tx=0x7f8b140092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.180+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8b20020ec0 con 0x7f8b1003cb10 2024-04-25T00:12:26.978 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8b24063890 con 0x7f8b1003cb10 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b1b7fe700 1 -- 172.21.15.45:0/1471115826 <== mgr.14150 v2:172.21.15.45:6800/3305016128 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8b24063890 con 0x7f8b1003cb10 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 msgr2=0x7f8b1003ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 0x7f8b1003ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8b14003a10 tx=0x7f8b140092b0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 msgr2=0x7f8b2419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b200116a0 tx=0x7f8b20011ba0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 shutdown_connections 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8b1003cb10 0x7f8b1003ef00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b24106b80 0x7f8b2419bc60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 --2- 172.21.15.45:0/1471115826 >> [v2:172.21.15.45:6800/1299966585,v1:172.21.15.45:6801/1299966585] conn(0x7f8b10039050 0x7f8b1003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 >> 172.21.15.45:0/1471115826 conn(0x7f8b24102100 msgr2=0x7f8b24102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 shutdown_connections 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:26.187+0000 7f8b2c2be700 1 -- 172.21.15.45:0/1471115826 wait complete. 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-04-25T00:12:26.979 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.419+0000 7f1607222700 1 Processor -- start 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.419+0000 7f1607222700 1 -- start start 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.419+0000 7f1607222700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.419+0000 7f1607222700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1600107510 con 0x7f1600106b20 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f1604fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f1604fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55902/0 (socket says 172.21.15.45:55902) 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f1604fbe700 1 -- 172.21.15.45:0/230342029 learned_addr learned my addr 172.21.15.45:0/230342029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:28.337 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f1604fbe700 1 -- 172.21.15.45:0/230342029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1600107d20 con 0x7f1600106b20 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f1604fbe700 1 --2- 172.21.15.45:0/230342029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15f40098a0 tx=0x7f15f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a043f2851c7632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f15ff7fe700 1 -- 172.21.15.45:0/230342029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f4004160 con 0x7f1600106b20 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f15ff7fe700 1 -- 172.21.15.45:0/230342029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f4004300 con 0x7f1600106b20 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.420+0000 7f15ff7fe700 1 -- 172.21.15.45:0/230342029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f4011b20 con 0x7f1600106b20 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 -- 172.21.15.45:0/230342029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 msgr2=0x7f1600106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 --2- 172.21.15.45:0/230342029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15f40098a0 tx=0x7f15f4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 -- 172.21.15.45:0/230342029 shutdown_connections 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 --2- 172.21.15.45:0/230342029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f1600106f40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 -- 172.21.15.45:0/230342029 >> 172.21.15.45:0/230342029 conn(0x7f1600102100 msgr2=0x7f1600104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 -- 172.21.15.45:0/230342029 shutdown_connections 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.421+0000 7f1607222700 1 -- 172.21.15.45:0/230342029 wait complete. 2024-04-25T00:12:28.338 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1607222700 1 Processor -- start 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1607222700 1 -- start start 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1607222700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1607222700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f160010b0d0 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1604fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1604fbe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55918/0 (socket says 172.21.15.45:55918) 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1604fbe700 1 -- 172.21.15.45:0/2753068388 learned_addr learned my addr 172.21.15.45:0/2753068388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1604fbe700 1 -- 172.21.15.45:0/2753068388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160019c0f0 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.422+0000 7f1604fbe700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15f40117e0 tx=0x7f15f4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.423+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f4003bf0 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.423+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f401a910 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.423+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160019c380 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.423+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f4024970 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.423+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160019c7e0 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.424+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1600063890 con 0x7f1600106b20 2024-04-25T00:12:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.424+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f15f4022020 con 0x7f1600106b20 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.424+0000 7f15fdffb700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 0x7f15f003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.424+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f15f404d790 con 0x7f1600106b20 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.424+0000 7f15fffff700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 0x7f15f003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.425+0000 7f15fffff700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 0x7f15f003b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15ec006fd0 tx=0x7f15ec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.429+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15f402cdf0 con 0x7f1600106b20 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.644+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1600062670 con 0x7f15f0038f80 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.756+0000 7f15fdffb700 1 -- 172.21.15.45:0/2753068388 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1600062670 con 0x7f15f0038f80 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 msgr2=0x7f15f003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 0x7f15f003b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15ec006fd0 tx=0x7f15ec006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 msgr2=0x7f160019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15f40117e0 tx=0x7f15f4011ba0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 shutdown_connections 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f15f0038f80 0x7f15f003b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 --2- 172.21.15.45:0/2753068388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1600106b20 0x7f160019bbb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:28.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 >> 172.21.15.45:0/2753068388 conn(0x7f1600102100 msgr2=0x7f1600102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:28.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 shutdown_connections 2024-04-25T00:12:28.341 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:27.758+0000 7f1607222700 1 -- 172.21.15.45:0/2753068388 wait complete. 2024-04-25T00:12:28.341 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-04-25T00:12:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:28 smithi045 ceph-mon[29194]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:12:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:28 smithi045 ceph-mon[29194]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:12:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:29.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:29 smithi045 ceph-mon[29194]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:29.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:29 smithi045 ceph-mon[29194]: mgrmap e12: x(active, since 2s) 2024-04-25T00:12:29.278 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:30.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:30 smithi045 ceph-mon[29194]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:30.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$i0L0CLb3nuGM6hiXk0IJku.1akR/D2AvPQlsL5CtThPgGn1IHUpI6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714003949, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.772+0000 7f0a4679e700 1 Processor -- start 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4679e700 1 -- start start 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4679e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4679e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4007be70 con 0x7f0a40108560 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4579c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4579c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55934/0 (socket says 172.21.15.45:55934) 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.773+0000 7f0a4579c700 1 -- 172.21.15.45:0/1934958975 learned_addr learned my addr 172.21.15.45:0/1934958975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.774+0000 7f0a4579c700 1 -- 172.21.15.45:0/1934958975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4010ae90 con 0x7f0a40108560 2024-04-25T00:12:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.774+0000 7f0a4579c700 1 --2- 172.21.15.45:0/1934958975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a300098a0 tx=0x7f0a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e81145ebec37be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.774+0000 7f0a37fff700 1 -- 172.21.15.45:0/1934958975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30004160 con 0x7f0a40108560 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.774+0000 7f0a37fff700 1 -- 172.21.15.45:0/1934958975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a30004300 con 0x7f0a40108560 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.774+0000 7f0a37fff700 1 -- 172.21.15.45:0/1934958975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30011b20 con 0x7f0a40108560 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 -- 172.21.15.45:0/1934958975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 msgr2=0x7f0a4010a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 --2- 172.21.15.45:0/1934958975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a300098a0 tx=0x7f0a30009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 -- 172.21.15.45:0/1934958975 shutdown_connections 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 --2- 172.21.15.45:0/1934958975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4010a950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 -- 172.21.15.45:0/1934958975 >> 172.21.15.45:0/1934958975 conn(0x7f0a40101f70 msgr2=0x7f0a40104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 -- 172.21.15.45:0/1934958975 shutdown_connections 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.775+0000 7f0a4679e700 1 -- 172.21.15.45:0/1934958975 wait complete. 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4679e700 1 Processor -- start 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4679e700 1 -- start start 2024-04-25T00:12:30.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4679e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4679e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4010bef0 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4579c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4579c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55950/0 (socket says 172.21.15.45:55950) 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4579c700 1 -- 172.21.15.45:0/392974535 learned_addr learned my addr 172.21.15.45:0/392974535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.776+0000 7f0a4579c700 1 -- 172.21.15.45:0/392974535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4019bf20 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a4579c700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a300117e0 tx=0x7f0a30011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30003bf0 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a3001a910 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4019c1b0 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30024970 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.777+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4019c610 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.778+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f0a30022020 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.778+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a40063890 con 0x7f0a40108560 2024-04-25T00:12:30.602 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.778+0000 7f0a367fc700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 0x7f0a2c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.778+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a3004d4d0 con 0x7f0a40108560 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.778+0000 7f0a44f9b700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 0x7f0a2c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.779+0000 7f0a44f9b700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 0x7f0a2c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006fd0 tx=0x7f0a3c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:28.783+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a30011e20 con 0x7f0a40108560 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.018+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0a4019f040 con 0x7f0a2c039050 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.264+0000 7f0a367fc700 1 -- 172.21.15.45:0/392974535 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0a4019f040 con 0x7f0a2c039050 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.266+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 msgr2=0x7f0a2c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.266+0000 7f0a4679e700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 0x7f0a2c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006fd0 tx=0x7f0a3c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.266+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 msgr2=0x7f0a4019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.266+0000 7f0a4679e700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a300117e0 tx=0x7f0a30011c10 comp rx=0 tx=0).stop 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 shutdown_connections 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a2c039050 0x7f0a2c03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 --2- 172.21.15.45:0/392974535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a40108560 0x7f0a4019b9e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 >> 172.21.15.45:0/392974535 conn(0x7f0a40101f70 msgr2=0x7f0a401029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 shutdown_connections 2024-04-25T00:12:30.604 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:29.267+0000 7f0a4679e700 1 -- 172.21.15.45:0/392974535 wait complete. 2024-04-25T00:12:30.604 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.022+0000 7fdfce98b700 1 Processor -- start 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.022+0000 7fdfce98b700 1 -- start start 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.022+0000 7fdfce98b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfce98b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc807e160 con 0x7fdfc807dd40 2024-04-25T00:12:31.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55956/0 (socket says 172.21.15.45:55956) 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc7fff700 1 -- 172.21.15.45:0/338099479 learned_addr learned my addr 172.21.15.45:0/338099479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc7fff700 1 -- 172.21.15.45:0/338099479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc807c840 con 0x7fdfc807dd40 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc7fff700 1 --2- 172.21.15.45:0/338099479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098a0 tx=0x7fdfb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815c0197796ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc6ffd700 1 -- 172.21.15.45:0/338099479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8004160 con 0x7fdfc807dd40 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc6ffd700 1 -- 172.21.15.45:0/338099479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb8004300 con 0x7fdfc807dd40 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.023+0000 7fdfc6ffd700 1 -- 172.21.15.45:0/338099479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8011b20 con 0x7fdfc807dd40 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 -- 172.21.15.45:0/338099479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 msgr2=0x7fdfc807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 --2- 172.21.15.45:0/338099479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098a0 tx=0x7fdfb8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 -- 172.21.15.45:0/338099479 shutdown_connections 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 --2- 172.21.15.45:0/338099479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc807c300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 -- 172.21.15.45:0/338099479 >> 172.21.15.45:0/338099479 conn(0x7fdfc81033a0 msgr2=0x7fdfc81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 -- 172.21.15.45:0/338099479 shutdown_connections 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.024+0000 7fdfce98b700 1 -- 172.21.15.45:0/338099479 wait complete. 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfce98b700 1 Processor -- start 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfce98b700 1 -- start start 2024-04-25T00:12:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfce98b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfce98b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc810e1b0 con 0x7fdfc807dd40 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfc7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55964/0 (socket says 172.21.15.45:55964) 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfc7fff700 1 -- 172.21.15.45:0/2165012589 learned_addr learned my addr 172.21.15.45:0/2165012589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfc7fff700 1 -- 172.21.15.45:0/2165012589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc819e300 con 0x7fdfc807dd40 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.025+0000 7fdfc7fff700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80117e0 tx=0x7fdfb8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.026+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8003bf0 con 0x7fdfc807dd40 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.026+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb801a910 con 0x7fdfc807dd40 2024-04-25T00:12:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.026+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc819e590 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.026+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8024970 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.026+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc819e9f0 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fdfb8022020 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfc57fa700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 0x7fdfb003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8063890 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdfb804d600 con 0x7fdfc807dd40 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfc77fe700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 0x7fdfb003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.027+0000 7fdfc77fe700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 0x7fdfb003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006fd0 tx=0x7fdfbc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:31.829 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.032+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfb8011e20 con 0x7fdfc807dd40 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.228+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdfb802d770 con 0x7fdfc807dd40 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.241+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdfc81a13c0 con 0x7fdfc807dd40 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.242+0000 7fdfc57fa700 1 -- 172.21.15.45:0/2165012589 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdfb800ba30 con 0x7fdfc807dd40 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 msgr2=0x7fdfb003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 0x7fdfb003b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc006fd0 tx=0x7fdfbc006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 msgr2=0x7fdfc819ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80117e0 tx=0x7fdfb8011c10 comp rx=0 tx=0).stop 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 shutdown_connections 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdfb00390b0 0x7fdfb003b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 --2- 172.21.15.45:0/2165012589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc807dd40 0x7fdfc819ddc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 >> 172.21.15.45:0/2165012589 conn(0x7fdfc81033a0 msgr2=0x7fdfc8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:31.830 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 shutdown_connections 2024-04-25T00:12:31.831 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:31.244+0000 7fdfce98b700 1 -- 172.21.15.45:0/2165012589 wait complete. 2024-04-25T00:12:31.833 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:12:31.833 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T00:12:31.841 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:12:31.841 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T00:12:31.841 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T00:12:31.841 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T00:12:31.843 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-04-25T00:12:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045:8443/ 2024-04-25T00:12:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-04-25T00:12:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: Password: cleypmbfpe 2024-04-25T00:12:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:31.844 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config directory 2024-04-25T00:12:31.844 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-04-25T00:12:32.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:32 smithi045 ceph-mon[29194]: mgrmap e13: x(active, since 6s) 2024-04-25T00:12:32.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:32 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2165012589' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T00:12:33.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb370c81700 1 Processor -- start 2024-04-25T00:12:33.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb370c81700 1 -- start start 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb370c81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb370c81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c108660 con 0x7fb36c107d00 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb36a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.326+0000 7fb36a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55978/0 (socket says 172.21.15.45:55978) 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36a59c700 1 -- 172.21.15.45:0/1256875735 learned_addr learned my addr 172.21.15.45:0/1256875735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:33.091 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36a59c700 1 -- 172.21.15.45:0/1256875735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c1087e0 con 0x7fb36c107d00 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36a59c700 1 --2- 172.21.15.45:0/1256875735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0098a0 tx=0x7fb35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d392072ce1ff286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36959a700 1 -- 172.21.15.45:0/1256875735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb35c004160 con 0x7fb36c107d00 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36959a700 1 -- 172.21.15.45:0/1256875735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c004300 con 0x7fb36c107d00 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.327+0000 7fb36959a700 1 -- 172.21.15.45:0/1256875735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb35c011b20 con 0x7fb36c107d00 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 -- 172.21.15.45:0/1256875735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 msgr2=0x7fb36c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 --2- 172.21.15.45:0/1256875735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0098a0 tx=0x7fb35c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 -- 172.21.15.45:0/1256875735 shutdown_connections 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 --2- 172.21.15.45:0/1256875735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c108120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 -- 172.21.15.45:0/1256875735 >> 172.21.15.45:0/1256875735 conn(0x7fb36c1033a0 msgr2=0x7fb36c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 -- 172.21.15.45:0/1256875735 shutdown_connections 2024-04-25T00:12:33.092 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.328+0000 7fb370c81700 1 -- 172.21.15.45:0/1256875735 wait complete. 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb370c81700 1 Processor -- start 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb370c81700 1 -- start start 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb370c81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb370c81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c10c120 con 0x7fb36c107d00 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb36a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb36a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55986/0 (socket says 172.21.15.45:55986) 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.329+0000 7fb36a59c700 1 -- 172.21.15.45:0/1891772060 learned_addr learned my addr 172.21.15.45:0/1891772060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb36a59c700 1 -- 172.21.15.45:0/1891772060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c19c090 con 0x7fb36c107d00 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb36a59c700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0117e0 tx=0x7fb35c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.093 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb35c003ba0 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c01a910 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c19c320 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb35c0248f0 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.330+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c19c780 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.331+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb35c022020 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.331+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36c063890 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.331+0000 7fb35b7fe700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 0x7fb35403b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.331+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb35c04d800 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.331+0000 7fb369d9b700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 0x7fb35403b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.332+0000 7fb369d9b700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 0x7fb35403b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.336+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb35c00b3e0 con 0x7fb36c107d00 2024-04-25T00:12:33.094 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.542+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fb36c19f210 con 0x7fb36c107d00 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.543+0000 7fb35b7fe700 1 -- 172.21.15.45:0/1891772060 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fb35c01ab30 con 0x7fb36c107d00 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 msgr2=0x7fb35403b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 0x7fb35403b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 msgr2=0x7fb36c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0117e0 tx=0x7fb35c011ba0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 shutdown_connections 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb3540391a0 0x7fb35403b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 --2- 172.21.15.45:0/1891772060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb36c107d00 0x7fb36c19bb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 >> 172.21.15.45:0/1891772060 conn(0x7fb36c1033a0 msgr2=0x7fb36c103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 shutdown_connections 2024-04-25T00:12:33.095 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:32.545+0000 7fb370c81700 1 -- 172.21.15.45:0/1891772060 wait complete. 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6d03f9700 1 Processor -- start 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6d03f9700 1 -- start start 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6d03f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6d03f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c8108660 con 0x7ff6c8107d00 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6ce195700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.371 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6ce195700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55994/0 (socket says 172.21.15.45:55994) 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.523+0000 7ff6ce195700 1 -- 172.21.15.45:0/2506654817 learned_addr learned my addr 172.21.15.45:0/2506654817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.524+0000 7ff6ce195700 1 -- 172.21.15.45:0/2506654817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c81087e0 con 0x7ff6c8107d00 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.524+0000 7ff6ce195700 1 --2- 172.21.15.45:0/2506654817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098a0 tx=0x7ff6c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f4041202a1bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.524+0000 7ff6cd193700 1 -- 172.21.15.45:0/2506654817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4004160 con 0x7ff6c8107d00 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.524+0000 7ff6cd193700 1 -- 172.21.15.45:0/2506654817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4004300 con 0x7ff6c8107d00 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.524+0000 7ff6cd193700 1 -- 172.21.15.45:0/2506654817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4011b20 con 0x7ff6c8107d00 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2506654817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 msgr2=0x7ff6c8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2506654817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40098a0 tx=0x7ff6c4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.372 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2506654817 shutdown_connections 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2506654817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c8108120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2506654817 >> 172.21.15.45:0/2506654817 conn(0x7ff6c81033a0 msgr2=0x7ff6c81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2506654817 shutdown_connections 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2506654817 wait complete. 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 Processor -- start 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- start start 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.525+0000 7ff6d03f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c810c120 con 0x7ff6c8107d00 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6ce195700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6ce195700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56004/0 (socket says 172.21.15.45:56004) 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6ce195700 1 -- 172.21.15.45:0/2881450188 learned_addr learned my addr 172.21.15.45:0/2881450188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6ce195700 1 -- 172.21.15.45:0/2881450188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c819c090 con 0x7ff6c8107d00 2024-04-25T00:12:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6ce195700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40117e0 tx=0x7ff6c4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4003bf0 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c401a910 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c819c320 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6c4024970 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.526+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c819c780 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.527+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c8063890 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.527+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff6c4022020 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.527+0000 7ff6bf7fe700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 0x7ff6b403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.527+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6c404d5b0 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.527+0000 7ff6cd994700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 0x7ff6b403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.528+0000 7ff6cd994700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 0x7ff6b403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8006fd0 tx=0x7ff6b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.533+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6c4011e20 con 0x7ff6c8107d00 2024-04-25T00:12:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.853+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff6c819f250 con 0x7ff6c8107d00 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.855+0000 7ff6bf7fe700 1 -- 172.21.15.45:0/2881450188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7ff6c400b3e0 con 0x7ff6c8107d00 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 msgr2=0x7ff6b403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 0x7ff6b403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8006fd0 tx=0x7ff6b8006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 msgr2=0x7ff6c819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40117e0 tx=0x7ff6c4011c10 comp rx=0 tx=0).stop 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 shutdown_connections 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff6b4039150 0x7ff6b403b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 --2- 172.21.15.45:0/2881450188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c8107d00 0x7ff6c819bb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 >> 172.21.15.45:0/2881450188 conn(0x7ff6c81033a0 msgr2=0x7ff6c8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 shutdown_connections 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T00:12:33.857+0000 7ff6d03f9700 1 -- 172.21.15.45:0/2881450188 wait complete. 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T00:12:34.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:34.376 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-04-25T00:12:34.409 INFO:tasks.cephadm:Fetching config... 2024-04-25T00:12:34.410 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:12:34.410 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T00:12:34.428 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T00:12:34.428 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:12:34.428 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T00:12:34.488 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T00:12:34.489 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:12:34.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T00:12:34.556 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T00:12:34.556 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:12:34.556 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T00:12:34.615 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T00:12:34.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:12:34.693 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:12:34.704 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:12:34.746 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:12:34.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:12:34.799 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc2vBiUEpDounDr4UuGe6jvbMfRMAqduTxNeidtRAHimGx+WJMdpm/lLgkZFGhRt5a6T16w7CsQ0+fbNgXMqMd/S2Q0U/PLmKx0Rxus1YanvUNoGvcTVW+8JdiJtOhxg8FGlCxQ8zvnQZVLD7R6FggSjRQAPf3dyE3VRsqHW9VvS0da1Ll3lYbEPWyjxROz2sk1x+XoXemcKrtZ0l+2Yh9N//M1W3y8EPQdWLEteklN4aPNJwKsIrQahI0TZedMLU/gmc/2Enp7/9yoBrrqy4l4TcV5TYwKHyACFONuEauXWINgGMugi0YXXjMBkrrKQ/T5j42UEMUe+AFCYr6IDUtG0uxstHBl0L9vAOvvFFcqvIcuZQ+3flMfalh4fKrlpLx+8YWQuJwYWJdVPeHXLb/IdgNRDFsgPfM2YkFXzNWgaDSEuAWow6KxXssfLiyhBqWE3JDa8Z3GizgrUO1TPIumXMadIY56ApdkLa81vFN5fYtaiOgIlidWkyKARWNRj8= ceph-24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:12:34.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T00:12:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:34 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2881450188' entity='client.admin' 2024-04-25T00:12:37.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 -- 172.21.15.45:0/1243784245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 msgr2=0x7fa5d0101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1243784245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d0101000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009910 tx=0x7fa5cc009c20 comp rx=0 tx=0).stop 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 -- 172.21.15.45:0/1243784245 shutdown_connections 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1243784245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d0101000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 -- 172.21.15.45:0/1243784245 >> 172.21.15.45:0/1243784245 conn(0x7fa5d00fc140 msgr2=0x7fa5d00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 -- 172.21.15.45:0/1243784245 shutdown_connections 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.319+0000 7fa5d862a700 1 -- 172.21.15.45:0/1243784245 wait complete. 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d862a700 1 Processor -- start 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d862a700 1 -- start start 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d862a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d862a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0106c70 con 0x7fa5d0100be0 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d63c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d63c6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50018/0 (socket says 172.21.15.45:50018) 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d63c6700 1 -- 172.21.15.45:0/1300610334 learned_addr learned my addr 172.21.15.45:0/1300610334 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d63c6700 1 -- 172.21.15.45:0/1300610334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d01978f0 con 0x7fa5d0100be0 2024-04-25T00:12:37.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.320+0000 7fa5d63c6700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc005850 tx=0x7fa5cc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.321+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc026070 con 0x7fa5d0100be0 2024-04-25T00:12:37.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.321+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc0044a0 con 0x7fa5d0100be0 2024-04-25T00:12:37.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.321+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d0197b80 con 0x7fa5d0100be0 2024-04-25T00:12:37.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.321+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5cc003ab0 con 0x7fa5d0100be0 2024-04-25T00:12:37.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.321+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d019a870 con 0x7fa5d0100be0 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa5cc029070 con 0x7fa5d0100be0 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d01073f0 con 0x7fa5d0100be0 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5c37fe700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 0x7fa5bc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa5cc01e070 con 0x7fa5d0100be0 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5d5bc5700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 0x7fa5bc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.322+0000 7fa5d5bc5700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 0x7fa5bc03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:37.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.327+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5cc013070 con 0x7fa5d0100be0 2024-04-25T00:12:37.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.536+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa5d002dc40 con 0x7fa5d0100be0 2024-04-25T00:12:37.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.538+0000 7fa5c37fe700 1 -- 172.21.15.45:0/1300610334 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa5cc030020 con 0x7fa5d0100be0 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:37.539 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 msgr2=0x7fa5bc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 0x7fa5bc03b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 msgr2=0x7fa5d01973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc005850 tx=0x7fa5cc005030 comp rx=0 tx=0).stop 2024-04-25T00:12:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 shutdown_connections 2024-04-25T00:12:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5bc039150 0x7fa5bc03b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 --2- 172.21.15.45:0/1300610334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d0100be0 0x7fa5d01973b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 >> 172.21.15.45:0/1300610334 conn(0x7fa5d00fc140 msgr2=0x7fa5d006a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 shutdown_connections 2024-04-25T00:12:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:37.541+0000 7fa5d862a700 1 -- 172.21.15.45:0/1300610334 wait complete. 2024-04-25T00:12:38.129 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T00:12:38.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T00:12:38.376 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:38.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:38 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1300610334' entity='client.admin' 2024-04-25T00:12:38.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:38.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:39.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2438154339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 msgr2=0x7faaf41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2438154339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf41037c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faae4009910 tx=0x7faae4009c20 comp rx=0 tx=0).stop 2024-04-25T00:12:39.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2438154339 shutdown_connections 2024-04-25T00:12:39.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2438154339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf41037c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2438154339 >> 172.21.15.45:0/2438154339 conn(0x7faaf40fd280 msgr2=0x7faaf40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2438154339 shutdown_connections 2024-04-25T00:12:39.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.969+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2438154339 wait complete. 2024-04-25T00:12:39.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.970+0000 7faaf9ed1700 1 Processor -- start 2024-04-25T00:12:39.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.970+0000 7faaf9ed1700 1 -- start start 2024-04-25T00:12:39.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.970+0000 7faaf9ed1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.970+0000 7faaf9ed1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4107e60 con 0x7faaf41033a0 2024-04-25T00:12:39.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.970+0000 7faaf37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50032/0 (socket says 172.21.15.45:50032) 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf37fe700 1 -- 172.21.15.45:0/2845974290 learned_addr learned my addr 172.21.15.45:0/2845974290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf37fe700 1 -- 172.21.15.45:0/2845974290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf4076430 con 0x7faaf41033a0 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf37fe700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faae4006010 tx=0x7faae4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae4027070 con 0x7faaf41033a0 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae400be40 con 0x7faaf41033a0 2024-04-25T00:12:39.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf4074620 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faae400f460 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.971+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf4074b60 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.972+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faae402a070 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.972+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf41085e0 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.972+0000 7faaf0ff9700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 0x7faadc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.972+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faae401e030 con 0x7faaf41033a0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.972+0000 7faaf2ffd700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 0x7faadc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.973+0000 7faaf2ffd700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 0x7faadc03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae8006fd0 tx=0x7faae8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:39.977+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faae4013070 con 0x7faaf41033a0 2024-04-25T00:12:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.187+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7faaf41907d0 con 0x7faadc0390f0 2024-04-25T00:12:40.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.190+0000 7faaf0ff9700 1 -- 172.21.15.45:0/2845974290 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faaf41907d0 con 0x7faadc0390f0 2024-04-25T00:12:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 msgr2=0x7faadc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 0x7faadc03b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae8006fd0 tx=0x7faae8006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 msgr2=0x7faaf4075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faae4006010 tx=0x7faae4005050 comp rx=0 tx=0).stop 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 shutdown_connections 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faadc0390f0 0x7faadc03b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 --2- 172.21.15.45:0/2845974290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf41033a0 0x7faaf4075ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 >> 172.21.15.45:0/2845974290 conn(0x7faaf40fd280 msgr2=0x7faaf40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 shutdown_connections 2024-04-25T00:12:40.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:40.192+0000 7faaf9ed1700 1 -- 172.21.15.45:0/2845974290 wait complete. 2024-04-25T00:12:40.884 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-04-25T00:12:40.885 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:12:40.885 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:12:40.903 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:12:40.904 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:12:40.960 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-04-25T00:12:40.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch host add smithi071 2024-04-25T00:12:41.206 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:41.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:41.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:41.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 -- 172.21.15.45:0/916883467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 msgr2=0x7ff028106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 --2- 172.21.15.45:0/916883467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff028106fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff018009910 tx=0x7ff018009c20 comp rx=0 tx=0).stop 2024-04-25T00:12:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 -- 172.21.15.45:0/916883467 shutdown_connections 2024-04-25T00:12:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 --2- 172.21.15.45:0/916883467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff028106fe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 -- 172.21.15.45:0/916883467 >> 172.21.15.45:0/916883467 conn(0x7ff028076690 msgr2=0x7ff028076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 -- 172.21.15.45:0/916883467 shutdown_connections 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.671+0000 7ff02ee81700 1 -- 172.21.15.45:0/916883467 wait complete. 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02ee81700 1 Processor -- start 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02ee81700 1 -- start start 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02ee81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02ee81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810b1c0 con 0x7ff028106c00 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02cc1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02cc1d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50050/0 (socket says 172.21.15.45:50050) 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02cc1d700 1 -- 172.21.15.45:0/399809005 learned_addr learned my addr 172.21.15.45:0/399809005 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02cc1d700 1 -- 172.21.15.45:0/399809005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02819fb40 con 0x7ff028106c00 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.672+0000 7ff02cc1d700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff018005850 tx=0x7ff018005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.673+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018026070 con 0x7ff028106c00 2024-04-25T00:12:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.673+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff01800bc30 con 0x7ff028106c00 2024-04-25T00:12:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.673+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02819fdd0 con 0x7ff028106c00 2024-04-25T00:12:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.673+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018003a20 con 0x7ff028106c00 2024-04-25T00:12:42.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.673+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff028199780 con 0x7ff028106c00 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff018029070 con 0x7ff028106c00 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028102200 con 0x7ff028106c00 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff025ffb700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 0x7ff01003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff01801e070 con 0x7ff028106c00 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff027fff700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 0x7ff01003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:42.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.674+0000 7ff027fff700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 0x7ff01003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff01c006fd0 tx=0x7ff01c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:42.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.679+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff018050030 con 0x7ff028106c00 2024-04-25T00:12:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:42.890+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) -- 0x7ff028063890 con 0x7ff010039150 2024-04-25T00:12:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:43 smithi045 ceph-mon[29194]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:44.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:44 smithi045 ceph-mon[29194]: Deploying cephadm binary to smithi071 2024-04-25T00:12:46.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:46 smithi045 ceph-mon[29194]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:47.212 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-04-25T00:12:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.211+0000 7ff025ffb700 1 -- 172.21.15.45:0/399809005 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff028063890 con 0x7ff010039150 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 msgr2=0x7ff01003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 0x7ff01003b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff01c006fd0 tx=0x7ff01c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 msgr2=0x7ff02819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff018005850 tx=0x7ff018005030 comp rx=0 tx=0).stop 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 shutdown_connections 2024-04-25T00:12:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff010039150 0x7ff01003b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 --2- 172.21.15.45:0/399809005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028106c00 0x7ff02819f600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 >> 172.21.15.45:0/399809005 conn(0x7ff028076690 msgr2=0x7ff0280fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 shutdown_connections 2024-04-25T00:12:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:47.214+0000 7ff02ee81700 1 -- 172.21.15.45:0/399809005 wait complete. 2024-04-25T00:12:47.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:12:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:48.447 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:48 smithi045 ceph-mon[29194]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:48.447 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:48.447 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:48 smithi045 ceph-mon[29194]: Added host smithi071 2024-04-25T00:12:48.447 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:48.447 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:49.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 -- 172.21.15.45:0/1385663307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 msgr2=0x7f7374103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 --2- 172.21.15.45:0/1385663307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374103220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7364009910 tx=0x7f7364009c20 comp rx=0 tx=0).stop 2024-04-25T00:12:49.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 -- 172.21.15.45:0/1385663307 shutdown_connections 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 --2- 172.21.15.45:0/1385663307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374103220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 -- 172.21.15.45:0/1385663307 >> 172.21.15.45:0/1385663307 conn(0x7f73740fc690 msgr2=0x7f73740feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 -- 172.21.15.45:0/1385663307 shutdown_connections 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.613+0000 7f737a36d700 1 -- 172.21.15.45:0/1385663307 wait complete. 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f737a36d700 1 Processor -- start 2024-04-25T00:12:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f737a36d700 1 -- start start 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f737a36d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f737a36d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f737410c310 con 0x7f7374102e40 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f7373fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f7373fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38796/0 (socket says 172.21.15.45:38796) 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.614+0000 7f7373fff700 1 -- 172.21.15.45:0/2685069642 learned_addr learned my addr 172.21.15.45:0/2685069642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f7373fff700 1 -- 172.21.15.45:0/2685069642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7374102990 con 0x7f7374102e40 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f7373fff700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f7364005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7364026070 con 0x7f7374102e40 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736400bc30 con 0x7f7374102e40 2024-04-25T00:12:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7364003ab0 con 0x7f7374102e40 2024-04-25T00:12:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7374100fe0 con 0x7f7374102e40 2024-04-25T00:12:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.615+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73741014a0 con 0x7f7374102e40 2024-04-25T00:12:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.616+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7364029070 con 0x7f7374102e40 2024-04-25T00:12:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.616+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737406a980 con 0x7f7374102e40 2024-04-25T00:12:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.616+0000 7f73717fa700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 0x7f735c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.616+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f736401e070 con 0x7f7374102e40 2024-04-25T00:12:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.616+0000 7f73737fe700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 0x7f735c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.617+0000 7f73737fe700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 0x7f735c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7368006fd0 tx=0x7f7368006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.621+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7364013070 con 0x7f7374102e40 2024-04-25T00:12:49.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:49.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.832+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73741016d0 con 0x7f735c0390f0 2024-04-25T00:12:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.834+0000 7f73717fa700 1 -- 172.21.15.45:0/2685069642 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f73741016d0 con 0x7f735c0390f0 2024-04-25T00:12:49.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:49.834 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-04-25T00:12:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 msgr2=0x7f735c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 0x7f735c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7368006fd0 tx=0x7f7368006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 msgr2=0x7f7374100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f7364005050 comp rx=0 tx=0).stop 2024-04-25T00:12:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 shutdown_connections 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f735c0390f0 0x7f735c03b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 --2- 172.21.15.45:0/2685069642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7374102e40 0x7f7374100aa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 >> 172.21.15.45:0/2685069642 conn(0x7f73740fc690 msgr2=0x7f7374103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 shutdown_connections 2024-04-25T00:12:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:49.836+0000 7f737a36d700 1 -- 172.21.15.45:0/2685069642 wait complete. 2024-04-25T00:12:50.446 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi122 2024-04-25T00:12:50.447 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:12:50.447 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:12:50.465 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:12:50.465 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:12:50.522 INFO:tasks.cephadm:Adding host smithi122 to orchestrator... 2024-04-25T00:12:50.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch host add smithi122 2024-04-25T00:12:50.771 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:50 smithi045 ceph-mon[29194]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:50 smithi045 ceph-mon[29194]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:12:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:50 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:52.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 -- 172.21.15.45:0/3043945774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 msgr2=0x7f9378102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:52.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 --2- 172.21.15.45:0/3043945774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f9378102280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f937000b0a0 tx=0x7f937000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 -- 172.21.15.45:0/3043945774 shutdown_connections 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 --2- 172.21.15.45:0/3043945774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f9378102280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 -- 172.21.15.45:0/3043945774 >> 172.21.15.45:0/3043945774 conn(0x7f93780fd750 msgr2=0x7f93780ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 -- 172.21.15.45:0/3043945774 shutdown_connections 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.350+0000 7f938035e700 1 -- 172.21.15.45:0/3043945774 wait complete. 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f938035e700 1 Processor -- start 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f938035e700 1 -- start start 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f938035e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f938035e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f937810c270 con 0x7f9378101ea0 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f937e0fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f937e0fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38820/0 (socket says 172.21.15.45:38820) 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f937e0fa700 1 -- 172.21.15.45:0/39175562 learned_addr learned my addr 172.21.15.45:0/39175562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f937e0fa700 1 -- 172.21.15.45:0/39175562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9378197be0 con 0x7f9378101ea0 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.351+0000 7f937e0fa700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9370015040 tx=0x7f9370003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.352+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9370017040 con 0x7f9378101ea0 2024-04-25T00:12:52.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.352+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9370022070 con 0x7f9378101ea0 2024-04-25T00:12:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.352+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9378197e70 con 0x7f9378101ea0 2024-04-25T00:12:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.352+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f937001d5d0 con 0x7f9378101ea0 2024-04-25T00:12:52.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.352+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937819bf70 con 0x7f9378101ea0 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9370004000 con 0x7f9378101ea0 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9378103380 con 0x7f9378101ea0 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f93777fe700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 0x7f936003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f937000f050 con 0x7f9378101ea0 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f937d8f9700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 0x7f936003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:52.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.353+0000 7f937d8f9700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 0x7f936003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.358+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9370007e20 con 0x7f9378101ea0 2024-04-25T00:12:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:52.568+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}) -- 0x7f9378063890 con 0x7f93600390f0 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:54.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:54.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:12:54.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:12:54.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:54.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:54.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:54 smithi045 ceph-mon[29194]: Deploying cephadm binary to smithi122 2024-04-25T00:12:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:54 smithi045 ceph-mon[29194]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:54 smithi045 ceph-mon[29194]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:12:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:54 smithi045 ceph-mon[29194]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:12:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.862+0000 7f93777fe700 1 -- 172.21.15.45:0/39175562 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9378063890 con 0x7f93600390f0 2024-04-25T00:12:56.864 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi122' with addr '172.21.15.122' 2024-04-25T00:12:56.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 msgr2=0x7f936003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 0x7f936003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 msgr2=0x7f93781976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9370015040 tx=0x7f9370003d50 comp rx=0 tx=0).stop 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 shutdown_connections 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93600390f0 0x7f936003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 --2- 172.21.15.45:0/39175562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9378101ea0 0x7f93781976a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 >> 172.21.15.45:0/39175562 conn(0x7f93780fd750 msgr2=0x7f9378103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 shutdown_connections 2024-04-25T00:12:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:56.865+0000 7f938035e700 1 -- 172.21.15.45:0/39175562 wait complete. 2024-04-25T00:12:57.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:56 smithi045 ceph-mon[29194]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:57.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:57.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:57.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:12:57.783 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:58.220 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:57 smithi045 ceph-mon[29194]: Added host smithi122 2024-04-25T00:12:58.220 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:58.220 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:12:59.180 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:12:58 smithi045 ceph-mon[29194]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 -- 172.21.15.45:0/1959455837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 msgr2=0x7f8eac1032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/1959455837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac1032a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009910 tx=0x7f8e9c009c20 comp rx=0 tx=0).stop 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 -- 172.21.15.45:0/1959455837 shutdown_connections 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/1959455837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac1032a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 -- 172.21.15.45:0/1959455837 >> 172.21.15.45:0/1959455837 conn(0x7f8eac0fc630 msgr2=0x7f8eac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 -- 172.21.15.45:0/1959455837 shutdown_connections 2024-04-25T00:12:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.402+0000 7f8eb1a15700 1 -- 172.21.15.45:0/1959455837 wait complete. 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.403+0000 7f8eb1a15700 1 Processor -- start 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.403+0000 7f8eb1a15700 1 -- start start 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.403+0000 7f8eb1a15700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.403+0000 7f8eb1a15700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac10c2e0 con 0x7f8eac102ec0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.403+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34100/0 (socket says 172.21.15.45:34100) 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eaaffd700 1 -- 172.21.15.45:0/2964105318 learned_addr learned my addr 172.21.15.45:0/2964105318 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eaaffd700 1 -- 172.21.15.45:0/2964105318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac19d970 con 0x7f8eac102ec0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eaaffd700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006010 tx=0x7f8e9c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c026070 con 0x7f8eac102ec0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9c00bc30 con 0x7f8eac102ec0 2024-04-25T00:12:59.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac19dc00 con 0x7f8eac102ec0 2024-04-25T00:12:59.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e9c003ab0 con 0x7f8eac102ec0 2024-04-25T00:12:59.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.404+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1975f0 con 0x7f8eac102ec0 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.405+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8e9c029070 con 0x7f8eac102ec0 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.405+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eac06a8c0 con 0x7f8eac102ec0 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.405+0000 7f8eb0a13700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 0x7f8e9803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.405+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8e9c01e070 con 0x7f8eac102ec0 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.405+0000 7f8eaa7fc700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 0x7f8e9803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:59.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.406+0000 7f8eaa7fc700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 0x7f8e9803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:59.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.410+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e9c013070 con 0x7f8eac102ec0 2024-04-25T00:12:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.627+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8eac198b20 con 0x7f8e980390f0 2024-04-25T00:12:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.628+0000 7f8eb0a13700 1 -- 172.21.15.45:0/2964105318 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8eac198b20 con 0x7f8e980390f0 2024-04-25T00:12:59.629 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:12:59.629 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}] 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 msgr2=0x7f8e9803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 0x7f8e9803b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8e94006fd0 tx=0x7f8e94006e40 comp rx=0 tx=0).stop 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 msgr2=0x7f8eac19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006010 tx=0x7f8e9c005050 comp rx=0 tx=0).stop 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 shutdown_connections 2024-04-25T00:12:59.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8e980390f0 0x7f8e9803b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 --2- 172.21.15.45:0/2964105318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eac102ec0 0x7f8eac19d430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:59.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 >> 172.21.15.45:0/2964105318 conn(0x7f8eac0fc630 msgr2=0x7f8eac103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:59.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 shutdown_connections 2024-04-25T00:12:59.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:12:59.630+0000 7f8eb1a15700 1 -- 172.21.15.45:0/2964105318 wait complete. 2024-04-25T00:13:00.226 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T00:13:00.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T00:13:00.468 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:00.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:00 smithi045 ceph-mon[29194]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:00.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:00.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:00 smithi045 ceph-mon[29194]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:13:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 -- 172.21.15.45:0/4006209022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 msgr2=0x7fabf0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/4006209022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf0102280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009910 tx=0x7fabe4009c20 comp rx=0 tx=0).stop 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 -- 172.21.15.45:0/4006209022 shutdown_connections 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/4006209022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf0102280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 -- 172.21.15.45:0/4006209022 >> 172.21.15.45:0/4006209022 conn(0x7fabf00fd770 msgr2=0x7fabf00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 -- 172.21.15.45:0/4006209022 shutdown_connections 2024-04-25T00:13:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.013+0000 7fabf6c5b700 1 -- 172.21.15.45:0/4006209022 wait complete. 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf6c5b700 1 Processor -- start 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf6c5b700 1 -- start start 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf6c5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf6c5b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fabf010c270 con 0x7fabf0101ea0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf49f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf49f7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34118/0 (socket says 172.21.15.45:34118) 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf49f7700 1 -- 172.21.15.45:0/1201993210 learned_addr learned my addr 172.21.15.45:0/1201993210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf49f7700 1 -- 172.21.15.45:0/1201993210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf019d920 con 0x7fabf0101ea0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabf49f7700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005850 tx=0x7fabe4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.014+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4026070 con 0x7fabf0101ea0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.015+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe400bbd0 con 0x7fabf0101ea0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.015+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf019dc10 con 0x7fabf0101ea0 2024-04-25T00:13:02.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.015+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabe4003970 con 0x7fabf0101ea0 2024-04-25T00:13:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.015+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf01975a0 con 0x7fabf0101ea0 2024-04-25T00:13:02.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fabe4029070 con 0x7fabf0101ea0 2024-04-25T00:13:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf0103380 con 0x7fabf0101ea0 2024-04-25T00:13:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabedffb700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 0x7fabe003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabe401e070 con 0x7fabf0101ea0 2024-04-25T00:13:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabeffff700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 0x7fabe003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.016+0000 7fabeffff700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 0x7fabe003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.021+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabe4013070 con 0x7fabf0101ea0 2024-04-25T00:13:02.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.232+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fabf0198ad0 con 0x7fabf0101ea0 2024-04-25T00:13:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.450+0000 7fabedffb700 1 -- 172.21.15.45:0/1201993210 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fabe4030020 con 0x7fabf0101ea0 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 msgr2=0x7fabe003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 0x7fabe003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc006e40 comp rx=0 tx=0).stop 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 msgr2=0x7fabf019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005850 tx=0x7fabe4005030 comp rx=0 tx=0).stop 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 shutdown_connections 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fabe0039150 0x7fabe003b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 --2- 172.21.15.45:0/1201993210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabf0101ea0 0x7fabf019d3e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 >> 172.21.15.45:0/1201993210 conn(0x7fabf00fd770 msgr2=0x7fabf0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 shutdown_connections 2024-04-25T00:13:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:02.452+0000 7fabf6c5b700 1 -- 172.21.15.45:0/1201993210 wait complete. 2024-04-25T00:13:02.454 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-04-25T00:13:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:02 smithi045 ceph-mon[29194]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:13:03.114 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-04-25T00:13:03.114 INFO:tasks.cephadm:Adding mon.b on smithi071 2024-04-25T00:13:03.114 INFO:tasks.cephadm:Adding mon.c on smithi122 2024-04-25T00:13:03.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c' 2024-04-25T00:13:03.363 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:03.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:03.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:04.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 -- 172.21.15.122:0/550318089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 msgr2=0x7f08ac101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 --2- 172.21.15.122:0/550318089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac101a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009910 tx=0x7f08a0009c20 comp rx=0 tx=0).stop 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 -- 172.21.15.122:0/550318089 shutdown_connections 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 --2- 172.21.15.122:0/550318089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac101a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 -- 172.21.15.122:0/550318089 >> 172.21.15.122:0/550318089 conn(0x7f08ac078160 msgr2=0x7f08ac078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 -- 172.21.15.122:0/550318089 shutdown_connections 2024-04-25T00:13:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.805+0000 7f08b0c77700 1 -- 172.21.15.122:0/550318089 wait complete. 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08b0c77700 1 Processor -- start 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08b0c77700 1 -- start start 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08b0c77700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08b0c77700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac10b1f0 con 0x7f08ac101690 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08aa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08aa59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.122:42362/0 (socket says 172.21.15.122:42362) 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.806+0000 7f08aa59c700 1 -- 172.21.15.122:0/414452861 learned_addr learned my addr 172.21.15.122:0/414452861 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f08aa59c700 1 -- 172.21.15.122:0/414452861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac196b60 con 0x7f08ac101690 2024-04-25T00:13:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f08aa59c700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006010 tx=0x7f08a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0026070 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac196e50 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a000bbd0 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac19aef0 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.807+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a0003980 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.808+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f08a0029070 con 0x7f08ac101690 2024-04-25T00:13:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.808+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac102440 con 0x7f08ac101690 2024-04-25T00:13:04.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.808+0000 7f089b7fe700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 0x7f089403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:04.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.808+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f08a001e070 con 0x7f08ac101690 2024-04-25T00:13:04.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.808+0000 7f08a9d9b700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 0x7f089403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:04.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.809+0000 7f08a9d9b700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 0x7f089403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f089c006fd0 tx=0x7f089c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:04.813+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08a0013070 con 0x7f08ac101690 2024-04-25T00:13:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:04 smithi045 ceph-mon[29194]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:04.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:04 smithi045 ceph-mon[29194]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:13:05.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.019+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}) -- 0x7f08ac197980 con 0x7f08940390f0 2024-04-25T00:13:05.026 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mon update... 2024-04-25T00:13:05.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.025+0000 7f089b7fe700 1 -- 172.21.15.122:0/414452861 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f08ac197980 con 0x7f08940390f0 2024-04-25T00:13:05.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 msgr2=0x7f089403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 0x7f089403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f089c006fd0 tx=0x7f089c006e40 comp rx=0 tx=0).stop 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 msgr2=0x7f08ac196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006010 tx=0x7f08a0005030 comp rx=0 tx=0).stop 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 shutdown_connections 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f08940390f0 0x7f089403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 --2- 172.21.15.122:0/414452861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08ac101690 0x7f08ac196620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 >> 172.21.15.122:0/414452861 conn(0x7f08ac078160 msgr2=0x7f08ac1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 shutdown_connections 2024-04-25T00:13:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:05.028+0000 7f08b0c77700 1 -- 172.21.15.122:0/414452861 wait complete. 2024-04-25T00:13:05.620 DEBUG:teuthology.orchestra.run.smithi071:mon.b> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T00:13:05.623 DEBUG:teuthology.orchestra.run.smithi122:mon.c> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T00:13:05.625 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:13:05.625 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:13:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c;count:3 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:06 smithi045 ceph-mon[29194]: Deploying daemon mon.c on smithi122 2024-04-25T00:13:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:07 smithi045 ceph-mon[29194]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:09 smithi045 ceph-mon[29194]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 -- 172.21.15.122:0/2212651851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 msgr2=0x7fc3f4104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 --2- 172.21.15.122:0/2212651851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f4104490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009910 tx=0x7fc3e8009c20 comp rx=0 tx=0).stop 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 -- 172.21.15.122:0/2212651851 shutdown_connections 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 --2- 172.21.15.122:0/2212651851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f4104490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 -- 172.21.15.122:0/2212651851 >> 172.21.15.122:0/2212651851 conn(0x7fc3f40ff980 msgr2=0x7fc3f4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 -- 172.21.15.122:0/2212651851 shutdown_connections 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.462+0000 7fc3faa53700 1 -- 172.21.15.122:0/2212651851 wait complete. 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.463+0000 7fc3faa53700 1 Processor -- start 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.463+0000 7fc3faa53700 1 -- start start 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.463+0000 7fc3faa53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.463+0000 7fc3faa53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f410e480 con 0x7fc3f41040b0 2024-04-25T00:13:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.122:42376/0 (socket says 172.21.15.122:42376) 2024-04-25T00:13:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f3fff700 1 -- 172.21.15.122:0/958232602 learned_addr learned my addr 172.21.15.122:0/958232602 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:13:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f3fff700 1 -- 172.21.15.122:0/958232602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f419fbe0 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f3fff700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005850 tx=0x7fc3e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e8026070 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f419fed0 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e800bbd0 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e8003970 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.464+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f4199940 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.465+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc3e8029070 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.465+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f4105590 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.465+0000 7fc3f17fa700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 0x7fc3dc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.465+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc3e801e070 con 0x7fc3f41040b0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.466+0000 7fc3f37fe700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 0x7fc3dc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.466+0000 7fc3f37fe700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 0x7fc3dc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006fd0 tx=0x7fc3e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:09.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.471+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3e8013070 con 0x7fc3f41040b0 2024-04-25T00:13:09.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.771+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc3f40008d0 con 0x7fc3f41040b0 2024-04-25T00:13:09.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.772+0000 7fc3f17fa700 1 -- 172.21.15.122:0/958232602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fc3e8030020 con 0x7fc3f41040b0 2024-04-25T00:13:09.772 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:13:09.772 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","modified":"2024-04-25T00:11:28.889760Z","created":"2024-04-25T00:11:28.889760Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 msgr2=0x7fc3dc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 0x7fc3dc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006fd0 tx=0x7fc3e4006e40 comp rx=0 tx=0).stop 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 msgr2=0x7fc3f419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005850 tx=0x7fc3e8005030 comp rx=0 tx=0).stop 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 shutdown_connections 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc3dc0390f0 0x7fc3dc03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 --2- 172.21.15.122:0/958232602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f41040b0 0x7fc3f419f6a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:09.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 >> 172.21.15.122:0/958232602 conn(0x7fc3f40ff980 msgr2=0x7fc3f4105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:09.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 shutdown_connections 2024-04-25T00:13:09.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:09.775+0000 7fc3faa53700 1 -- 172.21.15.122:0/958232602 wait complete. 2024-04-25T00:13:09.776 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 1 2024-04-25T00:13:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:10 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/958232602' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:10 smithi122 podman[32940]: 2024-04-25 00:13:10.772309594 +0000 UTC m=+0.122120644 container create d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:13:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:10 smithi122 podman[32940]: 2024-04-25 00:13:10.991477873 +0000 UTC m=+0.341288924 container init d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:10 smithi122 podman[32940]: 2024-04-25 00:13:10.994470989 +0000 UTC m=+0.344282032 container start d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pidfile_write: ignore empty --pid-file 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: load: jerasure load: lrc 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Git sha 0 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: DB SUMMARY 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: DB Session ID: GHRHREZH2O00I11BSQ5O 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:13:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.env: 0x55fd00d9dc60 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.info_log: 0x55fd0267daa0 2024-04-25T00:13:11.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.statistics: (nil) 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.use_fsync: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.db_log_dir: 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.wal_dir: 2024-04-25T00:13:11.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.write_buffer_manager: 0x55fd026501e0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:13:11.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.unordered_write: 0 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.row_cache: None 2024-04-25T00:13:11.311 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.wal_filter: None 2024-04-25T00:13:11.312 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.wal_compression: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:13:11.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:13:11.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:13:11.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_open_files: -1 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Compression algorithms supported: 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kZSTD supported: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:13:11.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.merge_operator: 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_filter: None 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:13:11.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd0267dba0) 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_type: 0 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: data_block_index_type: 0 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_shortening: 1 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: checksum: 4 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: no_block_cache: 0 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache: 0x55fd02681090 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_options: 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: capacity : 536870912 2024-04-25T00:13:11.318 INFO:journalctl@ceph.mon.c.smithi122.stdout: num_shard_bits : 4 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: strict_capacity_limit : 0 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_compressed: (nil) 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: persistent_cache: (nil) 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_size: 4096 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_size_deviation: 10 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_restart_interval: 16 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_block_restart_interval: 1 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: metadata_block_size: 4096 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: partition_filters: 0 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: use_delta_encoding: 1 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: filter_policy: bloomfilter 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: whole_key_filtering: 1 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: verify_compression: 0 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: format_version: 5 2024-04-25T00:13:11.319 INFO:journalctl@ceph.mon.c.smithi122.stdout: enable_index_compression: 1 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_align: 0 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout: max_auto_readahead_size: 262144 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout: prepopulate_block_cache: 0 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression: NoCompression 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:13:11.320 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.num_levels: 7 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:13:11.321 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:13:11.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:13:11.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:13:11.324 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.ttl: 2592000 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:13:11.325 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 09ad8fb7-2d44-480e-be67-33d462ec1a54 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003991024561, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T00:13:11.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003991025206, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003991, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "09ad8fb7-2d44-480e-be67-33d462ec1a54", "db_session_id": "GHRHREZH2O00I11BSQ5O", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003991025310, "job": 1, "event": "recovery_finished"} 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fd02772000 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: DB pointer 0x55fd0275c000 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** DB Stats ** 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:13:11.327 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** Compaction Stats [default] ** 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** Compaction Stats [default] ** 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.328 INFO:journalctl@ceph.mon.c.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Block cache BinnedLRUCache@0x55fd02681090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.329 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: using public_addr v2:172.21.15.122:0/0 -> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: starting mon.c rank -1 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(???) e0 preinit fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: e1 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: legacy client fscid: -1 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout: No filesystems configured 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T00:13:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Deploying cephadm binary to smithi122 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Added host smithi122 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.332 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c;count:3 2024-04-25T00:13:11.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: Deploying daemon mon.c on smithi122 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/958232602' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 ceph-mon[32959]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 bash[32940]: d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 2024-04-25T00:13:11.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:11 smithi122 systemd[1]: Started Ceph mon.c for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:13:11.566 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:13:11.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:13:11.798 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:16.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.250+0000 7f851ed85700 1 -- 172.21.15.122:0/4151580547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 msgr2=0x7f8508007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.250+0000 7f851ed85700 1 --2- 172.21.15.122:0/4151580547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f8508007c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8518100b70 tx=0x7f850c00feb0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.250+0000 7f851ed85700 1 -- 172.21.15.122:0/4151580547 shutdown_connections 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.250+0000 7f851ed85700 1 --2- 172.21.15.122:0/4151580547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f8508007c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.250+0000 7f851ed85700 1 -- 172.21.15.122:0/4151580547 >> 172.21.15.122:0/4151580547 conn(0x7f85180fc770 msgr2=0x7f85180feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.251+0000 7f851ed85700 1 -- 172.21.15.122:0/4151580547 shutdown_connections 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.251+0000 7f851ed85700 1 -- 172.21.15.122:0/4151580547 wait complete. 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.251+0000 7f851ed85700 1 Processor -- start 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851ed85700 1 -- start start 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851ed85700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f851819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851ed85700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851ed85700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8518107ca0 con 0x7f85080057d0 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851ed85700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8518107b20 con 0x7f851819fcc0 2024-04-25T00:13:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48778/0 (socket says 172.21.15.122:48778) 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 -- 172.21.15.122:0/3366720323 learned_addr learned my addr 172.21.15.122:0/3366720323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851cb21700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f851819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 -- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 msgr2=0x7f8518199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 -- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 msgr2=0x7f8518199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:13:16.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f8517fff700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851cb21700 1 -- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 msgr2=0x7f8518199940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.252+0000 7f851cb21700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f851cb21700 1 -- 172.21.15.122:0/3366720323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181a0300 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f851cb21700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f851819f780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f850c0059d0 tx=0x7f850c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f850c004b20 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c013e40 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8518199e80 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f850c011cd0 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.253+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851819a410 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.254+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f850c023070 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.255+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8518102380 con 0x7f85080057d0 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.255+0000 7f8515ffb700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 0x7f84fc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.255+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f850c020050 con 0x7f85080057d0 2024-04-25T00:13:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.255+0000 7f8517fff700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 0x7f84fc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.255+0000 7f8517fff700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 0x7f84fc03b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8504009180 tx=0x7f8504008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:16.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.260+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f850c031020 con 0x7f85080057d0 2024-04-25T00:13:16.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:16.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: mon.a calling monitor election 2024-04-25T00:13:16.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: mon.c calling monitor election 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: monmap epoch 2 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: last_changed 2024-04-25T00:13:11.049547+0000 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: min_mon_release 19 (squid) 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: election_strategy: 1 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: fsmap 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: mgrmap e13: x(active, since 50s) 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: overall HEALTH_OK 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:16.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:16 smithi045 ceph-mon[29194]: Deploying daemon mon.b on smithi071 2024-04-25T00:13:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.543+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f851819ae70 con 0x7f85080057d0 2024-04-25T00:13:16.543 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: mon.a calling monitor election 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: mon.c calling monitor election 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: monmap epoch 2 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:16.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: last_changed 2024-04-25T00:13:11.049547+0000 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: min_mon_release 19 (squid) 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: election_strategy: 1 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: fsmap 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: mgrmap e13: x(active, since 50s) 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: overall HEALTH_OK 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:16.545 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:16 smithi122 ceph-mon[32959]: Deploying daemon mon.b on smithi071 2024-04-25T00:13:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.543+0000 7f8515ffb700 1 -- 172.21.15.122:0/3366720323 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f850c028030 con 0x7f85080057d0 2024-04-25T00:13:16.546 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:13:16.546 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","modified":"2024-04-25T00:13:11.049547Z","created":"2024-04-25T00:11:28.889760Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:13:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 msgr2=0x7f84fc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 0x7f84fc03b3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8504009180 tx=0x7f8504008040 comp rx=0 tx=0).stop 2024-04-25T00:13:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 msgr2=0x7f851819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f851819f780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f850c0059d0 tx=0x7f850c0057d0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 shutdown_connections 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f84fc038ee0 0x7f84fc03b3a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85080057d0 0x7f851819f780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 --2- 172.21.15.122:0/3366720323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f851819fcc0 0x7f8518199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 >> 172.21.15.122:0/3366720323 conn(0x7f85180fc770 msgr2=0x7f85180ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 shutdown_connections 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:16.545+0000 7f851ed85700 1 -- 172.21.15.122:0/3366720323 wait complete. 2024-04-25T00:13:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 2 2024-04-25T00:13:17.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:17 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/3366720323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:17.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:17 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/3366720323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:18.248 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:13:18.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:13:18.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:18 smithi045 ceph-mon[29194]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:18.494 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:18 smithi122 ceph-mon[32959]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:19.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T00:13:19.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: e1 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: legacy client fscid: -1 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout: No filesystems configured 2024-04-25T00:13:19.731 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:20.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Deploying cephadm binary to smithi122 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Added host smithi122 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1201993210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:20.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c;count:3 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:20.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Deploying daemon mon.c on smithi122 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/958232602' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.a calling monitor election 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.c calling monitor election 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.123 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: monmap epoch 2 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: last_changed 2024-04-25T00:13:11.049547+0000 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: min_mon_release 19 (squid) 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: election_strategy: 1 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:20.124 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: fsmap 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mgrmap e13: x(active, since 50s) 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: Deploying daemon mon.b on smithi071 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/3366720323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:20.125 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 ceph-mon[32667]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:13:20.126 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 bash[32648]: bfaa5703b9bf86dbbc3ab02812ed59f6d2365c2825baac8c0248e5138b68d656 2024-04-25T00:13:20.126 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:19 smithi071 systemd[1]: Started Ceph mon.b for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:13:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: mon.a calling monitor election 2024-04-25T00:13:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: mon.c calling monitor election 2024-04-25T00:13:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: mon.b calling monitor election 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: monmap epoch 3 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: last_changed 2024-04-25T00:13:19.736580+0000 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:25.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: min_mon_release 19 (squid) 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: election_strategy: 1 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: fsmap 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: mgrmap e13: x(active, since 59s) 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: overall HEALTH_OK 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: mon.a calling monitor election 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: mon.c calling monitor election 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: mon.b calling monitor election 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: monmap epoch 3 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: last_changed 2024-04-25T00:13:19.736580+0000 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: min_mon_release 19 (squid) 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: election_strategy: 1 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:25.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: fsmap 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: mgrmap e13: x(active, since 59s) 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:13:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: mon.a calling monitor election 2024-04-25T00:13:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: mon.c calling monitor election 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: mon.b calling monitor election 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: monmap epoch 3 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: last_changed 2024-04-25T00:13:19.736580+0000 2024-04-25T00:13:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: created 2024-04-25T00:11:28.889760+0000 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: min_mon_release 19 (squid) 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: election_strategy: 1 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: 2: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.b 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: fsmap 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: mgrmap e13: x(active, since 59s) 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: overall HEALTH_OK 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:25.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:26.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 -- 172.21.15.122:0/2230611566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00062a0 msgr2=0x7f52a0008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:26.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 --2- 172.21.15.122:0/2230611566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00062a0 0x7f52a0008710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52b4101470 tx=0x7f52a400fe90 comp rx=0 tx=0).stop 2024-04-25T00:13:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 -- 172.21.15.122:0/2230611566 shutdown_connections 2024-04-25T00:13:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 --2- 172.21.15.122:0/2230611566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a00062a0 0x7f52a0008710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 -- 172.21.15.122:0/2230611566 >> 172.21.15.122:0/2230611566 conn(0x7f52b40770b0 msgr2=0x7f52b40774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 -- 172.21.15.122:0/2230611566 shutdown_connections 2024-04-25T00:13:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 -- 172.21.15.122:0/2230611566 wait complete. 2024-04-25T00:13:26.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:25 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:25 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:26.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:25 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.002+0000 7f52b944c700 1 Processor -- start 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 -- start start 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 0x7f52b41a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4109da0 con 0x7f52b41a3c80 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4109c20 con 0x7f52b4101b20 2024-04-25T00:13:26.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b944c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4109f20 con 0x7f52a00062a0 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37222/0 (socket says 172.21.15.122:37222) 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 -- 172.21.15.122:0/1519858112 learned_addr learned my addr 172.21.15.122:0/1519858112 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 0x7f52b41a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 msgr2=0x7f52b41015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 msgr2=0x7f52b41015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52aa7fc700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b2ffd700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52aa7fc700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 msgr2=0x7f52b4101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52aa7fc700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 msgr2=0x7f52b4101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52aa7fc700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52aa7fc700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 msgr2=0x7f52b41015e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:13:26.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 msgr2=0x7f52b4101fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.003+0000 7f52b37fe700 1 -- 172.21.15.122:0/1519858112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41a67a0 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b37fe700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 0x7f52b41a6070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bd50 tx=0x7f52ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac019070 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac00cd10 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0127e0 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b41a6a90 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.004+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a6fd0 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f52ac01c070 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b4063890 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b0ff9700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 0x7f529403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f52ac04bb80 con 0x7f52b41a3c80 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b2ffd700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 0x7f529403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.005+0000 7f52b2ffd700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 0x7f529403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52a400f550 tx=0x7f52a400f4e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:26.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.010+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f52ac007290 con 0x7f52b41a3c80 2024-04-25T00:13:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:25 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:25 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:25 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:13:26.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.308+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f52b402dd90 con 0x7f52b41a3c80 2024-04-25T00:13:26.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.308+0000 7f52b0ff9700 1 -- 172.21.15.122:0/1519858112 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f52ac00e090 con 0x7f52b41a3c80 2024-04-25T00:13:26.309 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-25T00:13:26.309 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","modified":"2024-04-25T00:13:19.736580Z","created":"2024-04-25T00:11:28.889760Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 msgr2=0x7f529403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 0x7f529403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52a400f550 tx=0x7f52a400f4e0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 msgr2=0x7f52b41a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 0x7f52b41a6070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bd50 tx=0x7f52ac00be30 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 shutdown_connections 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52a00062a0 0x7f52b41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5294039580 0x7f529403ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b41a3c80 0x7f52b41a6070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 --2- 172.21.15.122:0/1519858112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52b4101b20 0x7f52b4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 >> 172.21.15.122:0/1519858112 conn(0x7f52b40770b0 msgr2=0x7f52b40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 shutdown_connections 2024-04-25T00:13:26.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:26.311+0000 7f52b944c700 1 -- 172.21.15.122:0/1519858112 wait complete. 2024-04-25T00:13:26.313 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 3 2024-04-25T00:13:27.009 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T00:13:27.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T00:13:27.260 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/1519858112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/1519858112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/1519858112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:13:27.383 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:27.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:13:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:28 smithi122 ceph-mon[32959]: Reconfiguring daemon mon.a on smithi045 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:13:28.325 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:28 smithi045 ceph-mon[29194]: Reconfiguring daemon mon.a on smithi045 2024-04-25T00:13:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:13:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:28 smithi071 ceph-mon[32667]: Reconfiguring daemon mon.a on smithi045 2024-04-25T00:13:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:29.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:29.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2264948436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e0102e90 msgr2=0x7ff3e0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2264948436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e0102e90 0x7ff3e0106ec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009990 tx=0x7ff3d0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2264948436 shutdown_connections 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2264948436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e01075a0 0x7ff3e010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2264948436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01024e0 0x7ff3e01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2264948436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e0102e90 0x7ff3e0106ec0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2264948436 >> 172.21.15.45:0/2264948436 conn(0x7ff3e00793e0 msgr2=0x7ff3e00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2264948436 shutdown_connections 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.856+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2264948436 wait complete. 2024-04-25T00:13:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 Processor -- start 2024-04-25T00:13:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 -- start start 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 0x7ff3e0074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01075a0 0x7ff3e0074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e01125c0 con 0x7ff3e01024e0 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112440 con 0x7ff3e01075a0 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e7a24700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112740 con 0x7ff3e0102e90 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 -- 172.21.15.45:0/2001733816 learned_addr learned my addr 172.21.15.45:0/2001733816 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e5fc1700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01075a0 0x7ff3e0074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e4fbf700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 0x7ff3e0074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 -- 172.21.15.45:0/2001733816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 msgr2=0x7ff3e0074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 0x7ff3e0074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 -- 172.21.15.45:0/2001733816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01075a0 msgr2=0x7ff3e0074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01075a0 0x7ff3e0074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 -- 172.21.15.45:0/2001733816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e01a6f70 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e4fbf700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 0x7ff3e0074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.857+0000 7ff3e57c0700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00d950 tx=0x7ff3dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.858+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc016070 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.858+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3dc009e20 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.858+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e01a7260 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.858+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00f620 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.858+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e01a77f0 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff3dc019070 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e0068f20 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3d67fc700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 0x7ff3cc03b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff3dc04be90 con 0x7ff3e01024e0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3e4fbf700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 0x7ff3cc03b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.859+0000 7ff3e4fbf700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 0x7ff3cc03b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0108ad0 tx=0x7ff3d000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:29.864+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff3dc010780 con 0x7ff3e01024e0 2024-04-25T00:13:30.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.069+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff3e002db30 con 0x7ff3e01024e0 2024-04-25T00:13:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.069+0000 7ff3d67fc700 1 -- 172.21.15.45:0/2001733816 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ff3dc00acf0 con 0x7ff3e01024e0 2024-04-25T00:13:30.070 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:30.070 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T00:13:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:13:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 msgr2=0x7ff3cc03b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 0x7ff3cc03b7f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0108ad0 tx=0x7ff3d000b540 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 msgr2=0x7ff3e0074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00d950 tx=0x7ff3dc00dc60 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 shutdown_connections 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3cc039330 0x7ff3cc03b7f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3e01075a0 0x7ff3e0074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e01024e0 0x7ff3e0074530 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 --2- 172.21.15.45:0/2001733816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3e0102e90 0x7ff3e0074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 >> 172.21.15.45:0/2001733816 conn(0x7ff3e00793e0 msgr2=0x7ff3e0100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 shutdown_connections 2024-04-25T00:13:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:30.071+0000 7ff3e7a24700 1 -- 172.21.15.45:0/2001733816 wait complete. 2024-04-25T00:13:30.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:30 smithi122 ceph-mon[32959]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:13:30.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:30 smithi122 ceph-mon[32959]: Reconfiguring daemon mon.b on smithi071 2024-04-25T00:13:30.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:30 smithi071 ceph-mon[32667]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:13:30.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:30 smithi071 ceph-mon[32667]: Reconfiguring daemon mon.b on smithi071 2024-04-25T00:13:30.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:30 smithi045 ceph-mon[29194]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:13:30.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:30 smithi045 ceph-mon[29194]: Reconfiguring daemon mon.b on smithi071 2024-04-25T00:13:30.665 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T00:13:30.665 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:13:30.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:13:30.693 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:13:30.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:30.758 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:13:30.759 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:13:30.787 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:13:30.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:30.852 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:13:30.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:13:30.877 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:13:30.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:13:30.941 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-04-25T00:13:30.942 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-04-25T00:13:30.942 INFO:tasks.cephadm:Adding mgr.z on smithi122 2024-04-25T00:13:30.942 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi045=x;smithi071=y;smithi122=z' 2024-04-25T00:13:31.201 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2001733816' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:31.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2001733816' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:31.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:31.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2001733816' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:13:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:31.593 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:32 smithi122 ceph-mon[32959]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:13:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:32 smithi122 ceph-mon[32959]: Reconfiguring daemon mon.c on smithi122 2024-04-25T00:13:32.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:32 smithi071 ceph-mon[32667]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:13:32.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:32 smithi071 ceph-mon[32667]: Reconfiguring daemon mon.c on smithi122 2024-04-25T00:13:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:32 smithi045 ceph-mon[29194]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:13:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:32 smithi045 ceph-mon[29194]: Reconfiguring daemon mon.c on smithi122 2024-04-25T00:13:33.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:33 smithi122 ceph-mon[32959]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:33.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:33 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:33 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:33 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:33 smithi071 ceph-mon[32667]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:33 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:33 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:33 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:33 smithi045 ceph-mon[29194]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:33 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:33 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:33 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:33.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 -- 172.21.15.122:0/1872449865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 msgr2=0x7fba501027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 --2- 172.21.15.122:0/1872449865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba501027f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba40009990 tx=0x7fba40009ca0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 -- 172.21.15.122:0/1872449865 shutdown_connections 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 --2- 172.21.15.122:0/1872449865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 0x7fba501095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 --2- 172.21.15.122:0/1872449865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba501015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 --2- 172.21.15.122:0/1872449865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba501027f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 -- 172.21.15.122:0/1872449865 >> 172.21.15.122:0/1872449865 conn(0x7fba500fc730 msgr2=0x7fba500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 -- 172.21.15.122:0/1872449865 shutdown_connections 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.457+0000 7fba569aa700 1 -- 172.21.15.122:0/1872449865 wait complete. 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 Processor -- start 2024-04-25T00:13:33.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 -- start start 2024-04-25T00:13:33.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba50197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 0x7fba5019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010ce10 con 0x7fba50101190 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010cc90 con 0x7fba50102d30 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.458+0000 7fba569aa700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba5010cf90 con 0x7fba50102390 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43744/0 (socket says 172.21.15.122:43744) 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 -- 172.21.15.122:0/84758060 learned_addr learned my addr 172.21.15.122:0/84758060 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba54f47700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 0x7fba5019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4ffff700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba50197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 -- 172.21.15.122:0/84758060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 msgr2=0x7fba5019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 0x7fba5019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 -- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 msgr2=0x7fba50197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba50197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 -- 172.21.15.122:0/84758060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5019f0b0 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4ffff700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba50197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4f7fe700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba40004790 tx=0x7fba40009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40027070 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba40004150 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5019f340 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4000f460 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.459+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5019f870 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.460+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fba4002a070 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.460+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba50068f20 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.460+0000 7fba4d7fa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 0x7fba3803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.460+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba4004d280 con 0x7fba50102390 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.460+0000 7fba4ffff700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 0x7fba3803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:33.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.461+0000 7fba4ffff700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 0x7fba3803b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba501021f0 tx=0x7fba44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:33.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.465+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fba40013070 con 0x7fba50102390 2024-04-25T00:13:33.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.674+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi071=y;smithi122=z", "target": ["mon-mgr", ""]}) -- 0x7fba50062670 con 0x7fba38039330 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.680+0000 7fba4d7fa700 1 -- 172.21.15.122:0/84758060 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fba50062670 con 0x7fba38039330 2024-04-25T00:13:33.680 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mgr update... 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 msgr2=0x7fba3803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 0x7fba3803b7f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba501021f0 tx=0x7fba44008040 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 msgr2=0x7fba50197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba40004790 tx=0x7fba40009580 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 shutdown_connections 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fba38039330 0x7fba3803b7f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba50102d30 0x7fba5019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba50101190 0x7fba50197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 --2- 172.21.15.122:0/84758060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba50102390 0x7fba50197990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 >> 172.21.15.122:0/84758060 conn(0x7fba500fc730 msgr2=0x7fba501056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 shutdown_connections 2024-04-25T00:13:33.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:13:33.682+0000 7fba569aa700 1 -- 172.21.15.122:0/84758060 wait complete. 2024-04-25T00:13:34.254 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T00:13:34.263 DEBUG:teuthology.orchestra.run.smithi122:mgr.z> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T00:13:34.265 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T00:13:34.265 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:13:34.265 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:13:34.285 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:13:34.285 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 00:12:35.466936517 +0000 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 00:07:32.690107136 +0000 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 00:07:32.690107136 +0000 2024-04-25T00:13:34.347 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 00:07:32.690107136 +0000 2024-04-25T00:13:34.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:13:34.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:34.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:34.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:34.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:34.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:34.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T00:13:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T00:13:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000109268 s, 4.7 MB/s 2024-04-25T00:13:34.422 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:13:34.440 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.440 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:34.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:34.442 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:34.442 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:34.442 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:34.488 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-25T00:13:34.552 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 00:12:35.501935978 +0000 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 00:07:33.036101734 +0000 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 00:07:33.036101734 +0000 2024-04-25T00:13:34.553 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 00:07:33.036101734 +0000 2024-04-25T00:13:34.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:13:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T00:13:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T00:13:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000125643 s, 4.1 MB/s 2024-04-25T00:13:34.628 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:13:34.691 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-25T00:13:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:13:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 00:12:35.544935317 +0000 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 00:07:33.372096489 +0000 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 00:07:33.372096489 +0000 2024-04-25T00:13:34.754 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 00:07:33.372096489 +0000 2024-04-25T00:13:34.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T00:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T00:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.0001001 s, 5.1 MB/s 2024-04-25T00:13:34.824 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:13:34.885 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 00:12:35.579934779 +0000 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 00:07:33.716091118 +0000 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 00:07:33.716091118 +0000 2024-04-25T00:13:34.944 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 00:07:33.716091118 +0000 2024-04-25T00:13:34.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:13:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T00:13:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T00:13:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000110526 s, 4.6 MB/s 2024-04-25T00:13:35.015 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:13:35.075 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-25T00:13:35.075 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:13:35.097 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:13:35.097 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-25T00:13:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:35 smithi045 ceph-mon[29194]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:35 smithi045 ceph-mon[29194]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi071=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:35 smithi045 ceph-mon[29194]: Saving service mgr spec with placement smithi045=x;smithi071=y;smithi122=z;count:3 2024-04-25T00:13:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:35 smithi045 ceph-mon[29194]: Deploying daemon mgr.z on smithi122 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-25 00:13:19.170609378 +0000 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-25 00:07:46.888218989 +0000 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-25 00:07:46.888218989 +0000 2024-04-25T00:13:35.159 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-25 00:07:46.888218989 +0000 2024-04-25T00:13:35.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:13:35.226 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-25T00:13:35.226 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-25T00:13:35.226 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000281711 s, 1.8 MB/s 2024-04-25T00:13:35.227 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:13:35.287 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-25T00:13:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:13:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-25 00:13:19.171609361 +0000 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-25 00:07:47.226213060 +0000 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-25 00:07:47.226213060 +0000 2024-04-25T00:13:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-25 00:07:47.226213060 +0000 2024-04-25T00:13:35.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:13:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:35 smithi071 ceph-mon[32667]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:35 smithi071 ceph-mon[32667]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi071=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:35 smithi071 ceph-mon[32667]: Saving service mgr spec with placement smithi045=x;smithi071=y;smithi122=z;count:3 2024-04-25T00:13:35.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:35 smithi071 ceph-mon[32667]: Deploying daemon mgr.z on smithi122 2024-04-25T00:13:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-25T00:13:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-25T00:13:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000212751 s, 2.4 MB/s 2024-04-25T00:13:35.393 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:13:35.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:35 smithi122 ceph-mon[32959]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:35.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:35 smithi122 ceph-mon[32959]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi071=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:35.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:35 smithi122 ceph-mon[32959]: Saving service mgr spec with placement smithi045=x;smithi071=y;smithi122=z;count:3 2024-04-25T00:13:35.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:35 smithi122 ceph-mon[32959]: Deploying daemon mgr.z on smithi122 2024-04-25T00:13:35.455 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-25 00:13:19.171609361 +0000 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-25 00:07:47.576206920 +0000 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-25 00:07:47.576206920 +0000 2024-04-25T00:13:35.514 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-25 00:07:47.576206920 +0000 2024-04-25T00:13:35.515 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-25T00:13:35.578 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000218216 s, 2.3 MB/s 2024-04-25T00:13:35.580 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:13:35.637 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-25T00:13:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:13:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:35.693 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-25 00:13:19.171609361 +0000 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-25 00:07:47.908201095 +0000 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-25 00:07:47.908201095 +0000 2024-04-25T00:13:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-25 00:07:47.908201095 +0000 2024-04-25T00:13:35.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:13:35.757 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-25T00:13:35.757 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-25T00:13:35.757 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000216989 s, 2.4 MB/s 2024-04-25T00:13:35.758 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:13:35.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:35 smithi122 systemd[1]: Starting Ceph mgr.z for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:13:35.815 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-25T00:13:35.815 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:13:35.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:13:35.832 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-25 00:13:35.528973646 +0000 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-25 00:07:45.414109010 +0000 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-25 00:07:45.414109010 +0000 2024-04-25T00:13:35.887 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-25 00:07:45.414109010 +0000 2024-04-25T00:13:35.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:13:35.966 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-25T00:13:35.966 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-25T00:13:35.966 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.0145369 s, 35.2 kB/s 2024-04-25T00:13:35.967 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:13:36.023 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-25T00:13:36.057 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:35 smithi122 podman[34598]: 2024-04-25 00:13:35.709252189 +0000 UTC m=+0.017643072 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:36.057 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:35 smithi122 podman[34598]: 2024-04-25 00:13:35.826809997 +0000 UTC m=+0.135200869 container create e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T00:13:36.057 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 podman[34598]: 2024-04-25 00:13:36.054300189 +0000 UTC m=+0.362691061 container init e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T00:13:36.057 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 podman[34598]: 2024-04-25 00:13:36.057117707 +0000 UTC m=+0.365508580 container start e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:13:36.072 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:13:36.072 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:36.072 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T00:13:36.072 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:36.072 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-25 00:13:35.528973646 +0000 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-25 00:07:45.770101121 +0000 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-25 00:07:45.770101121 +0000 2024-04-25T00:13:36.073 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-25 00:07:45.770101121 +0000 2024-04-25T00:13:36.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:13:36.134 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-25T00:13:36.135 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-25T00:13:36.135 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000199857 s, 2.6 MB/s 2024-04-25T00:13:36.136 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:13:36.192 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-25 00:13:35.529973624 +0000 2024-04-25T00:13:36.247 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-25 00:07:46.117093431 +0000 2024-04-25T00:13:36.248 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-25 00:07:46.117093431 +0000 2024-04-25T00:13:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-25 00:07:46.117093431 +0000 2024-04-25T00:13:36.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:13:36.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-mgr[34686]: -- 172.21.15.122:0/1061578660 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b98c465a0 con 0x562b98c1a800 2024-04-25T00:13:36.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 bash[34598]: e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 2024-04-25T00:13:36.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 systemd[1]: Started Ceph mgr.z for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:13:36.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:36.217+0000 7f0207a4c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:13:36.310 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-25T00:13:36.310 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-25T00:13:36.310 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000178702 s, 2.9 MB/s 2024-04-25T00:13:36.311 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:13:36.367 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-25T00:13:36.422 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:13:36.422 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:13:36.422 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T00:13:36.422 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:13:36.422 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:13:36.423 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-25 00:13:35.529973624 +0000 2024-04-25T00:13:36.423 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-25 00:07:46.454085963 +0000 2024-04-25T00:13:36.423 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-25 00:07:46.454085963 +0000 2024-04-25T00:13:36.423 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-25 00:07:46.454085963 +0000 2024-04-25T00:13:36.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:13:36.484 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-25T00:13:36.485 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-25T00:13:36.485 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000199913 s, 2.6 MB/s 2024-04-25T00:13:36.486 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:13:36.542 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-04-25T00:13:36.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:13:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:36.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:36.408+0000 7f0207a4c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:13:36.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:36.513+0000 7f0207a4c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:13:37.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:37.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:37.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:37.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:37.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:36.821+0000 7f0207a4c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:13:37.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:36.904+0000 7f0207a4c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:37.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-mon[32959]: Deploying daemon mgr.y on smithi071 2024-04-25T00:13:38.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:38 smithi071 ceph-mon[32667]: Deploying daemon mgr.y on smithi071 2024-04-25T00:13:38.369 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 systemd[1]: Starting Ceph mgr.y for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:13:38.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:38 smithi045 ceph-mon[29194]: Deploying daemon mgr.y on smithi071 2024-04-25T00:13:38.474 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:38.125+0000 7f0207a4c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:13:38.474 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:38.300+0000 7f0207a4c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:13:38.474 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:38.472+0000 7f0207a4c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00833689 s, 1.3 GB/s 2024-04-25T00:13:38.624 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 podman[34075]: 2024-04-25 00:13:38.292685664 +0000 UTC m=+0.017579490 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 podman[34075]: 2024-04-25 00:13:38.397742948 +0000 UTC m=+0.122636830 container create bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 podman[34075]: 2024-04-25 00:13:38.591533236 +0000 UTC m=+0.316427048 container init bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 podman[34075]: 2024-04-25 00:13:38.594266937 +0000 UTC m=+0.319160759 container start bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 ceph-mgr[34094]: -- 172.21.15.71:0/2930345451 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fdd0845a0 con 0x556fdd058800 2024-04-25T00:13:38.675 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 bash[34075]: bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 2024-04-25T00:13:38.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 systemd[1]: Started Ceph mgr.y for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:13:38.945 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:38.758+0000 7fb1ef58e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:13:38.946 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:38.945+0000 7fb1ef58e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:13:39.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:38.766+0000 7f0207a4c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:13:39.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:38.859+0000 7f0207a4c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:13:39.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-04-25T00:13:39.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:39.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.198 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:39.368 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:39.052+0000 7fb1ef58e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:13:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:39.419 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:39.070+0000 7f0207a4c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:39.269+0000 7f0207a4c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:39.654 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:39.386+0000 7fb1ef58e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:13:39.655 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:39.479+0000 7fb1ef58e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:13:40.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:39.913+0000 7f0207a4c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:13:40.647 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:40.351+0000 7f0207a4c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:13:40.647 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:40.446+0000 7f0207a4c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:13:40.647 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:40.646+0000 7f0207a4c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 -- 172.21.15.45:0/2650525550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 msgr2=0x7f5c901038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 --2- 172.21.15.45:0/2650525550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c901038f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5c840099c0 tx=0x7f5c84009cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 -- 172.21.15.45:0/2650525550 shutdown_connections 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 --2- 172.21.15.45:0/2650525550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90102290 0x7f5c901026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 --2- 172.21.15.45:0/2650525550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90103e30 0x7f5c9010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 --2- 172.21.15.45:0/2650525550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c901038f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 -- 172.21.15.45:0/2650525550 >> 172.21.15.45:0/2650525550 conn(0x7f5c900fd830 msgr2=0x7f5c900ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 -- 172.21.15.45:0/2650525550 shutdown_connections 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.998+0000 7f5c9538d700 1 -- 172.21.15.45:0/2650525550 wait complete. 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 Processor -- start 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 -- start start 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90102290 0x7f5c90075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90103e30 0x7f5c90074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9010de70 con 0x7f5c90103490 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9010dcf0 con 0x7f5c90102290 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c9538d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9010dff0 con 0x7f5c90103e30 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c8e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c8e7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33468/0 (socket says 172.21.15.45:33468) 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/1942786372 learned_addr learned my addr 172.21.15.45:0/1942786372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c8effd700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90102290 0x7f5c90075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:40.999+0000 7f5c8f7fe700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90103e30 0x7f5c90074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/1942786372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90103e30 msgr2=0x7f5c90074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90103e30 0x7f5c90074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/1942786372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90102290 msgr2=0x7f5c90075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90102290 0x7f5c90075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 -- 172.21.15.45:0/1942786372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90075410 con 0x7f5c90103490 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c8e7fc700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005910 tx=0x7f5c8400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84027070 con 0x7f5c90103490 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8400fe40 con 0x7f5c90103490 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c901a2610 con 0x7f5c90103490 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84017650 con 0x7f5c90103490 2024-04-25T00:13:41.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.000+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c901a2a60 con 0x7f5c90103490 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.001+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5c8402a070 con 0x7f5c90103490 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.001+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c90068f20 con 0x7f5c90103490 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.001+0000 7f5c77fff700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 0x7f5c7803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.001+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5c8404d490 con 0x7f5c90103490 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.001+0000 7f5c8effd700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 0x7f5c7803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.002+0000 7f5c8effd700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 0x7f5c7803b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c901032f0 tx=0x7f5c80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.006+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c84013070 con 0x7f5c90103490 2024-04-25T00:13:41.037 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:40.700+0000 7fb1ef58e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:13:41.037 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:40.867+0000 7fb1ef58e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:13:41.038 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:41.037+0000 7fb1ef58e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:13:41.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:41.232+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5c90062670 con 0x7f5c78039330 2024-04-25T00:13:41.320 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:41.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:41.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:41.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:41.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:41.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.490 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.160+0000 7f0207a4c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:13:41.490 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.251+0000 7f0207a4c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:13:41.490 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:13:41.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.611 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:41.319+0000 7fb1ef58e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:13:41.611 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:41.410+0000 7fb1ef58e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:13:41.611 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:41.610+0000 7fb1ef58e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:13:41.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.489+0000 7f0207a4c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:13:41.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.620+0000 7f0207a4c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:13:41.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.716+0000 7f0207a4c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:13:41.868 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:41.801+0000 7fb1ef58e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:13:42.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:42 smithi045 ceph-mon[29194]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:13:42.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:42 smithi045 ceph-mon[29194]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T00:13:42.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:42.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:42.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.830+0000 7f0207a4c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:41.921+0000 7f0207a4c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-mon[32959]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-mon[32959]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:42.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:42.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-mon[32667]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:13:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-mon[32667]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T00:13:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:42.819 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:42.403+0000 7fb1ef58e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:13:43.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:42.736+0000 7f0207a4c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:13:43.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:13:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:13:42.951+0000 7f0207a4c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:13:43.072 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:42.818+0000 7fb1ef58e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:13:43.073 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:42.914+0000 7fb1ef58e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:13:43.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:43.186+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f5c8402c440 con 0x7f5c90103490 2024-04-25T00:13:43.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: Standby manager daemon z started 2024-04-25T00:13:43.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:13:43.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:43.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:13:43.190 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:43 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:43.368 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:43.113+0000 7fb1ef58e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: Standby manager daemon z started 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:13:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: Standby manager daemon z started 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:13:43.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:43 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/1817194490' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:43.957 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:43.627+0000 7fb1ef58e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:13:43.957 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:43.715+0000 7fb1ef58e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:13:44.295 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-mon[32667]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:13:44.295 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:13:44.295 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-mon[32667]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:44.296 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:43.956+0000 7fb1ef58e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:13:44.296 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:44.089+0000 7fb1ef58e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:13:44.296 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:44.180+0000 7fb1ef58e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:13:44.296 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:44.295+0000 7fb1ef58e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:13:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:44 smithi122 ceph-mon[32959]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:13:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:13:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:44 smithi122 ceph-mon[32959]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:44.618 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:44 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:44.384+0000 7fb1ef58e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:13:44.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:44 smithi045 ceph-mon[29194]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T00:13:44.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:13:44.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:44 smithi045 ceph-mon[29194]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]: dispatch 2024-04-25T00:13:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]': finished 2024-04-25T00:13:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:45 smithi122 ceph-mon[32959]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:13:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:13:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3825597315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:45.618 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:45.183+0000 7fb1ef58e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:13:45.396+0000 7fb1ef58e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]: dispatch 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]': finished 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-mon[32667]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:13:45.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3825597315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]: dispatch 2024-04-25T00:13:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1243357704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47ff99bd-1736-4c56-8965-bd5199120838"}]': finished 2024-04-25T00:13:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:45 smithi045 ceph-mon[29194]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:13:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:13:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3825597315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:46.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:13:46.206+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5c84017b70 con 0x7f5c90103490 2024-04-25T00:13:46.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:46.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: Standby manager daemon y started 2024-04-25T00:13:46.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:13:46.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:46.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:13:46.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:46 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: Standby manager daemon y started 2024-04-25T00:13:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:13:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:13:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:46 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: Standby manager daemon y started 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:13:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:46 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/668995106' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:13:47.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:47 smithi122 ceph-mon[32959]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:13:47.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:47 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:13:47.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:47 smithi071 ceph-mon[32667]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:13:47.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:47 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:13:47.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:47 smithi045 ceph-mon[29194]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T00:13:47.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:47 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:13:48.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:48 smithi045 ceph-mon[29194]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:48.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:48 smithi122 ceph-mon[32959]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:48.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:48 smithi071 ceph-mon[32667]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:50.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:50 smithi045 ceph-mon[29194]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:50.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:50 smithi122 ceph-mon[32959]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:50.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:50 smithi071 ceph-mon[32667]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:52.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:52 smithi045 ceph-mon[29194]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:52.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:13:52.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:52.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:52 smithi045 ceph-mon[29194]: Deploying daemon osd.0 on smithi045 2024-04-25T00:13:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:52 smithi122 ceph-mon[32959]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:13:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:52 smithi122 ceph-mon[32959]: Deploying daemon osd.0 on smithi045 2024-04-25T00:13:52.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:52 smithi071 ceph-mon[32667]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:52.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:13:52.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:52.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:52 smithi071 ceph-mon[32667]: Deploying daemon osd.0 on smithi045 2024-04-25T00:13:54.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:54 smithi045 ceph-mon[29194]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:54.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:54 smithi122 ceph-mon[32959]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:54.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:54 smithi071 ceph-mon[32667]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:56.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:56 smithi045 ceph-mon[29194]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:56.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:56 smithi122 ceph-mon[32959]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:56.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:56 smithi071 ceph-mon[32667]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:58.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:13:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:59.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:13:58 smithi045 ceph-mon[29194]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:13:58 smithi071 ceph-mon[32667]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:13:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:13:58 smithi122 ceph-mon[32959]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.191+0000 7f5c77fff700 1 -- 172.21.15.45:0/1942786372 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c90062670 con 0x7f5c78039330 2024-04-25T00:14:00.192 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 msgr2=0x7f5c7803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 0x7f5c7803b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c901032f0 tx=0x7f5c80009260 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 msgr2=0x7f5c90074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005910 tx=0x7f5c8400fa00 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 shutdown_connections 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c90103e30 0x7f5c90074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c78039330 0x7f5c7803b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c90102290 0x7f5c90075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 --2- 172.21.15.45:0/1942786372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c90103490 0x7f5c90074620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 >> 172.21.15.45:0/1942786372 conn(0x7f5c900fd830 msgr2=0x7f5c901067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 shutdown_connections 2024-04-25T00:14:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:00.193+0000 7f5c9538d700 1 -- 172.21.15.45:0/1942786372 wait complete. 2024-04-25T00:14:00.792 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T00:14:00.835 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_4... 2024-04-25T00:14:00.835 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:14:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:14:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:00 smithi071 ceph-mon[32667]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:00 smithi071 ceph-mon[32667]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:14:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:00 smithi045 ceph-mon[29194]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:00 smithi045 ceph-mon[29194]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:14:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:00 smithi122 ceph-mon[32959]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:00 smithi122 ceph-mon[32959]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:14:02.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:14:02.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:14:02.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:14:02.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:01 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:02.169 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.169 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.169 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:14:02.170 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:14:02.170 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:14:02.170 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:01 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:14:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:01 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:02.834 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0[41753]: 2024-04-25T00:14:02.697+0000 7f1f9a0d5700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:14:02.834 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0[41753]: 2024-04-25T00:14:02.700+0000 7f1f91ec0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:14:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:14:02.852 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-25T00:14:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-25T00:14:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113982 s, 920 MB/s 2024-04-25T00:14:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-25T00:14:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:14:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:14:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:02 smithi071 ceph-mon[32667]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:02 smithi045 ceph-mon[29194]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' 2024-04-25T00:14:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:14:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:14:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:03.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:02 smithi122 ceph-mon[32959]: from='osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100]' entity='osd.0' 2024-04-25T00:14:03.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-04-25T00:14:03.649 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] boot 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:03 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:04.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:14:04.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:14:04.285 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:04.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] boot 2024-04-25T00:14:04.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:14:04.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: osd.0 [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] boot 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:14:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:03 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:14:05.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:05.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:05.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:04 smithi071 ceph-mon[32667]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:14:05.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 -- 172.21.15.71:0/4272233644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4101df0 msgr2=0x7f70a4109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 --2- 172.21.15.71:0/4272233644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4101df0 0x7f70a4109690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 -- 172.21.15.71:0/4272233644 shutdown_connections 2024-04-25T00:14:05.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 --2- 172.21.15.71:0/4272233644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4102780 0x7f70a4102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 --2- 172.21.15.71:0/4272233644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4101df0 0x7f70a4109690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 --2- 172.21.15.71:0/4272233644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101430 0x7f70a41018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 -- 172.21.15.71:0/4272233644 >> 172.21.15.71:0/4272233644 conn(0x7f70a40fbec0 msgr2=0x7f70a40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 -- 172.21.15.71:0/4272233644 shutdown_connections 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.181+0000 7f70aadd6700 1 -- 172.21.15.71:0/4272233644 wait complete. 2024-04-25T00:14:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 Processor -- start 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 -- start start 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 0x7f70a41051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101df0 0x7f70a4103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410ce50 con 0x7f70a4101df0 2024-04-25T00:14:05.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410ccd0 con 0x7f70a4102780 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70aadd6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410cfd0 con 0x7f70a4101430 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70a9373700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70a9373700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46272/0 (socket says 172.21.15.71:46272) 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70a9373700 1 -- 172.21.15.71:0/3866039479 learned_addr learned my addr 172.21.15.71:0/3866039479 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70a3fff700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101df0 0x7f70a4103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.182+0000 7f70a8b72700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 0x7f70a41051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 -- 172.21.15.71:0/3866039479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 msgr2=0x7f70a41051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 0x7f70a41051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 -- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101df0 msgr2=0x7f70a4103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101df0 0x7f70a4103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 -- 172.21.15.71:0/3866039479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a4104610 con 0x7f70a4102780 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a8b72700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 0x7f70a41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a9373700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b070 tx=0x7f709c004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c021070 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c007cc0 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a41a2620 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c005490 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.183+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a2a80 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.184+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a4068f20 con 0x7f70a4102780 2024-04-25T00:14:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.185+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f709c024070 con 0x7f70a4102780 2024-04-25T00:14:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.186+0000 7f70a1ffb700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 0x7f70940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.186+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f709c00f020 con 0x7f70a4102780 2024-04-25T00:14:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.186+0000 7f70a8b72700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 0x7f70940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.187+0000 7f70a8b72700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 0x7f70940a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a4069cb0 tx=0x7f7098008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.191+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f709c02b020 con 0x7f70a4102780 2024-04-25T00:14:05.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:05.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.209 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.209 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:05.209 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:04 smithi122 ceph-mon[32959]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:14:05.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:05.394+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f70a40fdf90 con 0x7f70940a2b90 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:04 smithi045 ceph-mon[29194]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:14:06.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi045 2024-04-25T00:14:06.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-25T00:14:06.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:06.196 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:06.196 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:06.196 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:06.196 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi045 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi045 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:07.155 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:07 smithi071 ceph-mon[32667]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:07.155 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:07 smithi071 ceph-mon[32667]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:07 smithi045 ceph-mon[29194]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:07 smithi045 ceph-mon[29194]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:07.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:07 smithi122 ceph-mon[32959]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:07.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:07 smithi122 ceph-mon[32959]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:08 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/2682606155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:08 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:08 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]': finished 2024-04-25T00:14:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:08 smithi071 ceph-mon[32667]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:14:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:08 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/2682606155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:08 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:08 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]': finished 2024-04-25T00:14:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:08 smithi045 ceph-mon[29194]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:14:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:08 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/2682606155' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:08 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]: dispatch 2024-04-25T00:14:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:08 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0c603f9-9f3e-4af1-9a59-e3e887208af5"}]': finished 2024-04-25T00:14:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:08 smithi122 ceph-mon[32959]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:14:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:09.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:09 smithi045 ceph-mon[29194]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:09.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:09 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/2554044992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:09 smithi071 ceph-mon[32667]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:09 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/2554044992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:09 smithi122 ceph-mon[32959]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:09 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/2554044992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:11 smithi071 ceph-mon[32667]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:11 smithi045 ceph-mon[29194]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:11 smithi122 ceph-mon[32959]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:13 smithi071 ceph-mon[32667]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:13 smithi045 ceph-mon[29194]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:13.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:13 smithi122 ceph-mon[32959]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:15 smithi071 ceph-mon[32667]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:14:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:15 smithi045 ceph-mon[29194]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:14:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:15 smithi122 ceph-mon[32959]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:14:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:16.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:16 smithi122 ceph-mon[32959]: Deploying daemon osd.1 on smithi071 2024-04-25T00:14:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:16 smithi071 ceph-mon[32667]: Deploying daemon osd.1 on smithi071 2024-04-25T00:14:16.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:16 smithi045 ceph-mon[29194]: Deploying daemon osd.1 on smithi071 2024-04-25T00:14:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:17 smithi071 ceph-mon[32667]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:17 smithi045 ceph-mon[29194]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:17 smithi122 ceph-mon[32959]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:19.336 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:19 smithi071 ceph-mon[32667]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:19 smithi045 ceph-mon[29194]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:19.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:19 smithi122 ceph-mon[32959]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:21 smithi071 ceph-mon[32667]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:21 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:21 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:21 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:21 smithi045 ceph-mon[29194]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:21 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:21 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:21 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:21 smithi122 ceph-mon[32959]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:21 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:21 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:21 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:22.252 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:22 smithi071 ceph-mon[32667]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:22.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:22 smithi122 ceph-mon[32959]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:22.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:22 smithi045 ceph-mon[29194]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:23.661 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-04-25T00:14:23.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.660+0000 7f70a1ffb700 1 -- 172.21.15.71:0/3866039479 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f70a40fdf90 con 0x7f70940a2b90 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 msgr2=0x7f70940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 0x7f70940a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a4069cb0 tx=0x7f7098008040 comp rx=0 tx=0).stop 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 msgr2=0x7f70a4103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b070 tx=0x7f709c004dc0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 shutdown_connections 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70a4101430 0x7f70a41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f70940a2b90 0x7f70940a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a4102780 0x7f70a4103d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 --2- 172.21.15.71:0/3866039479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70a4101df0 0x7f70a4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:23.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 >> 172.21.15.71:0/3866039479 conn(0x7f70a40fbec0 msgr2=0x7f70a40fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:23.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 shutdown_connections 2024-04-25T00:14:23.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:14:23.662+0000 7f70aadd6700 1 -- 172.21.15.71:0/3866039479 wait complete. 2024-04-25T00:14:23.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:23.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:23.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:23.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:23.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:23.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:23.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:23.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:23 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:23 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:24.300 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T00:14:24.302 INFO:tasks.cephadm:Deploying osd.2 on smithi122 with /dev/vg_nvme/lv_4... 2024-04-25T00:14:24.302 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:14:24.305 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:23 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:24.544 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:14:24.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:24 smithi071 ceph-mon[32667]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:24.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:24 smithi071 ceph-mon[32667]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:24.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:24 smithi071 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:25.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:24 smithi122 ceph-mon[32959]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:25.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:24 smithi122 ceph-mon[32959]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:25.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:24 smithi122 ceph-mon[32959]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:24 smithi045 ceph-mon[29194]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:24 smithi045 ceph-mon[29194]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:24 smithi045 ceph-mon[29194]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:14:25.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:25 smithi122 ceph-mon[32959]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:14:25.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:25 smithi122 ceph-mon[32959]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:25.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:25 smithi122 ceph-mon[32959]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:14:25.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:25 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:25.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:25 smithi122 ceph-mon[32959]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:26.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:25 smithi071 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:14:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:25 smithi071 ceph-mon[32667]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:25 smithi071 ceph-mon[32667]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:14:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:25 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:26.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:25 smithi071 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:25 smithi045 ceph-mon[29194]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:14:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:25 smithi045 ceph-mon[29194]: from='osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:25 smithi045 ceph-mon[29194]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:14:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:25 smithi045 ceph-mon[29194]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:14:26.437 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:14:26.438 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:14:26.438 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-25T00:14:26.438 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-25T00:14:26.438 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109629 s, 956 MB/s 2024-04-25T00:14:26.438 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-25T00:14:26.580 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1[37297]: 2024-04-25T00:14:26.261+0000 7feb2f2da700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:14:26.580 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1[37297]: 2024-04-25T00:14:26.267+0000 7feb270c5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.103 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:26 smithi122 ceph-mon[32959]: from='osd.1 ' entity='osd.1' 2024-04-25T00:14:27.104 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-04-25T00:14:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:14:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:14:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:26 smithi071 ceph-mon[32667]: from='osd.1 ' entity='osd.1' 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:26 smithi045 ceph-mon[29194]: from='osd.1 ' entity='osd.1' 2024-04-25T00:14:27.394 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] boot 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:28.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:28.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] boot 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:28.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:28.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: osd.1 [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] boot 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:28.914 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:28 smithi122 ceph-mon[32959]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:28.914 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:28 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi071 2024-04-25T00:14:28.915 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:28 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 -- 172.21.15.122:0/3528125306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 msgr2=0x7f56a01015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 --2- 172.21.15.122:0/3528125306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a01015b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5690009990 tx=0x7f5690009ca0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 -- 172.21.15.122:0/3528125306 shutdown_connections 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 --2- 172.21.15.122:0/3528125306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a01015b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 --2- 172.21.15.122:0/3528125306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102d30 0x7f56a01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 --2- 172.21.15.122:0/3528125306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102390 0x7f56a01027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 -- 172.21.15.122:0/3528125306 >> 172.21.15.122:0/3528125306 conn(0x7f56a00fc730 msgr2=0x7f56a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 -- 172.21.15.122:0/3528125306 shutdown_connections 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.913+0000 7f56a6a14700 1 -- 172.21.15.122:0/3528125306 wait complete. 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 Processor -- start 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 -- start start 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102390 0x7f56a019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102d30 0x7f56a019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010ce30 con 0x7f56a0102d30 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.914+0000 7f56a6a14700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010ccb0 con 0x7f56a0102390 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f56a6a14700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010cfb0 con 0x7f56a0101190 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36178/0 (socket says 172.21.15.122:36178) 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 -- 172.21.15.122:0/2730991858 learned_addr learned my addr 172.21.15.122:0/2730991858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f56a4fb1700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102d30 0x7f56a019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569f7fe700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102390 0x7f56a019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 -- 172.21.15.122:0/2730991858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102390 msgr2=0x7f56a019d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102390 0x7f56a019d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 -- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102d30 msgr2=0x7f56a019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102d30 0x7f56a019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 -- 172.21.15.122:0/2730991858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a01a0210 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569ffff700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5690003a60 tx=0x7f56900047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004830 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5690005020 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01a04a0 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004040 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.915+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a0910 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.916+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a0068f20 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.918+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5690028060 con 0x7f56a0101190 2024-04-25T00:14:28.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.918+0000 7f569d7fa700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 0x7f56880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.918+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f56900ce160 con 0x7f56a0101190 2024-04-25T00:14:28.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.918+0000 7f569f7fe700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 0x7f56880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.919+0000 7f569f7fe700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 0x7f56880a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5694005950 tx=0x7f56940058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:28.923+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5690013070 con 0x7f56a0101190 2024-04-25T00:14:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:28 smithi071 ceph-mon[32667]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:28 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi071 2024-04-25T00:14:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:28 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-25T00:14:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:29.127+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f56a0062670 con 0x7f56880a2b90 2024-04-25T00:14:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:28 smithi045 ceph-mon[29194]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:14:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:28 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi071 2024-04-25T00:14:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:28 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-25T00:14:30.047 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:29 smithi122 ceph-mon[32959]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:14:30.047 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:30.047 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:30.047 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:30.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:29 smithi071 ceph-mon[32667]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:14:30.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:30.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:30.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:29 smithi045 ceph-mon[29194]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:14:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:31.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:30 smithi071 ceph-mon[32667]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:30 smithi071 ceph-mon[32667]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:31.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:30 smithi045 ceph-mon[29194]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:30 smithi045 ceph-mon[29194]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:30 smithi122 ceph-mon[32959]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:30 smithi122 ceph-mon[32959]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:32.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:31 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/3355718794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:31 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:31 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]': finished 2024-04-25T00:14:32.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:31 smithi071 ceph-mon[32667]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:14:32.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:31 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:31 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/3355718794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:31 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:31 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]': finished 2024-04-25T00:14:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:31 smithi045 ceph-mon[29194]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:14:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:31 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:31 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/3355718794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:31 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]: dispatch 2024-04-25T00:14:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:31 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e528381f-f657-4e55-a279-3f45e35fa585"}]': finished 2024-04-25T00:14:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:31 smithi122 ceph-mon[32959]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:14:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:31 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:33.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:32 smithi071 ceph-mon[32667]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:33.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:32 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/2620043031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:32 smithi045 ceph-mon[29194]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:32 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/2620043031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:32 smithi122 ceph-mon[32959]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:32 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/2620043031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:35.109 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:34 smithi122 ceph-mon[32959]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:34 smithi071 ceph-mon[32667]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:34 smithi045 ceph-mon[29194]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:37.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:36 smithi071 ceph-mon[32667]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:36 smithi045 ceph-mon[29194]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:36 smithi122 ceph-mon[32959]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:38 smithi122 ceph-mon[32959]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:39.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:38 smithi071 ceph-mon[32667]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:38 smithi045 ceph-mon[29194]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:14:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:39 smithi045 ceph-mon[29194]: Deploying daemon osd.2 on smithi122 2024-04-25T00:14:40.177 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:14:40.177 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:40.177 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:39 smithi122 ceph-mon[32959]: Deploying daemon osd.2 on smithi122 2024-04-25T00:14:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:14:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:39 smithi071 ceph-mon[32667]: Deploying daemon osd.2 on smithi122 2024-04-25T00:14:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:40 smithi122 ceph-mon[32959]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:40 smithi045 ceph-mon[29194]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:40 smithi071 ceph-mon[32667]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:43.045 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:42 smithi122 ceph-mon[32959]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:43.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:42 smithi045 ceph-mon[29194]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:42 smithi071 ceph-mon[32667]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:45.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:44 smithi122 ceph-mon[32959]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:45.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:44 smithi045 ceph-mon[29194]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:44 smithi071 ceph-mon[32667]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:47.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:46 smithi045 ceph-mon[29194]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:47.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:46 smithi122 ceph-mon[32959]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:46 smithi071 ceph-mon[32667]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:47.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.547+0000 7f569d7fa700 1 -- 172.21.15.122:0/2730991858 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56a0062670 con 0x7f56880a2b90 2024-04-25T00:14:47.548 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 2 on host 'smithi122' 2024-04-25T00:14:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 msgr2=0x7f56880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 0x7f56880a5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5694005950 tx=0x7f56940058e0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 msgr2=0x7f56a006aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5690003a60 tx=0x7f56900047b0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 shutdown_connections 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0101190 0x7f56a006aa80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f56880a2b90 0x7f56880a5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a0102d30 0x7f56a019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 --2- 172.21.15.122:0/2730991858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a0102390 0x7f56a019d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 >> 172.21.15.122:0/2730991858 conn(0x7f56a00fc730 msgr2=0x7f56a01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 shutdown_connections 2024-04-25T00:14:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:14:47.550+0000 7f56a6a14700 1 -- 172.21.15.122:0/2730991858 wait complete. 2024-04-25T00:14:48.191 DEBUG:teuthology.orchestra.run.smithi122:osd.2> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T00:14:48.194 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-04-25T00:14:48.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:14:48.450 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:48.523 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:48.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:48 smithi122 ceph-mon[32959]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:48.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:48.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:48.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:48.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:48 smithi045 ceph-mon[29194]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:48.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:48 smithi071 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:14:50.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-mon[32959]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:14:50.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-mon[32959]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:14:50.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:50.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-mon[32959]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-mon[32959]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.057 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2[38137]: 2024-04-25T00:14:49.889+0000 7f14715f2700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:14:50.057 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2[38137]: 2024-04-25T00:14:49.893+0000 7f146d3e5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:49 smithi045 ceph-mon[29194]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:14:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:49 smithi045 ceph-mon[29194]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:14:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:49 smithi045 ceph-mon[29194]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:49 smithi045 ceph-mon[29194]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:49 smithi071 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:14:50.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:49 smithi071 ceph-mon[32667]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:14:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:49 smithi071 ceph-mon[32667]: from='osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:49 smithi071 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:14:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:14:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:14:50.475 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T00:14:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T00:14:50.475 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00826749 s, 1.3 GB/s 2024-04-25T00:14:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T00:14:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:50 smithi122 ceph-mon[32959]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:50 smithi122 ceph-mon[32959]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:14:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:50 smithi122 ceph-mon[32959]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:14:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:50 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:50 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:50 smithi045 ceph-mon[29194]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:50 smithi045 ceph-mon[29194]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:14:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:50 smithi045 ceph-mon[29194]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:14:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:50 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:50 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-04-25T00:14:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:50 smithi071 ceph-mon[32667]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:50 smithi071 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:14:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:50 smithi071 ceph-mon[32667]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:14:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:50 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:50 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:52.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:14:52.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:14:52.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] boot 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:51 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:52.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:14:52.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] boot 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:51 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: osd.2 [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] boot 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:52.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:52.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:52.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:51 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.032+0000 7fd448478700 1 -- 172.21.15.45:0/337610333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440107f70 msgr2=0x7fd44010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.032+0000 7fd448478700 1 --2- 172.21.15.45:0/337610333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440107f70 0x7fd44010a3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b0a0 tx=0x7fd43800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 -- 172.21.15.45:0/337610333 shutdown_connections 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 --2- 172.21.15.45:0/337610333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440102d10 0x7fd440105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 --2- 172.21.15.45:0/337610333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440107f70 0x7fd44010a3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 --2- 172.21.15.45:0/337610333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440105640 0x7fd440107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 -- 172.21.15.45:0/337610333 >> 172.21.15.45:0/337610333 conn(0x7fd4400fc720 msgr2=0x7fd4400feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 -- 172.21.15.45:0/337610333 shutdown_connections 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 -- 172.21.15.45:0/337610333 wait complete. 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 Processor -- start 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.033+0000 7fd448478700 1 -- start start 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440105640 0x7fd44019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440107f70 0x7fd4401a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010de50 con 0x7fd440102d10 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010dcd0 con 0x7fd440107f70 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd44010dfd0 con 0x7fd440105640 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58640/0 (socket says 172.21.15.45:58640) 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 -- 172.21.15.45:0/4242292081 learned_addr learned my addr 172.21.15.45:0/4242292081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446a15700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440107f70 0x7fd4401a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd445a13700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440105640 0x7fd44019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 -- 172.21.15.45:0/4242292081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440105640 msgr2=0x7fd44019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440105640 0x7fd44019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 -- 172.21.15.45:0/4242292081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440107f70 msgr2=0x7fd4401a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440107f70 0x7fd4401a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 -- 172.21.15.45:0/4242292081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4401a22d0 con 0x7fd440102d10 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd446214700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd43000d9b0 tx=0x7fd43000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430017070 con 0x7fd440102d10 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.034+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4401a25c0 con 0x7fd440102d10 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.035+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a2b00 con 0x7fd440102d10 2024-04-25T00:14:53.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.035+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd430005020 con 0x7fd440102d10 2024-04-25T00:14:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.035+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430009bd0 con 0x7fd440102d10 2024-04-25T00:14:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.035+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424005330 con 0x7fd440102d10 2024-04-25T00:14:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.037+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fd430004a30 con 0x7fd440102d10 2024-04-25T00:14:53.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.038+0000 7fd4377fe700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 0x7fd42c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.038+0000 7fd445a13700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 0x7fd42c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.038+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fd4300cddd0 con 0x7fd440102d10 2024-04-25T00:14:53.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.038+0000 7fd445a13700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 0x7fd42c0a5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009bf0 tx=0x7fd43c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:53.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.049+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd430020020 con 0x7fd440102d10 2024-04-25T00:14:53.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi122 2024-04-25T00:14:53.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-25T00:14:53.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:53.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:53.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:14:53.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.267+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd424000bc0 con 0x7fd42c0a2b90 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi122 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:14:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi122 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:14:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:14:53.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46119]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:53.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46119]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:53.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46119]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:53.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46119]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:53.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:14:53.918+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd430098150 con 0x7fd440102d10 2024-04-25T00:14:54.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:54.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:53 smithi045 ceph-mon[29194]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:14:54.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:54.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:54.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:54.152 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:54.152 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:54.153 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46178]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:54.153 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:14:53 smithi045 sudo[46178]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:54.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:54.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:53 smithi071 ceph-mon[32667]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:14:54.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:54.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:54.177 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:14:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:53 smithi122 ceph-mon[32959]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:14:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:54.430 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:54.430 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:54.430 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41271]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:54.430 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41271]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:54.868 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41274]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:54.868 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41274]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:54.869 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41274]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:54.869 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:14:54 smithi071 sudo[41274]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:54.952 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42195]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:54.952 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:54.952 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42195]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:54.952 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42195]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:54.952 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:54.952 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:54.952 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:14:54.953 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:14:54.953 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:54.953 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:54 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:55.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:55.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:55.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:14:55.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:14:55.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:55.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:54 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:55.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:55.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:55.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42198]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:55.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:14:54 smithi122 sudo[42198]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:14:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:14:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:54 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:55.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46227]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:55.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46227]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:55.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46227]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:55.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46227]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:56.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:56.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:56.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46255]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:56.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:55 smithi045 sudo[46255]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:56.381 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42201]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:56.381 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42201]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:56.382 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42201]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:56.382 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42201]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:56.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42204]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42204]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42204]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 sudo[42204]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:56.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:56.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:56.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:56.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:56.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:14:56.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:56.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41277]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:56.877 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41277]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41280]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 sudo[41280]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:57.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:57.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]: dispatch 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]': finished 2024-04-25T00:14:57.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3368831864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:57.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:57.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]: dispatch 2024-04-25T00:14:57.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]': finished 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3368831864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:57.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:57.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]: dispatch 2024-04-25T00:14:57.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/911530878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f7644b-671b-4e42-b161-8895d46c48a7"}]': finished 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:14:57.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3368831864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:14:58 smithi045 ceph-mon[29194]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:58.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:14:58 smithi122 ceph-mon[32959]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:14:58.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:14:58 smithi071 ceph-mon[32667]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:00.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:00 smithi045 ceph-mon[29194]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:00.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:00 smithi122 ceph-mon[32959]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:00.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:00 smithi071 ceph-mon[32667]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:02.618 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:02 smithi045 ceph-mon[29194]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:02.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:02 smithi122 ceph-mon[32959]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:02.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:02 smithi071 ceph-mon[32667]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:04 smithi045 ceph-mon[29194]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:15:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:04 smithi045 ceph-mon[29194]: Deploying daemon osd.3 on smithi045 2024-04-25T00:15:04.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:04 smithi122 ceph-mon[32959]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:04.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:15:04.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:04.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:04 smithi122 ceph-mon[32959]: Deploying daemon osd.3 on smithi045 2024-04-25T00:15:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:04 smithi071 ceph-mon[32667]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:15:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:04 smithi071 ceph-mon[32667]: Deploying daemon osd.3 on smithi045 2024-04-25T00:15:06.656 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:06 smithi045 ceph-mon[29194]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:06.670 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:06 smithi071 ceph-mon[32667]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:06.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:06 smithi122 ceph-mon[32959]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:08.771 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:08 smithi045 ceph-mon[29194]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:08.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:08 smithi122 ceph-mon[32959]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:08.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:08 smithi071 ceph-mon[32667]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:10 smithi045 ceph-mon[29194]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:10 smithi122 ceph-mon[32959]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:10.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:10 smithi071 ceph-mon[32667]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:10.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:10.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:12 smithi045 ceph-mon[29194]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:12.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.735+0000 7fd4377fe700 1 -- 172.21.15.45:0/4242292081 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd424000bc0 con 0x7fd42c0a2b90 2024-04-25T00:15:12.736 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-04-25T00:15:12.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 msgr2=0x7fd42c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 0x7fd42c0a5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009bf0 tx=0x7fd43c009260 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 msgr2=0x7fd44019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd43000d9b0 tx=0x7fd43000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 shutdown_connections 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd42c0a2b90 0x7fd42c0a5050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd440107f70 0x7fd4401a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd440102d10 0x7fd44019a670 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 --2- 172.21.15.45:0/4242292081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd440105640 0x7fd44019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 >> 172.21.15.45:0/4242292081 conn(0x7fd4400fc720 msgr2=0x7fd440102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 shutdown_connections 2024-04-25T00:15:12.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:15:12.737+0000 7fd448478700 1 -- 172.21.15.45:0/4242292081 wait complete. 2024-04-25T00:15:12.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:12 smithi122 ceph-mon[32959]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:12 smithi071 ceph-mon[32667]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:13.431 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T00:15:13.434 INFO:tasks.cephadm:Deploying osd.4 on smithi071 with /dev/vg_nvme/lv_3... 2024-04-25T00:15:13.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:15:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:15:14.034 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.034 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.034 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:13 smithi071 ceph-mon[32667]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:15:14.035 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.035 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:13 smithi122 ceph-mon[32959]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:15:14.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:13 smithi045 ceph-mon[29194]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:15:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:14.988 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:14 smithi071 ceph-mon[32667]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:14.988 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:14 smithi071 ceph-mon[32667]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:15:14.988 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:14 smithi071 ceph-mon[32667]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:15:14.988 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:14 smithi071 ceph-mon[32667]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:15:14.989 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-mon[29194]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:14.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-mon[29194]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:15:14.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-mon[29194]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:15:14.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-mon[29194]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:15:14.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:14.999 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:15:14.728+0000 7fccdb4b2700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:15:14.999 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:15:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:15:14.733+0000 7fccd72a5700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:15:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:14 smithi122 ceph-mon[32959]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:14 smithi122 ceph-mon[32959]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:15:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:14 smithi122 ceph-mon[32959]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:15:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:14 smithi122 ceph-mon[32959]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:15:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:15.648 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:15:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:15:15.649 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-25T00:15:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-25T00:15:15.649 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124469 s, 842 MB/s 2024-04-25T00:15:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-25T00:15:15.968 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:15 smithi071 ceph-mon[32667]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:15:15.968 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:15 smithi071 ceph-mon[32667]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:15:15.968 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:15.968 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:15 smithi122 ceph-mon[32959]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:15:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:15 smithi122 ceph-mon[32959]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:15:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:15 smithi045 ceph-mon[29194]: from='osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:15:16.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:15 smithi045 ceph-mon[29194]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:15:16.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-04-25T00:15:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:15:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:15:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:15:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] boot 2024-04-25T00:15:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:15:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] boot 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:15:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: osd.3 [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] boot 2024-04-25T00:15:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:15:17.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi045 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-25T00:15:17.984 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:17.985 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:15:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi045 2024-04-25T00:15:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:18.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:15:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi045 2024-04-25T00:15:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.207+0000 7f2958a95700 1 -- 172.21.15.71:0/3234723460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954101180 msgr2=0x7f29541015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.207+0000 7f2958a95700 1 --2- 172.21.15.71:0/3234723460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954101180 0x7f29541015a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f29440099c0 tx=0x7f2944009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- 172.21.15.71:0/3234723460 shutdown_connections 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- 172.21.15.71:0/3234723460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102d20 0x7f29541095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- 172.21.15.71:0/3234723460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954101180 0x7f29541015a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- 172.21.15.71:0/3234723460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954102380 0x7f29541027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- 172.21.15.71:0/3234723460 >> 172.21.15.71:0/3234723460 conn(0x7f29540fc720 msgr2=0x7f29540feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- 172.21.15.71:0/3234723460 shutdown_connections 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- 172.21.15.71:0/3234723460 wait complete. 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 Processor -- start 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- start start 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954101180 0x7f29541984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102380 0x7f29541989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f295410cd60 con 0x7f2954101180 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f295410cbe0 con 0x7f2954102d20 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.208+0000 7f2958a95700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f295410cee0 con 0x7f2954102380 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f295259c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954101180 0x7f29541984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56612/0 (socket says 172.21.15.71:56612) 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 -- 172.21.15.71:0/2015019791 learned_addr learned my addr 172.21.15.71:0/2015019791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2951d9b700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102380 0x7f29541989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:18.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 -- 172.21.15.71:0/2015019791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102380 msgr2=0x7f29541989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102380 0x7f29541989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 -- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954101180 msgr2=0x7f29541984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954101180 0x7f29541984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 -- 172.21.15.71:0/2015019791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29541a0100 con 0x7f2954102d20 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2952d9d700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f294c00ada0 tx=0x7f294c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c017020 con 0x7f2954102d20 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c00cdb0 con 0x7f2954102d20 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c005310 con 0x7f2954102d20 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a03f0 con 0x7f2954102d20 2024-04-25T00:15:18.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.209+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a0980 con 0x7f2954102d20 2024-04-25T00:15:18.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.210+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2954068f20 con 0x7f2954102d20 2024-04-25T00:15:18.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.211+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f294c00c8d0 con 0x7f2954102d20 2024-04-25T00:15:18.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.212+0000 7f29437fe700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 0x7f293c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:18.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.212+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f294c0cda30 con 0x7f2954102d20 2024-04-25T00:15:18.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.212+0000 7f295259c700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 0x7f293c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:18.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.213+0000 7f295259c700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 0x7f293c0a5160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f294400b580 tx=0x7f2944005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:18.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.216+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f294c021020 con 0x7f2954102d20 2024-04-25T00:15:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:18.424+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f29540fe920 con 0x7f293c0a2ca0 2024-04-25T00:15:19.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:18 smithi071 ceph-mon[32667]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:19.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:18 smithi071 ceph-mon[32667]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:15:19.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:19.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:19.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:18 smithi122 ceph-mon[32959]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:18 smithi122 ceph-mon[32959]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:15:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:18 smithi045 ceph-mon[29194]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:18 smithi045 ceph-mon[29194]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:15:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:20.050 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:19 smithi071 ceph-mon[32667]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:20.050 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:19 smithi071 ceph-mon[32667]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:15:20.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:19 smithi122 ceph-mon[32959]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:20.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:19 smithi122 ceph-mon[32959]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:15:20.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:19 smithi045 ceph-mon[29194]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:20.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:19 smithi045 ceph-mon[29194]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:15:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:20 smithi122 ceph-mon[32959]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:20 smithi122 ceph-mon[32959]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:15:21.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:20 smithi071 ceph-mon[32667]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:21.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:20 smithi071 ceph-mon[32667]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:15:21.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:20 smithi045 ceph-mon[29194]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:21.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:20 smithi045 ceph-mon[29194]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/165166040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]': finished 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/3095380619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:22.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/165166040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]': finished 2024-04-25T00:15:22.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:15:22.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:22.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/3095380619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/165166040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]: dispatch 2024-04-25T00:15:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f29fc7-dd8a-45cc-b8b5-d837364a6001"}]': finished 2024-04-25T00:15:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:15:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/3095380619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:23 smithi071 ceph-mon[32667]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:23 smithi045 ceph-mon[29194]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:23 smithi122 ceph-mon[32959]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:15:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:25 smithi071 ceph-mon[32667]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T00:15:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:25 smithi045 ceph-mon[29194]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T00:15:25.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:25 smithi122 ceph-mon[32959]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T00:15:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:27 smithi071 ceph-mon[32667]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:27 smithi045 ceph-mon[29194]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:27.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:27 smithi122 ceph-mon[32959]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:29 smithi071 ceph-mon[32667]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:15:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:29 smithi045 ceph-mon[29194]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:15:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:29 smithi122 ceph-mon[32959]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:15:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:30.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:30 smithi071 ceph-mon[32667]: Deploying daemon osd.4 on smithi071 2024-04-25T00:15:30.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:30 smithi045 ceph-mon[29194]: Deploying daemon osd.4 on smithi071 2024-04-25T00:15:30.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:30 smithi122 ceph-mon[32959]: Deploying daemon osd.4 on smithi071 2024-04-25T00:15:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:31 smithi071 ceph-mon[32667]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:31 smithi045 ceph-mon[29194]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:31.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:31 smithi122 ceph-mon[32959]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:33 smithi071 ceph-mon[32667]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:33 smithi045 ceph-mon[29194]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:33 smithi122 ceph-mon[32959]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:34 smithi071 ceph-mon[32667]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:34.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:34.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:34 smithi045 ceph-mon[29194]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:34 smithi122 ceph-mon[32959]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:36.761 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:36 smithi071 ceph-mon[32667]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:36.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.840+0000 7f29437fe700 1 -- 172.21.15.71:0/2015019791 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f29540fe920 con 0x7f293c0a2ca0 2024-04-25T00:15:36.841 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 4 on host 'smithi071' 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 msgr2=0x7f293c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 0x7f293c0a5160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f294400b580 tx=0x7f2944005fb0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 msgr2=0x7f295419fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f294c00ada0 tx=0x7f294c00c330 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 shutdown_connections 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f293c0a2ca0 0x7f293c0a5160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954102d20 0x7f295419fa60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2954101180 0x7f29541984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 --2- 172.21.15.71:0/2015019791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2954102380 0x7f29541989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 >> 172.21.15.71:0/2015019791 conn(0x7f29540fc720 msgr2=0x7f29540fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 shutdown_connections 2024-04-25T00:15:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:15:36.843+0000 7f2958a95700 1 -- 172.21.15.71:0/2015019791 wait complete. 2024-04-25T00:15:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:36 smithi045 ceph-mon[29194]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:36 smithi122 ceph-mon[32959]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:37.864 DEBUG:teuthology.orchestra.run.smithi071:osd.4> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T00:15:37.866 INFO:tasks.cephadm:Deploying osd.5 on smithi122 with /dev/vg_nvme/lv_3... 2024-04-25T00:15:37.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:15:38.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:37 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:37 smithi122 ceph-mon[32959]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:37 smithi122 ceph-mon[32959]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.117 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:15:38.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:37 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:37 smithi071 ceph-mon[32667]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:37 smithi071 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:37 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:37 smithi045 ceph-mon[29194]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:37 smithi045 ceph-mon[29194]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:15:38.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:38.954 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:38.955 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:39.619 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:15:39.298+0000 7f5a5eb3d700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:15:39.619 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:15:39.304+0000 7f5a56928700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:15:40.025 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:15:40.026 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-mon[32667]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:15:40.026 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.026 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.026 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:39 smithi071 ceph-mon[32667]: from='osd.4 ' entity='osd.4' 2024-04-25T00:15:40.109 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:39 smithi122 ceph-mon[32959]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:15:40.109 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:39 smithi122 ceph-mon[32959]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:15:40.109 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:39 smithi122 ceph-mon[32959]: from='osd.4 ' entity='osd.4' 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111541 s, 940 MB/s 2024-04-25T00:15:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-25T00:15:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:39 smithi045 ceph-mon[29194]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:15:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:39 smithi045 ceph-mon[29194]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:15:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:39 smithi045 ceph-mon[29194]: from='osd.4 ' entity='osd.4' 2024-04-25T00:15:40.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-04-25T00:15:41.052 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:15:41.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] boot 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:15:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:40 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:41.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:15:41.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:15:41.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:41.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:41.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] boot 2024-04-25T00:15:41.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:15:41.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:40 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: osd.4 [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] boot 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:15:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:40 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:15:42.427 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:42.427 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi071 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi071 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.620 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.620 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.620 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.620 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 -- 172.21.15.122:0/1369151492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1011b0 msgr2=0x7f9a9c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 --2- 172.21.15.122:0/1369151492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c1015d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009990 tx=0x7f9a98009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 -- 172.21.15.122:0/1369151492 shutdown_connections 2024-04-25T00:15:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 --2- 172.21.15.122:0/1369151492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c1015d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 --2- 172.21.15.122:0/1369151492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a9c102d50 0x7f9a9c109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 --2- 172.21.15.122:0/1369151492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.645+0000 7f9aa341a700 1 -- 172.21.15.122:0/1369151492 >> 172.21.15.122:0/1369151492 conn(0x7f9a9c0fc790 msgr2=0x7f9a9c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- 172.21.15.122:0/1369151492 shutdown_connections 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- 172.21.15.122:0/1369151492 wait complete. 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 Processor -- start 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- start start 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a9c102d50 0x7f9a9c19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c10cdf0 con 0x7f9a9c1011b0 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c10cc70 con 0x7f9a9c102d50 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa341a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c10cf70 con 0x7f9a9c1023b0 2024-04-25T00:15:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa09b5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa09b5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32964/0 (socket says 172.21.15.122:32964) 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.646+0000 7f9aa09b5700 1 -- 172.21.15.122:0/3153459155 learned_addr learned my addr 172.21.15.122:0/3153459155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa11b6700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 -- 172.21.15.122:0/3153459155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a9c102d50 msgr2=0x7f9a9c19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a9c102d50 0x7f9a9c19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 -- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 msgr2=0x7f9a9c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 -- 172.21.15.122:0/3153459155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c1a01a0 con 0x7f9a9c1023b0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa11b6700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa09b5700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00d950 tx=0x7f9a8c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c016070 con 0x7f9a9c1023b0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c009b40 con 0x7f9a9c1023b0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c1a0490 con 0x7f9a9c1023b0 2024-04-25T00:15:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00f620 con 0x7f9a9c1023b0 2024-04-25T00:15:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.647+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c1a0a50 con 0x7f9a9c1023b0 2024-04-25T00:15:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.648+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9c068f20 con 0x7f9a9c1023b0 2024-04-25T00:15:42.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:42.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:15:42.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi071 2024-04-25T00:15:42.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:42.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.649+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9a8c019070 con 0x7f9a9c1023b0 2024-04-25T00:15:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.650+0000 7f9a927fc700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 0x7f9a880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.650+0000 7f9aa11b6700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 0x7f9a880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.650+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f9a8c0ce090 con 0x7f9a9c1023b0 2024-04-25T00:15:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.651+0000 7f9aa11b6700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 0x7f9a880a50e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a980039c0 tx=0x7f9a98005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.654+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9a8c09bfc0 con 0x7f9a9c1023b0 2024-04-25T00:15:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:15:42.855+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9a9c062670 con 0x7f9a880a2c20 2024-04-25T00:15:43.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:43 smithi122 ceph-mon[32959]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:43.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:43.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:43.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:43 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:43.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:43 smithi071 ceph-mon[32667]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:43.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:43.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:43.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:43 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:43.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:43 smithi045 ceph-mon[29194]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:43.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:43.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:43.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:43 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:44.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:44 smithi071 ceph-mon[32667]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:44.648 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:44 smithi122 ceph-mon[32959]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:44.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:44 smithi045 ceph-mon[29194]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/528104236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]': finished 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:15:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/942118095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/528104236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]': finished 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:15:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/942118095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/528104236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]: dispatch 2024-04-25T00:15:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fb2eebb-1460-46c7-bdf2-aa80df2b595d"}]': finished 2024-04-25T00:15:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:15:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:15:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/942118095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:48.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:48 smithi071 ceph-mon[32667]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:48.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:48 smithi045 ceph-mon[29194]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:48 smithi122 ceph-mon[32959]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:50.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:50 smithi122 ceph-mon[32959]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:50.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:50 smithi071 ceph-mon[32667]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:50.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:50 smithi045 ceph-mon[29194]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:52.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:52 smithi122 ceph-mon[32959]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:52.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:52 smithi071 ceph-mon[32667]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:52.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:52 smithi045 ceph-mon[29194]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:53.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:15:53.615 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:53.615 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:53 smithi122 ceph-mon[32959]: Deploying daemon osd.5 on smithi122 2024-04-25T00:15:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:15:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:53 smithi071 ceph-mon[32667]: Deploying daemon osd.5 on smithi122 2024-04-25T00:15:53.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:15:53.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:53.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:53 smithi045 ceph-mon[29194]: Deploying daemon osd.5 on smithi122 2024-04-25T00:15:54.507 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:54 smithi122 ceph-mon[32959]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:54.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:54 smithi071 ceph-mon[32667]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:54.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:54 smithi045 ceph-mon[29194]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:56.588 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:56 smithi122 ceph-mon[32959]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:56.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:56 smithi071 ceph-mon[32667]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:56.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:56 smithi045 ceph-mon[29194]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:58.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:58 smithi122 ceph-mon[32959]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:58.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:58 smithi071 ceph-mon[32667]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:58.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:58 smithi045 ceph-mon[29194]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:15:59.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:15:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:15:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:59.681 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.682 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:15:59.682 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:15:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:00.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:00 smithi122 ceph-mon[32959]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:00.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:00 smithi071 ceph-mon[32667]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:00.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:00 smithi045 ceph-mon[29194]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:01.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.923+0000 7f9a927fc700 1 -- 172.21.15.122:0/3153459155 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a9c062670 con 0x7f9a880a2c20 2024-04-25T00:16:01.924 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 5 on host 'smithi122' 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 msgr2=0x7f9a880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 0x7f9a880a50e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a980039c0 tx=0x7f9a98005fb0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 msgr2=0x7f9a9c19d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00d950 tx=0x7f9a8c00dd10 comp rx=0 tx=0).stop 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 shutdown_connections 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9a880a2c20 0x7f9a880a50e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a9c102d50 0x7f9a9c19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c1011b0 0x7f9a9c06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 --2- 172.21.15.122:0/3153459155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a9c1023b0 0x7f9a9c19d290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 >> 172.21.15.122:0/3153459155 conn(0x7f9a9c0fc790 msgr2=0x7f9a9c1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 shutdown_connections 2024-04-25T00:16:01.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:01.925+0000 7f9aa341a700 1 -- 172.21.15.122:0/3153459155 wait complete. 2024-04-25T00:16:02.541 DEBUG:teuthology.orchestra.run.smithi122:osd.5> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T00:16:02.543 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-04-25T00:16:02.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:16:02.800 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:03.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:03.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:02 smithi045 ceph-mon[29194]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:02 smithi122 ceph-mon[32959]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:02 smithi071 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:16:04.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:04 smithi045 ceph-mon[29194]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:16:04.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:04 smithi045 ceph-mon[29194]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:16:04.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:04 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:04.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:04 smithi045 ceph-mon[29194]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:04 smithi045 ceph-mon[29194]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-mon[32959]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:16:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-mon[32959]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:16:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-mon[32959]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-mon[32959]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:04 smithi071 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:16:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:04 smithi071 ceph-mon[32667]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:16:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:04 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:04 smithi071 ceph-mon[32667]: from='osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:04 smithi071 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:16:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:16:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:16:04.735 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T00:16:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T00:16:04.735 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852301 s, 1.2 GB/s 2024-04-25T00:16:04.736 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T00:16:04.807 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:16:04.419+0000 7f54a7c20700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:16:04.807 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:16:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:16:04.425+0000 7f549f20a700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:16:05.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:05.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:16:05.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:16:05.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.237 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:05 smithi122 ceph-mon[32959]: from='osd.5 ' entity='osd.5' 2024-04-25T00:16:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:16:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:16:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:05 smithi071 ceph-mon[32667]: from='osd.5 ' entity='osd.5' 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:05.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:05 smithi045 ceph-mon[29194]: from='osd.5 ' entity='osd.5' 2024-04-25T00:16:05.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-04-25T00:16:05.668 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:06.101 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] boot 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:06.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] boot 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:16:06.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:16:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:16:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: osd.5 [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] boot 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:06.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.250+0000 7f4606cb8700 1 -- 172.21.15.45:0/3599320797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001011d0 msgr2=0x7f46001015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.250+0000 7f4606cb8700 1 --2- 172.21.15.45:0/3599320797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001011d0 0x7f46001015f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f45f40099c0 tx=0x7f45f4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- 172.21.15.45:0/3599320797 shutdown_connections 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- 172.21.15.45:0/3599320797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600102d70 0x7f4600109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- 172.21.15.45:0/3599320797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001011d0 0x7f46001015f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- 172.21.15.45:0/3599320797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46001023d0 0x7f4600102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- 172.21.15.45:0/3599320797 >> 172.21.15.45:0/3599320797 conn(0x7f46000fc790 msgr2=0x7f46000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- 172.21.15.45:0/3599320797 shutdown_connections 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- 172.21.15.45:0/3599320797 wait complete. 2024-04-25T00:16:07.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 Processor -- start 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- start start 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 0x7f46001050a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4600102d70 0x7f4600103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f460010cdb0 con 0x7f46001023d0 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f460010cc30 con 0x7f46001011d0 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f4606cb8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f460010cf30 con 0x7f4600102d70 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.251+0000 7f45fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54312/0 (socket says 172.21.15.45:54312) 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 -- 172.21.15.45:0/2601099908 learned_addr learned my addr 172.21.15.45:0/2601099908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f4605255700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4600102d70 0x7f4600103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f4604a54700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 0x7f46001050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 -- 172.21.15.45:0/2601099908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4600102d70 msgr2=0x7f4600103cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4600102d70 0x7f4600103cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 -- 172.21.15.45:0/2601099908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 msgr2=0x7f46001050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 0x7f46001050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 -- 172.21.15.45:0/2601099908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4600104500 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f4604a54700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 0x7f46001050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fffff700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00d980 tx=0x7f45ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec016070 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec004d10 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a2650 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec005670 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.252+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a2b60 con 0x7f46001023d0 2024-04-25T00:16:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.253+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4600068f20 con 0x7f46001023d0 2024-04-25T00:16:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.254+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f45ec004750 con 0x7f46001023d0 2024-04-25T00:16:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.255+0000 7f45fdffb700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.255+0000 7f4604a54700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.255+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f45ec0cdff0 con 0x7f46001023d0 2024-04-25T00:16:07.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.256+0000 7f4604a54700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 0x7f45f00a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45f400b580 tx=0x7f45f4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.260+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45ec01f020 con 0x7f46001023d0 2024-04-25T00:16:07.285 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:07 smithi122 ceph-mon[32959]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:07.285 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:07 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi122 2024-04-25T00:16:07.285 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:07 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-25T00:16:07.286 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:07 smithi122 ceph-mon[32959]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:16:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:07 smithi071 ceph-mon[32667]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:07 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi122 2024-04-25T00:16:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:07 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-25T00:16:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:07 smithi071 ceph-mon[32667]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:16:07.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:07 smithi045 ceph-mon[29194]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:16:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:07 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi122 2024-04-25T00:16:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:07 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-25T00:16:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:07 smithi045 ceph-mon[29194]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:16:07.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:07.478+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f46000fe950 con 0x7f45f00a2c20 2024-04-25T00:16:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:08 smithi122 ceph-mon[32959]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:16:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:08.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:08 smithi045 ceph-mon[29194]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:16:08.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:08.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:08.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:08 smithi071 ceph-mon[32667]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:16:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:09.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:09 smithi122 ceph-mon[32959]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:09.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:09 smithi122 ceph-mon[32959]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:09.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:09 smithi122 ceph-mon[32959]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:16:09.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:09 smithi045 ceph-mon[29194]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:09.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:09 smithi045 ceph-mon[29194]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:09.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:09 smithi045 ceph-mon[29194]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:16:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:09 smithi071 ceph-mon[32667]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:09 smithi071 ceph-mon[32667]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:09 smithi071 ceph-mon[32667]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:16:10.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:10 smithi122 ceph-mon[32959]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:16:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:10 smithi071 ceph-mon[32667]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:16:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:10 smithi045 ceph-mon[29194]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]: dispatch 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]': finished 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:11.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4068967218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:16:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]: dispatch 2024-04-25T00:16:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]': finished 2024-04-25T00:16:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:16:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:11 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4068967218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:16:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]: dispatch 2024-04-25T00:16:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2606980802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6624e936-d5b3-40f7-9a81-cdca44201b04"}]': finished 2024-04-25T00:16:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:16:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:11 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4068967218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:13 smithi122 ceph-mon[32959]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T00:16:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:13 smithi071 ceph-mon[32667]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T00:16:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:13 smithi045 ceph-mon[29194]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T00:16:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:15 smithi122 ceph-mon[32959]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:16:15.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:15 smithi045 ceph-mon[29194]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:16:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:15 smithi071 ceph-mon[32667]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:16:17.293 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:17 smithi045 ceph-mon[29194]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:17 smithi122 ceph-mon[32959]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:17 smithi071 ceph-mon[32667]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:18.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:16:18.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:18.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:16:18.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:18.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:16:18.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:19 smithi071 ceph-mon[32667]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:19 smithi071 ceph-mon[32667]: Deploying daemon osd.6 on smithi045 2024-04-25T00:16:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:19 smithi045 ceph-mon[29194]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:19 smithi045 ceph-mon[29194]: Deploying daemon osd.6 on smithi045 2024-04-25T00:16:19.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:19 smithi122 ceph-mon[32959]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:19.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:19 smithi122 ceph-mon[32959]: Deploying daemon osd.6 on smithi045 2024-04-25T00:16:21.310 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:21 smithi045 ceph-mon[29194]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:21 smithi071 ceph-mon[32667]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:21 smithi122 ceph-mon[32959]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:23 smithi071 ceph-mon[32667]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:23 smithi045 ceph-mon[29194]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:23 smithi122 ceph-mon[32959]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:24.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:24 smithi071 ceph-mon[32667]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:24.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:24.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:24 smithi045 ceph-mon[29194]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:24.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:24.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:24 smithi122 ceph-mon[32959]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:24.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:24.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:26.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:26 smithi071 ceph-mon[32667]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:26.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:26 smithi045 ceph-mon[29194]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:26.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:26 smithi122 ceph-mon[32959]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:27.388 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-04-25T00:16:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.387+0000 7f45fdffb700 1 -- 172.21.15.45:0/2601099908 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46000fe950 con 0x7f45f00a2c20 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 msgr2=0x7f45f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 0x7f45f00a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45f400b580 tx=0x7f45f4005fb0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 msgr2=0x7f46001037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00d980 tx=0x7f45ec00dc90 comp rx=0 tx=0).stop 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 shutdown_connections 2024-04-25T00:16:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4600102d70 0x7f4600103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45f00a2c20 0x7f45f00a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46001011d0 0x7f46001050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 --2- 172.21.15.45:0/2601099908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46001023d0 0x7f46001037b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.389+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 >> 172.21.15.45:0/2601099908 conn(0x7f46000fc790 msgr2=0x7f46000fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.390+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 shutdown_connections 2024-04-25T00:16:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:16:27.390+0000 7f4606cb8700 1 -- 172.21.15.45:0/2601099908 wait complete. 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:27.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:27.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.005 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T00:16:28.007 INFO:tasks.cephadm:Deploying osd.7 on smithi071 with /dev/vg_nvme/lv_2... 2024-04-25T00:16:28.008 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.249 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:16:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:28 smithi071 ceph-mon[32667]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:28 smithi071 ceph-mon[32667]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:16:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:28 smithi071 ceph-mon[32667]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:16:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:28 smithi071 ceph-mon[32667]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:16:28.698 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:28 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:28 smithi045 ceph-mon[29194]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:28 smithi045 ceph-mon[29194]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:16:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:28 smithi045 ceph-mon[29194]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:16:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:28 smithi045 ceph-mon[29194]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:16:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:28 smithi122 ceph-mon[32959]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:28 smithi122 ceph-mon[32959]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:16:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:28 smithi122 ceph-mon[32959]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:16:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:28 smithi122 ceph-mon[32959]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:16:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:28 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.341 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:16:29.046+0000 7f8455f8d700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:16:29.341 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:16:29.055+0000 7f844dd78700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:16:29.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:29 smithi071 ceph-mon[32667]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:16:29.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:29 smithi071 ceph-mon[32667]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:16:29.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:29 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.730 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:29 smithi071 ceph-mon[32667]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' 2024-04-25T00:16:29.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-mon[29194]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:16:29.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-mon[29194]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:16:29.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:29.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:29 smithi045 ceph-mon[29194]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' 2024-04-25T00:16:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:29 smithi122 ceph-mon[32959]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:16:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:29 smithi122 ceph-mon[32959]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:16:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:29 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:29 smithi122 ceph-mon[32959]: from='osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117]' entity='osd.6' 2024-04-25T00:16:30.307 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:16:30.307 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:16:30.307 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-25T00:16:30.307 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-25T00:16:30.307 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112653 s, 931 MB/s 2024-04-25T00:16:30.308 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-25T00:16:30.642 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] boot 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:16:30.643 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] boot 2024-04-25T00:16:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:16:30.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:30.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: osd.6 [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] boot 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:16:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:31.217 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:32.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:32.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:32.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:32.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:32.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:32.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- 172.21.15.71:0/87784775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 msgr2=0x7fd10c102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 --2- 172.21.15.71:0/87784775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c102030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80099c0 tx=0x7fd0f8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- 172.21.15.71:0/87784775 shutdown_connections 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 --2- 172.21.15.71:0/87784775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c1009d0 0x7fd10c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 --2- 172.21.15.71:0/87784775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c102570 0x7fd10c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 --2- 172.21.15.71:0/87784775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c102030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- 172.21.15.71:0/87784775 >> 172.21.15.71:0/87784775 conn(0x7fd10c0fbf70 msgr2=0x7fd10c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- 172.21.15.71:0/87784775 shutdown_connections 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- 172.21.15.71:0/87784775 wait complete. 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 Processor -- start 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.848+0000 7fd112f81700 1 -- start start 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c1009d0 0x7fd10c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 0x7fd10c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10ce40 con 0x7fd10c1009d0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10ccc0 con 0x7fd10c101bd0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c10cfc0 con 0x7fd10c102570 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35598/0 (socket says 172.21.15.71:35598) 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 -- 172.21.15.71:0/853078435 learned_addr learned my addr 172.21.15.71:0/853078435 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd110d1d700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c1009d0 0x7fd10c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd11151e700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 0x7fd10c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 -- 172.21.15.71:0/853078435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 msgr2=0x7fd10c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 0x7fd10c19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 -- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c1009d0 msgr2=0x7fd10c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c1009d0 0x7fd10c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 -- 172.21.15.71:0/853078435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c1a0200 con 0x7fd10c101bd0 2024-04-25T00:16:32.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd11151e700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 0x7fd10c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd10bfff700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8027070 con 0x7fd10c101bd0 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c1a0490 con 0x7fd10c101bd0 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f8022950 con 0x7fd10c101bd0 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.849+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8009e60 con 0x7fd10c101bd0 2024-04-25T00:16:32.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.850+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c1a0940 con 0x7fd10c101bd0 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.850+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10c068f20 con 0x7fd10c101bd0 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.851+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0f802a070 con 0x7fd10c101bd0 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.852+0000 7fd109ffb700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 0x7fd0fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.852+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fd0f80cf040 con 0x7fd10c101bd0 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.852+0000 7fd110d1d700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 0x7fd0fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.853+0000 7fd110d1d700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 0x7fd0fc0a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd10c101a30 tx=0x7fd10000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:32.856+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd0f8013070 con 0x7fd10c101bd0 2024-04-25T00:16:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:33.062+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd10c0fe0b0 con 0x7fd0fc0a2c20 2024-04-25T00:16:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:33 smithi071 ceph-mon[32667]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:33 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi045 2024-04-25T00:16:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:33 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-25T00:16:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:33 smithi045 ceph-mon[29194]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:33 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi045 2024-04-25T00:16:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:33 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-25T00:16:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:33 smithi122 ceph-mon[32959]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:33 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi045 2024-04-25T00:16:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:33 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-25T00:16:34.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:34 smithi071 ceph-mon[32667]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:34.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:34.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:34.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:34 smithi045 ceph-mon[29194]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:34.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:34 smithi122 ceph-mon[32959]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:34.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:34.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:34.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:35 smithi071 ceph-mon[32667]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:35 smithi045 ceph-mon[29194]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:35.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:35 smithi122 ceph-mon[32959]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:36 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/3405032897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:36 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:36 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]': finished 2024-04-25T00:16:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:36 smithi071 ceph-mon[32667]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:16:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:36 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:36 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/3405032897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:36 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:36 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]': finished 2024-04-25T00:16:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:36 smithi045 ceph-mon[29194]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:16:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:36 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:36.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:36 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/3405032897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:36 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]: dispatch 2024-04-25T00:16:36.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:36 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e86e97d-8982-4535-9be0-961d692f1472"}]': finished 2024-04-25T00:16:36.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:36 smithi122 ceph-mon[32959]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:16:36.458 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:36 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:37 smithi071 ceph-mon[32667]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:37.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:37 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/3003492293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:37.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:37 smithi045 ceph-mon[29194]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:37.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:37 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/3003492293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:37.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:37 smithi122 ceph-mon[32959]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:37.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:37 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/3003492293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:16:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:39 smithi071 ceph-mon[32667]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:39 smithi045 ceph-mon[29194]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:39 smithi122 ceph-mon[32959]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:41 smithi071 ceph-mon[32667]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:41 smithi045 ceph-mon[29194]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:41.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:41 smithi122 ceph-mon[32959]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:43 smithi071 ceph-mon[32667]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:43 smithi045 ceph-mon[29194]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:43 smithi122 ceph-mon[32959]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:44.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:16:44.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:44 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:44.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:44 smithi071 ceph-mon[32667]: Deploying daemon osd.7 on smithi071 2024-04-25T00:16:44.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:16:44.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:44 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:44.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:44 smithi045 ceph-mon[29194]: Deploying daemon osd.7 on smithi071 2024-04-25T00:16:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:16:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:44 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:44 smithi122 ceph-mon[32959]: Deploying daemon osd.7 on smithi071 2024-04-25T00:16:45.362 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:45 smithi071 ceph-mon[32667]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:45 smithi045 ceph-mon[29194]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:45 smithi122 ceph-mon[32959]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:47 smithi071 ceph-mon[32667]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:47 smithi045 ceph-mon[29194]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:47.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:47 smithi122 ceph-mon[32959]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:49 smithi071 ceph-mon[32667]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:49.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:49 smithi045 ceph-mon[29194]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:49 smithi122 ceph-mon[32959]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:51 smithi071 ceph-mon[32667]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:51 smithi045 ceph-mon[29194]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:51.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:51 smithi122 ceph-mon[32959]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.632+0000 7fd109ffb700 1 -- 172.21.15.71:0/853078435 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd10c0fe0b0 con 0x7fd0fc0a2c20 2024-04-25T00:16:52.635 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 7 on host 'smithi071' 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.634+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 msgr2=0x7fd0fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.634+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 0x7fd0fc0a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd10c101a30 tx=0x7fd10000b410 comp rx=0 tx=0).stop 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 msgr2=0x7fd10c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f800fa00 comp rx=0 tx=0).stop 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 shutdown_connections 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10c102570 0x7fd10c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd0fc0a2c20 0x7fd0fc0a50e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c1009d0 0x7fd10c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 --2- 172.21.15.71:0/853078435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd10c101bd0 0x7fd10c198ae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 >> 172.21.15.71:0/853078435 conn(0x7fd10c0fbf70 msgr2=0x7fd10c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 shutdown_connections 2024-04-25T00:16:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:16:52.635+0000 7fd112f81700 1 -- 172.21.15.71:0/853078435 wait complete. 2024-04-25T00:16:52.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:52.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:52.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:52.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:52.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:52.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:52.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:52.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:52.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:52.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:52.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:53.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:53.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:16:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:53.267 DEBUG:teuthology.orchestra.run.smithi071:osd.7> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T00:16:53.269 INFO:tasks.cephadm:Deploying osd.8 on smithi122 with /dev/vg_nvme/lv_2... 2024-04-25T00:16:53.269 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:16:53.519 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:16:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:54 smithi122 ceph-mon[32959]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:16:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:54 smithi122 ceph-mon[32959]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:16:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:54 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:54 smithi122 ceph-mon[32959]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:54 smithi122 ceph-mon[32959]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:16:54.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-mon[32667]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:16:54.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:54.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-mon[32667]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:54 smithi045 ceph-mon[29194]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:16:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:54 smithi045 ceph-mon[29194]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:16:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:54 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:54 smithi045 ceph-mon[29194]: from='osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:54 smithi045 ceph-mon[29194]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:16:54.818 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:16:54.515+0000 7fbc75823700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:16:54.818 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:16:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:16:54.528+0000 7fbc6d60e700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:55 smithi045 ceph-mon[29194]: from='osd.7 ' entity='osd.7' 2024-04-25T00:16:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:16:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:16:55.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:55 smithi122 ceph-mon[32959]: from='osd.7 ' entity='osd.7' 2024-04-25T00:16:55.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:16:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:16:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:16:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:55 smithi071 ceph-mon[32667]: from='osd.7 ' entity='osd.7' 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111268 s, 942 MB/s 2024-04-25T00:16:55.663 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-25T00:16:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:56 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:16:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:56 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:16:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:56 smithi122 ceph-mon[32959]: osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] boot 2024-04-25T00:16:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:56 smithi122 ceph-mon[32959]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:16:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:56.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:56 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:16:56.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:56 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:16:56.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:56 smithi071 ceph-mon[32667]: osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] boot 2024-04-25T00:16:56.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:56 smithi071 ceph-mon[32667]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:16:56.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:56.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-04-25T00:16:56.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:56 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:16:56.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:56 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:16:56.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:56 smithi045 ceph-mon[29194]: osd.7 [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] boot 2024-04-25T00:16:56.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:56 smithi045 ceph-mon[29194]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:16:56.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:56.630 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:16:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:57 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:16:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:57 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:57.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:57 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:16:58.345 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:58 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi071 2024-04-25T00:16:58.345 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:58 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-25T00:16:58.345 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:58 smithi122 ceph-mon[32959]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:16:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 -- 172.21.15.122:0/525215980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54102390 msgr2=0x7f3b541027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/525215980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54102390 0x7f3b541027f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009990 tx=0x7f3b50009ca0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 -- 172.21.15.122:0/525215980 shutdown_connections 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/525215980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102d30 0x7f3b541095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/525215980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54101190 0x7f3b541015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/525215980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54102390 0x7f3b541027f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.345+0000 7f3b5bd93700 1 -- 172.21.15.122:0/525215980 >> 172.21.15.122:0/525215980 conn(0x7f3b540fc730 msgr2=0x7f3b540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- 172.21.15.122:0/525215980 shutdown_connections 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- 172.21.15.122:0/525215980 wait complete. 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 Processor -- start 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- start start 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102390 0x7f3b54197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54102d30 0x7f3b5419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410cde0 con 0x7f3b54102d30 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410cc60 con 0x7f3b54102390 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5bd93700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410cf60 con 0x7f3b54101190 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b59b2f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b59b2f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58436/0 (socket says 172.21.15.122:58436) 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.346+0000 7f3b5a330700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54102d30 0x7f3b5419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 -- 172.21.15.122:0/4272273184 learned_addr learned my addr 172.21.15.122:0/4272273184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b5932e700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102390 0x7f3b54197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 -- 172.21.15.122:0/4272273184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102390 msgr2=0x7f3b54197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102390 0x7f3b54197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 -- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54102d30 msgr2=0x7f3b5419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54102d30 0x7f3b5419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 -- 172.21.15.122:0/4272273184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5419f050 con 0x7f3b54101190 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b59b2f700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d9b0 tx=0x7f3b4400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44017030 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5406aa20 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b44004500 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44003970 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.347+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5406b030 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.348+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b54068f20 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.349+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3b4401a070 con 0x7f3b54101190 2024-04-25T00:16:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.350+0000 7f3b4affd700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:58.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.350+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f3b440ce100 con 0x7f3b54101190 2024-04-25T00:16:58.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.350+0000 7f3b5932e700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:58.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.351+0000 7f3b5932e700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 0x7f3b400a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b5c0 tx=0x7f3b50005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:58.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.354+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b4400e350 con 0x7f3b54101190 2024-04-25T00:16:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:58 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi071 2024-04-25T00:16:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:58 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-25T00:16:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:58 smithi045 ceph-mon[29194]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:16:58.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:16:58.556+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3b5406b2f0 con 0x7f3b400a2c20 2024-04-25T00:16:58.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:58 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi071 2024-04-25T00:16:58.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:58 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-25T00:16:58.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:58 smithi071 ceph-mon[32667]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:16:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:59 smithi045 ceph-mon[29194]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:59 smithi045 ceph-mon[29194]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:16:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:16:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:59.414 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:59 smithi122 ceph-mon[32959]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:59.414 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:59 smithi122 ceph-mon[32959]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:16:59.414 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:59.415 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:59.415 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:16:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:59.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:59 smithi071 ceph-mon[32667]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:16:59.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:59 smithi071 ceph-mon[32667]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:16:59.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:16:59.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:16:59.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:16:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:00 smithi045 ceph-mon[29194]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:00 smithi045 ceph-mon[29194]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:17:00.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:00 smithi122 ceph-mon[32959]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:00.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:00 smithi122 ceph-mon[32959]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:17:00.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:00 smithi071 ceph-mon[32667]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:00.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:00 smithi071 ceph-mon[32667]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:17:01.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:01 smithi045 ceph-mon[29194]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:01.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:01 smithi122 ceph-mon[32959]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:01.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:01 smithi071 ceph-mon[32667]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/252613192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]': finished 2024-04-25T00:17:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:17:02.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:02.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/3555227209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:02.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/252613192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]': finished 2024-04-25T00:17:02.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:17:02.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:02.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/3555227209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/252613192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]: dispatch 2024-04-25T00:17:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a36bdfc-d527-4419-a2d2-03a09912a395"}]': finished 2024-04-25T00:17:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:17:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/3555227209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:03 smithi122 ceph-mon[32959]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:03.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:03 smithi071 ceph-mon[32667]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:03.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:03 smithi045 ceph-mon[29194]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:04.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:04 smithi122 ceph-mon[32959]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:04.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:04 smithi071 ceph-mon[32667]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:04 smithi045 ceph-mon[29194]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:06.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:06 smithi122 ceph-mon[32959]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:06.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:06 smithi071 ceph-mon[32667]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:06.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:06 smithi045 ceph-mon[29194]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:08.552 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:08 smithi122 ceph-mon[32959]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:08.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:08 smithi071 ceph-mon[32667]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:08.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:08 smithi045 ceph-mon[29194]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:09.553 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:09 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:17:09.553 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:09 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:09.553 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:09 smithi122 ceph-mon[32959]: Deploying daemon osd.8 on smithi122 2024-04-25T00:17:09.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:09 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:17:09.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:09 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:09.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:09 smithi071 ceph-mon[32667]: Deploying daemon osd.8 on smithi122 2024-04-25T00:17:09.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:09 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:17:09.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:09 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:09.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:09 smithi045 ceph-mon[29194]: Deploying daemon osd.8 on smithi122 2024-04-25T00:17:10.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:10 smithi122 ceph-mon[32959]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:10.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:10 smithi071 ceph-mon[32667]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:10 smithi045 ceph-mon[29194]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:12.399 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:12 smithi122 ceph-mon[32959]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:12.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:12 smithi071 ceph-mon[32667]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:12 smithi045 ceph-mon[29194]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:14.443 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:14 smithi122 ceph-mon[32959]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:14.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:14 smithi071 ceph-mon[32667]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:14 smithi045 ceph-mon[29194]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:16.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:16.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:16.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:17.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:16 smithi071 ceph-mon[32667]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:16 smithi045 ceph-mon[29194]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:16 smithi122 ceph-mon[32959]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:18.827 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 8 on host 'smithi122' 2024-04-25T00:17:18.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.826+0000 7f3b4affd700 1 -- 172.21.15.122:0/4272273184 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b5406b2f0 con 0x7f3b400a2c20 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 msgr2=0x7f3b400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 0x7f3b400a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b5c0 tx=0x7f3b50005fd0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 msgr2=0x7f3b541973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d9b0 tx=0x7f3b4400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 shutdown_connections 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b54101190 0x7f3b541973f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3b400a2c20 0x7f3b400a50e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b54102d30 0x7f3b5419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 --2- 172.21.15.122:0/4272273184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b54102390 0x7f3b54197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:18.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 >> 172.21.15.122:0/4272273184 conn(0x7f3b540fc730 msgr2=0x7f3b54105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:18.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.828+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 shutdown_connections 2024-04-25T00:17:18.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:17:18.829+0000 7f3b5bd93700 1 -- 172.21.15.122:0/4272273184 wait complete. 2024-04-25T00:17:19.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:19.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:19.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:18 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:18 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:17:19.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:19.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:18 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:19.517 DEBUG:teuthology.orchestra.run.smithi122:osd.8> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T00:17:19.520 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-04-25T00:17:19.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:17:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:20.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:19 smithi045 ceph-mon[29194]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:17:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:19 smithi045 ceph-mon[29194]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:17:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:19 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:19 smithi045 ceph-mon[29194]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:19 smithi045 ceph-mon[29194]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:19 smithi122 ceph-mon[32959]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:17:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:19 smithi122 ceph-mon[32959]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:17:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:19 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:19 smithi122 ceph-mon[32959]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:19 smithi122 ceph-mon[32959]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:19 smithi071 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:17:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:19 smithi071 ceph-mon[32667]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:17:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:19 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:19 smithi071 ceph-mon[32667]: from='osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:19 smithi071 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:17:20.711 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:17:20.368+0000 7ff04b7e6700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T00:17:20.712 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:17:20.379+0000 7ff0435d1700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.189 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:20 smithi045 ceph-mon[29194]: from='osd.8 ' entity='osd.8' 2024-04-25T00:17:21.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:21.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:17:21.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:17:21.258 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.258 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.258 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:20 smithi122 ceph-mon[32959]: from='osd.8 ' entity='osd.8' 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:20 smithi071 ceph-mon[32667]: from='osd.8 ' entity='osd.8' 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00847993 s, 1.2 GB/s 2024-04-25T00:17:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] boot 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:22.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:21 smithi122 ceph-mon[32959]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] boot 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:22.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:21 smithi045 ceph-mon[29194]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:17:22.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:17:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:17:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: osd.8 [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] boot 2024-04-25T00:17:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T00:17:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:17:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:21 smithi071 ceph-mon[32667]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:17:22.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-04-25T00:17:22.776 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:22 smithi045 ceph-mon[29194]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:17:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:22 smithi122 ceph-mon[32959]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:17:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:22 smithi071 ceph-mon[32667]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:17:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:23 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi122 2024-04-25T00:17:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:23 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-25T00:17:24.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:23 smithi045 ceph-mon[29194]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:17:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:23 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi122 2024-04-25T00:17:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:23 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-25T00:17:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:23 smithi122 ceph-mon[32959]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:17:24.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:23 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi122 2024-04-25T00:17:24.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:23 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-25T00:17:24.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:23 smithi071 ceph-mon[32667]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 -- 172.21.15.45:0/2097733032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab101022b0 msgr2=0x7fab101026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 --2- 172.21.15.45:0/2097733032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab101022b0 0x7fab101026d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0099c0 tx=0x7fab0c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 -- 172.21.15.45:0/2097733032 shutdown_connections 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 --2- 172.21.15.45:0/2097733032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab10103e50 0x7fab1010a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 --2- 172.21.15.45:0/2097733032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab101022b0 0x7fab101026d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 --2- 172.21.15.45:0/2097733032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101034b0 0x7fab10103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 -- 172.21.15.45:0/2097733032 >> 172.21.15.45:0/2097733032 conn(0x7fab100fd850 msgr2=0x7fab100ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 -- 172.21.15.45:0/2097733032 shutdown_connections 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.406+0000 7fab187b6700 1 -- 172.21.15.45:0/2097733032 wait complete. 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 Processor -- start 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 -- start start 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 0x7fab1019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab101034b0 0x7fab1019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010def0 con 0x7fab10103e50 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010dd70 con 0x7fab101034b0 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab187b6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab1010e070 con 0x7fab101022b0 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab16d53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab16d53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38240/0 (socket says 172.21.15.45:38240) 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab16d53700 1 -- 172.21.15.45:0/2029478220 learned_addr learned my addr 172.21.15.45:0/2029478220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:17:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.407+0000 7fab15d51700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab101034b0 0x7fab1019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16552700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 0x7fab1019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 -- 172.21.15.45:0/2029478220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 msgr2=0x7fab1019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 0x7fab1019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 -- 172.21.15.45:0/2029478220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab101034b0 msgr2=0x7fab1019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab101034b0 0x7fab1019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 -- 172.21.15.45:0/2029478220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab101a67b0 con 0x7fab10103e50 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16552700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 0x7fab1019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab16d53700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab08007da0 tx=0x7fab0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08016020 con 0x7fab10103e50 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab101a6aa0 con 0x7fab10103e50 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab0800ad00 con 0x7fab10103e50 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101a6fe0 con 0x7fab10103e50 2024-04-25T00:17:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.408+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08011620 con 0x7fab10103e50 2024-04-25T00:17:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.409+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf4005360 con 0x7fab10103e50 2024-04-25T00:17:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.411+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab08019020 con 0x7fab10103e50 2024-04-25T00:17:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.411+0000 7fab077fe700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 0x7faafc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.411+0000 7fab16552700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 0x7faafc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.411+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7fab080ce8c0 con 0x7fab10103e50 2024-04-25T00:17:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.412+0000 7fab16552700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 0x7faafc0a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.415+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab08020020 con 0x7fab10103e50 2024-04-25T00:17:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:24.630+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7faaf4000bf0 con 0x7faafc0a2ca0 2024-04-25T00:17:25.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:24 smithi045 ceph-mon[29194]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:25.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:25.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:25.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:25.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:24 smithi045 ceph-mon[29194]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:17:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:24 smithi122 ceph-mon[32959]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:25.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:24 smithi122 ceph-mon[32959]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:17:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:24 smithi071 ceph-mon[32667]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:24 smithi071 ceph-mon[32667]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:17:26.282 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:25 smithi045 ceph-mon[29194]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:25 smithi122 ceph-mon[32959]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:26.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:25 smithi071 ceph-mon[32667]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:26 smithi122 ceph-mon[32959]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:17:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:26 smithi071 ceph-mon[32667]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:17:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:26 smithi045 ceph-mon[29194]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:17:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]: dispatch 2024-04-25T00:17:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]': finished 2024-04-25T00:17:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:27 smithi122 ceph-mon[32959]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T00:17:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]: dispatch 2024-04-25T00:17:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]': finished 2024-04-25T00:17:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:27 smithi071 ceph-mon[32667]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T00:17:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]: dispatch 2024-04-25T00:17:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/279546589' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8856abc5-be19-468e-9205-df339f75e018"}]': finished 2024-04-25T00:17:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:27 smithi045 ceph-mon[29194]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T00:17:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:28 smithi122 ceph-mon[32959]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:17:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:28 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2696995086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:28 smithi071 ceph-mon[32667]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:17:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:28 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2696995086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:28 smithi045 ceph-mon[29194]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:17:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:28 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2696995086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:30 smithi122 ceph-mon[32959]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:17:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:30 smithi071 ceph-mon[32667]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:17:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:30 smithi045 ceph-mon[29194]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:17:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:32 smithi122 ceph-mon[32959]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T00:17:33.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:32 smithi045 ceph-mon[29194]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T00:17:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:32 smithi071 ceph-mon[32667]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T00:17:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:35 smithi122 ceph-mon[32959]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:17:35.359 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:35 smithi045 ceph-mon[29194]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:17:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:35 smithi071 ceph-mon[32667]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T00:17:36.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:36 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:17:36.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:36 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:36 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:17:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:36 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:36.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:36 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:17:36.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:36 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:37 smithi122 ceph-mon[32959]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:37 smithi122 ceph-mon[32959]: Deploying daemon osd.9 on smithi045 2024-04-25T00:17:37.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:37 smithi045 ceph-mon[29194]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:37.335 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:37 smithi045 ceph-mon[29194]: Deploying daemon osd.9 on smithi045 2024-04-25T00:17:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:37 smithi071 ceph-mon[32667]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:37 smithi071 ceph-mon[32667]: Deploying daemon osd.9 on smithi045 2024-04-25T00:17:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:39 smithi045 ceph-mon[29194]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:39 smithi122 ceph-mon[32959]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:39 smithi071 ceph-mon[32667]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:41 smithi122 ceph-mon[32959]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:41 smithi071 ceph-mon[32667]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:41 smithi045 ceph-mon[29194]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:42.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:42 smithi122 ceph-mon[32959]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:42.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:42.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:42.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:42.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:42 smithi071 ceph-mon[32667]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:42.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:42 smithi045 ceph-mon[29194]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:44.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:44 smithi122 ceph-mon[32959]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:44.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:44 smithi045 ceph-mon[29194]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:44.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:44 smithi071 ceph-mon[32667]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:45.293 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-04-25T00:17:45.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.292+0000 7fab077fe700 1 -- 172.21.15.45:0/2029478220 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faaf4000bf0 con 0x7faafc0a2ca0 2024-04-25T00:17:45.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 msgr2=0x7faafc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:45.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 0x7faafc0a5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 msgr2=0x7fab101a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab08007da0 tx=0x7fab0800a340 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 shutdown_connections 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab101022b0 0x7fab1019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faafc0a2ca0 0x7faafc0a5160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab10103e50 0x7fab101a6110 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 --2- 172.21.15.45:0/2029478220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab101034b0 0x7fab1019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 >> 172.21.15.45:0/2029478220 conn(0x7fab100fd850 msgr2=0x7fab10074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 shutdown_connections 2024-04-25T00:17:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:17:45.295+0000 7fab187b6700 1 -- 172.21.15.45:0/2029478220 wait complete. 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:45 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:17:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:45 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:45.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:45.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:45 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:45.918 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T00:17:45.921 INFO:tasks.cephadm:Deploying osd.10 on smithi071 with /dev/vg_nvme/lv_1... 2024-04-25T00:17:45.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:17:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:17:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:46 smithi071 ceph-mon[32667]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:46 smithi071 ceph-mon[32667]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:17:46.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:46 smithi071 ceph-mon[32667]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:17:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:46 smithi071 ceph-mon[32667]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:17:46.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:46 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-mon[29194]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-mon[29194]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:17:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-mon[29194]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:17:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-mon[29194]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:17:46.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:46 smithi122 ceph-mon[32959]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:46 smithi122 ceph-mon[32959]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:17:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:46 smithi122 ceph-mon[32959]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:17:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:46 smithi122 ceph-mon[32959]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T00:17:46.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:46 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.112 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:17:46.879+0000 7f96c1b0b700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T00:17:47.112 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:17:46 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:17:46.892+0000 7f96b98f6700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:17:47.592 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:47 smithi071 ceph-mon[32667]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:17:47.593 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:47 smithi071 ceph-mon[32667]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:17:47.593 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:47 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.593 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:47 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.593 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:47 smithi071 ceph-mon[32667]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' 2024-04-25T00:17:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:47 smithi122 ceph-mon[32959]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:17:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:47 smithi122 ceph-mon[32959]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:17:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:47 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:47 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:47 smithi122 ceph-mon[32959]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' 2024-04-25T00:17:47.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:47 smithi045 ceph-mon[29194]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T00:17:47.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:47 smithi045 ceph-mon[29194]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:17:47.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:47 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:47 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:47.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:47 smithi045 ceph-mon[29194]: from='osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854]' entity='osd.9' 2024-04-25T00:17:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:17:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:17:48.304 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-25T00:17:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-25T00:17:48.305 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114707 s, 914 MB/s 2024-04-25T00:17:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] boot 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T00:17:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:48 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] boot 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T00:17:48.620 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:48 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: osd.9 [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] boot 2024-04-25T00:17:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T00:17:48.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:48 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:17:48.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-04-25T00:17:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi045 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:49 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:17:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi045 2024-04-25T00:17:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:50.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:49 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi045 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:49 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:17:51.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/3437527776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 msgr2=0x7f6bd01027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/3437527776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd01027e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0099c0 tx=0x7f6bbc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/3437527776 shutdown_connections 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/3437527776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0102d20 0x7f6bd01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/3437527776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0101180 0x7f6bd01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/3437527776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd01027e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/3437527776 >> 172.21.15.71:0/3437527776 conn(0x7f6bd00fc720 msgr2=0x7f6bd00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/3437527776 shutdown_connections 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.028+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/3437527776 wait complete. 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 Processor -- start 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 -- start start 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0101180 0x7f6bd01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0102d20 0x7f6bd019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010cdd0 con 0x7f6bd0102d20 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010cc50 con 0x7f6bd0102380 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd6f9c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd010cf50 con 0x7f6bd0101180 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44614/0 (socket says 172.21.15.71:44614) 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 -- 172.21.15.71:0/38112254 learned_addr learned my addr 172.21.15.71:0/38112254 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd5539700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0102d20 0x7f6bd019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bd4d38700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0101180 0x7f6bd01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 -- 172.21.15.71:0/38112254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0101180 msgr2=0x7f6bd01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0101180 0x7f6bd01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 -- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0102d20 msgr2=0x7f6bd019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0102d20 0x7f6bd019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.029+0000 7f6bcffff700 1 -- 172.21.15.71:0/38112254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd01a0140 con 0x7f6bd0102380 2024-04-25T00:17:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bcffff700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005910 tx=0x7f6bbc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc027070 con 0x7f6bd0102380 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd01a03d0 con 0x7f6bd0102380 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc004150 con 0x7f6bd0102380 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc022e60 con 0x7f6bd0102380 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.030+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a0900 con 0x7f6bd0102380 2024-04-25T00:17:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.031+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd0068f20 con 0x7f6bd0102380 2024-04-25T00:17:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.032+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6bbc02a070 con 0x7f6bd0102380 2024-04-25T00:17:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.033+0000 7f6bcdffb700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 0x7f6bc00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.033+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f6bbc0cf820 con 0x7f6bd0102380 2024-04-25T00:17:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.033+0000 7f6bd4d38700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 0x7f6bc00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.033+0000 7f6bd4d38700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 0x7f6bc00a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4005950 tx=0x7f6bc40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:51.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.036+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6bbc013070 con 0x7f6bd0102380 2024-04-25T00:17:51.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:17:51.241+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6bd01a0c20 con 0x7f6bc00a2c20 2024-04-25T00:17:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:51 smithi071 ceph-mon[32667]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:51 smithi045 ceph-mon[29194]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:51.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:51 smithi122 ceph-mon[32959]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:52.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:52.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:52.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:52.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:52.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:17:52.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:17:52.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:53 smithi071 ceph-mon[32667]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:53 smithi071 ceph-mon[32667]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:53 smithi045 ceph-mon[29194]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:53 smithi045 ceph-mon[29194]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:53 smithi122 ceph-mon[32959]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:53 smithi122 ceph-mon[32959]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:17:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:54 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/2250644847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:54 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:54 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]': finished 2024-04-25T00:17:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:54 smithi045 ceph-mon[29194]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T00:17:54.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:54 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:17:54.434 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:54 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/2250644847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.434 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:54 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.434 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:54 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]': finished 2024-04-25T00:17:54.435 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:54 smithi071 ceph-mon[32667]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T00:17:54.435 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:54 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:17:54.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:54 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/2250644847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:54 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]: dispatch 2024-04-25T00:17:54.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:54 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7602a6e-0267-4732-90f6-794201613999"}]': finished 2024-04-25T00:17:54.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:54 smithi122 ceph-mon[32959]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T00:17:54.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:54 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:17:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:55 smithi045 ceph-mon[29194]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:55 smithi045 ceph-mon[29194]: from='client.? 172.21.15.71:0/3396534419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:55 smithi122 ceph-mon[32959]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:55 smithi122 ceph-mon[32959]: from='client.? 172.21.15.71:0/3396534419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:55.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:55 smithi071 ceph-mon[32667]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:55.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:55 smithi071 ceph-mon[32667]: from='client.? 172.21.15.71:0/3396534419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:17:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:57 smithi045 ceph-mon[29194]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:57.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:57 smithi122 ceph-mon[32959]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:57.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:57 smithi071 ceph-mon[32667]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:59.340 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:17:59 smithi071 ceph-mon[32667]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:17:59 smithi045 ceph-mon[29194]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:17:59.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:17:59 smithi122 ceph-mon[32959]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:01.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:01 smithi045 ceph-mon[29194]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:01.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:01 smithi071 ceph-mon[32667]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:01.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:01 smithi122 ceph-mon[32959]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:02.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:18:02.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:18:02.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:18:02.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:03.478 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:03 smithi071 ceph-mon[32667]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:03.479 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:03 smithi071 ceph-mon[32667]: Deploying daemon osd.10 on smithi071 2024-04-25T00:18:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:03 smithi122 ceph-mon[32959]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:03 smithi122 ceph-mon[32959]: Deploying daemon osd.10 on smithi071 2024-04-25T00:18:03.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:03 smithi045 ceph-mon[29194]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:03.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:03 smithi045 ceph-mon[29194]: Deploying daemon osd.10 on smithi071 2024-04-25T00:18:05.406 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:05 smithi071 ceph-mon[32667]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:05 smithi122 ceph-mon[32959]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:05.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:05 smithi045 ceph-mon[29194]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:07.531 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:07 smithi071 ceph-mon[32667]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:07.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:07 smithi122 ceph-mon[32959]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:07.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:07 smithi045 ceph-mon[29194]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:08.355 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.355 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.355 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:09 smithi071 ceph-mon[32667]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:09 smithi122 ceph-mon[32959]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:09.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:09 smithi045 ceph-mon[29194]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:10.349 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:10 smithi071 ceph-mon[32667]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:10.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:10 smithi122 ceph-mon[32959]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:10.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:10 smithi045 ceph-mon[29194]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:11.058 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 10 on host 'smithi071' 2024-04-25T00:18:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.057+0000 7f6bcdffb700 1 -- 172.21.15.71:0/38112254 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6bd01a0c20 con 0x7f6bc00a2c20 2024-04-25T00:18:11.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 msgr2=0x7f6bc00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 0x7f6bc00a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4005950 tx=0x7f6bc40058e0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 msgr2=0x7f6bd0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005910 tx=0x7f6bbc009580 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 shutdown_connections 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6bd0101180 0x7f6bd01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6bc00a2c20 0x7f6bc00a50e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0102d20 0x7f6bd019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 --2- 172.21.15.71:0/38112254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bd0102380 0x7f6bd0198a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 >> 172.21.15.71:0/38112254 conn(0x7f6bd00fc720 msgr2=0x7f6bd00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 shutdown_connections 2024-04-25T00:18:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-25T00:18:11.060+0000 7f6bd6f9c700 1 -- 172.21.15.71:0/38112254 wait complete. 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:18:11.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:11.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.738 DEBUG:teuthology.orchestra.run.smithi071:osd.10> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T00:18:11.740 INFO:tasks.cephadm:Deploying osd.11 on smithi122 with /dev/vg_nvme/lv_1... 2024-04-25T00:18:11.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:18:11.986 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:12.404 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:12.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:18:12.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:12.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:18:12.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:12.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:12 smithi122 ceph-mon[32959]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:12.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:12.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:18:12.325+0000 7fded6b08700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T00:18:12.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:18:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:18:12.335+0000 7fded28fb700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: from='osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:12 smithi045 ceph-mon[29194]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-25T00:18:13.471 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:13 smithi122 ceph-mon[32959]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:18:13.472 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:13 smithi122 ceph-mon[32959]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:18:13.472 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.472 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.472 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:13 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:13 smithi071 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:18:13.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:13 smithi071 ceph-mon[32667]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:18:13.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:13 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:13 smithi045 ceph-mon[29194]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-25T00:18:13.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:13 smithi045 ceph-mon[29194]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:18:13.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:13.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:13 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:14.192 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:18:14.192 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:18:14.192 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-25T00:18:14.192 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-25T00:18:14.193 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112708 s, 930 MB/s 2024-04-25T00:18:14.193 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] boot 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T00:18:14.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:14.513 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:18:14.513 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:18:14.514 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:14.514 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] boot 2024-04-25T00:18:14.514 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T00:18:14.514 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: osd.10 [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] boot 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T00:18:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:18:14.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-04-25T00:18:15.187 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi071 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:15.564 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:15.564 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:15 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi071 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:15.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:15.620 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:15 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi071 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:15 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:16.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:16 smithi122 ceph-mon[32959]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:16.596 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:16 smithi122 ceph-mon[32959]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:18:16.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:16 smithi071 ceph-mon[32667]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:16.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:16 smithi071 ceph-mon[32667]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:18:16.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:16 smithi045 ceph-mon[29194]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:16.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:16 smithi045 ceph-mon[29194]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 -- 172.21.15.122:0/2340870289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 msgr2=0x7f55701015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 --2- 172.21.15.122:0/2340870289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f55701015b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5560009990 tx=0x7f5560009ca0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 -- 172.21.15.122:0/2340870289 shutdown_connections 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 --2- 172.21.15.122:0/2340870289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f55701015b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 --2- 172.21.15.122:0/2340870289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102d30 0x7f55701095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 --2- 172.21.15.122:0/2340870289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102390 0x7f55701027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.957+0000 7f5575e4f700 1 -- 172.21.15.122:0/2340870289 >> 172.21.15.122:0/2340870289 conn(0x7f55700fc730 msgr2=0x7f55700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- 172.21.15.122:0/2340870289 shutdown_connections 2024-04-25T00:18:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- 172.21.15.122:0/2340870289 wait complete. 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 Processor -- start 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- start start 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 0x7f5570103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102d30 0x7f5570103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f557010cd70 con 0x7f5570102390 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f557010cbf0 con 0x7f5570102d30 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f5575e4f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f557010cef0 con 0x7f5570101190 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f556f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60426/0 (socket says 172.21.15.122:60426) 2024-04-25T00:18:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.958+0000 7f556f7fe700 1 -- 172.21.15.122:0/1293957065 learned_addr learned my addr 172.21.15.122:0/1293957065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 -- 172.21.15.122:0/1293957065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102d30 msgr2=0x7f5570103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556ffff700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102d30 0x7f5570103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f5567fff700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 0x7f5570103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102d30 0x7f5570103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 -- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 msgr2=0x7f5570103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 0x7f5570103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 -- 172.21.15.122:0/1293957065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701041f0 con 0x7f5570101190 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f5567fff700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 0x7f5570103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556f7fe700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f556000b5c0 tx=0x7f5560004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560026070 con 0x7f5570101190 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5570104480 con 0x7f5570101190 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55600041d0 con 0x7f5570101190 2024-04-25T00:18:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560005430 con 0x7f5570101190 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.959+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701a6f80 con 0x7f5570101190 2024-04-25T00:18:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.960+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570068f20 con 0x7f5570101190 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.961+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5560029070 con 0x7f5570101190 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.962+0000 7f556d7fa700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 0x7f555c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.962+0000 7f5567fff700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 0x7f555c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.962+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f55600cfa50 con 0x7f5570101190 2024-04-25T00:18:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.963+0000 7f5567fff700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 0x7f555c0a50e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:16.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:16.966+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5560030020 con 0x7f5570101190 2024-04-25T00:18:17.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:17.169+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55700fe980 con 0x7f555c0a2c20 2024-04-25T00:18:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:17 smithi122 ceph-mon[32959]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:18:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:18:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:18:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:17.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:17 smithi071 ceph-mon[32667]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:18:17.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:18:17.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:18:17.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:17.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:17 smithi045 ceph-mon[29194]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:18:17.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:18:17.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:18:17.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:18.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:18 smithi071 ceph-mon[32667]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:18.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:18 smithi071 ceph-mon[32667]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:18.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:18 smithi071 ceph-mon[32667]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:18:18.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:18 smithi045 ceph-mon[29194]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:18.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:18 smithi045 ceph-mon[29194]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:18.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:18 smithi045 ceph-mon[29194]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:18:18.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:18 smithi122 ceph-mon[32959]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:18.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:18 smithi122 ceph-mon[32959]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:18.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:18 smithi122 ceph-mon[32959]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:18:20.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:20.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/90129307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]': finished 2024-04-25T00:18:20.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T00:18:20.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/90129307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]': finished 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T00:18:20.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/90129307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]: dispatch 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f89083ac-5fb2-4e72-ad19-20637464344f"}]': finished 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T00:18:20.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:21.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.122:0/4096933486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:18:21.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.122:0/4096933486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:18:21.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.122:0/4096933486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:18:22.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:22 smithi071 ceph-mon[32667]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T00:18:22.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:22 smithi045 ceph-mon[29194]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T00:18:22.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:22 smithi122 ceph-mon[32959]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T00:18:24.509 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:24 smithi122 ceph-mon[32959]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T00:18:24.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:24 smithi071 ceph-mon[32667]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T00:18:24.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:24 smithi045 ceph-mon[29194]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T00:18:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:26 smithi122 ceph-mon[32959]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:18:26.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:26 smithi071 ceph-mon[32667]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:18:26.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:26 smithi045 ceph-mon[29194]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:18:28.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:28 smithi122 ceph-mon[32959]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:28.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:28 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:18:28.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:28 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:28.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:28 smithi122 ceph-mon[32959]: Deploying daemon osd.11 on smithi122 2024-04-25T00:18:28.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:28 smithi071 ceph-mon[32667]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:28.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:28 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:18:28.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:28 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:28.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:28 smithi071 ceph-mon[32667]: Deploying daemon osd.11 on smithi122 2024-04-25T00:18:28.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:28 smithi045 ceph-mon[29194]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:28.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:18:28.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:28.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:28 smithi045 ceph-mon[29194]: Deploying daemon osd.11 on smithi122 2024-04-25T00:18:30.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:30 smithi045 ceph-mon[29194]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:30.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:30 smithi122 ceph-mon[32959]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:30.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:30 smithi071 ceph-mon[32667]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:18:32.512 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:32 smithi122 ceph-mon[32959]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:32.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:32 smithi045 ceph-mon[29194]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:32.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:32 smithi071 ceph-mon[32667]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:34.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:34 smithi122 ceph-mon[32959]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:34.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:34 smithi071 ceph-mon[32667]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:34 smithi045 ceph-mon[29194]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:36.724 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:36 smithi122 ceph-mon[32959]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:36.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:36 smithi071 ceph-mon[32667]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:36 smithi045 ceph-mon[29194]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:37.093 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 11 on host 'smithi122' 2024-04-25T00:18:37.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.092+0000 7f556d7fa700 1 -- 172.21.15.122:0/1293957065 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55700fe980 con 0x7f555c0a2c20 2024-04-25T00:18:37.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.094+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 msgr2=0x7f555c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 0x7f555c0a50e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005dc0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 msgr2=0x7f5570105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f556000b5c0 tx=0x7f5560004580 comp rx=0 tx=0).stop 2024-04-25T00:18:37.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 shutdown_connections 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5570101190 0x7f5570105120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f555c0a2c20 0x7f555c0a50e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5570102d30 0x7f5570103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 --2- 172.21.15.122:0/1293957065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5570102390 0x7f5570103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 >> 172.21.15.122:0/1293957065 conn(0x7f55700fc730 msgr2=0x7f55700fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 shutdown_connections 2024-04-25T00:18:37.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:37.095+0000 7f5575e4f700 1 -- 172.21.15.122:0/1293957065 wait complete. 2024-04-25T00:18:37.748 DEBUG:teuthology.orchestra.run.smithi122:osd.11> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T00:18:37.751 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T00:18:37.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:18:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:38.313 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:38 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:38 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:38.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:38 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-mon[32959]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:18:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:18:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:39 smithi071 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: from='osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:39 smithi045 ceph-mon[29194]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-25T00:18:39.806 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:18:39.536+0000 7f1197b4a700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T00:18:39.806 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:18:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:18:39.549+0000 7f118f935700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:18:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/3758431792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 msgr2=0x7f8cc8102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/3758431792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc8102800 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0099c0 tx=0x7f8cbc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/3758431792 shutdown_connections 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/3758431792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc81011a0 0x7f8cc81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/3758431792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc8102d40 0x7f8cc81095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/3758431792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc8102800 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/3758431792 >> 172.21.15.45:0/3758431792 conn(0x7f8cc80fc760 msgr2=0x7f8cc80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/3758431792 shutdown_connections 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/3758431792 wait complete. 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.829+0000 7f8ccfbd7700 1 Processor -- start 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 -- start start 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc81011a0 0x7f8cc806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc8102d40 0x7f8cc819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc810ce10 con 0x7f8cc81023a0 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc810cc90 con 0x7f8cc81011a0 2024-04-25T00:18:39.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccfbd7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc810cf90 con 0x7f8cc8102d40 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40208/0 (socket says 172.21.15.45:40208) 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 -- 172.21.15.45:0/2356591722 learned_addr learned my addr 172.21.15.45:0/2356591722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8cce174700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc8102d40 0x7f8cc819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd973700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc81011a0 0x7f8cc806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 -- 172.21.15.45:0/2356591722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc8102d40 msgr2=0x7f8cc819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc8102d40 0x7f8cc819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 -- 172.21.15.45:0/2356591722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc81011a0 msgr2=0x7f8cc806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc81011a0 0x7f8cc806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 -- 172.21.15.45:0/2356591722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc81a01f0 con 0x7f8cc81023a0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8ccd172700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc006010 tx=0x7f8cbc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.830+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc026070 con 0x7f8cc81023a0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.831+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc004150 con 0x7f8cc81023a0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.831+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc81a0480 con 0x7f8cc81023a0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.831+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc004a40 con 0x7f8cc81023a0 2024-04-25T00:18:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.831+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81a09b0 con 0x7f8cc81023a0 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.832+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc8068f20 con 0x7f8cc81023a0 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.833+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8cbc029070 con 0x7f8cc81023a0 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.833+0000 7f8cbaffd700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 0x7f8cb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.834+0000 7f8ccd973700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 0x7f8cb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.834+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f8cbc0cf4c0 con 0x7f8cc81023a0 2024-04-25T00:18:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.834+0000 7f8ccd973700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 0x7f8cb40a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005950 tx=0x7f8cc40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:39.838+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8cbc05d740 con 0x7f8cc81023a0 2024-04-25T00:18:40.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.045+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8cc8062e40 con 0x7f8cc81023a0 2024-04-25T00:18:40.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.045+0000 7f8cbaffd700 1 -- 172.21.15.45:0/2356591722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f8cbc013070 con 0x7f8cc81023a0 2024-04-25T00:18:40.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 msgr2=0x7f8cb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 0x7f8cb40a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005950 tx=0x7f8cc40058e0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 msgr2=0x7f8cc819d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc006010 tx=0x7f8cbc009f10 comp rx=0 tx=0).stop 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 shutdown_connections 2024-04-25T00:18:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cc8102d40 0x7f8cc819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8cb40a2ca0 0x7f8cb40a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc81011a0 0x7f8cc806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 --2- 172.21.15.45:0/2356591722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc81023a0 0x7f8cc819d2e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 >> 172.21.15.45:0/2356591722 conn(0x7f8cc80fc760 msgr2=0x7f8cc8105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.047+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 shutdown_connections 2024-04-25T00:18:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:40.048+0000 7f8ccfbd7700 1 -- 172.21.15.45:0/2356591722 wait complete. 2024-04-25T00:18:40.343 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:18:40.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:18:40.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: from='osd.11 ' entity='osd.11' 2024-04-25T00:18:40.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:40 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2356591722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:18:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:18:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: from='osd.11 ' entity='osd.11' 2024-04-25T00:18:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:40 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2356591722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:40.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-25T00:18:40.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:18:40.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:40.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: from='osd.11 ' entity='osd.11' 2024-04-25T00:18:40.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:40 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2356591722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:40.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1714004293,"num_in_osds":12,"osd_in_since":1714004300,"num_remapped_pgs":0} 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: purged_snaps scrub starts 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: purged_snaps scrub ok 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] boot 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T00:18:41.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:41 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T00:18:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T00:18:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] boot 2024-04-25T00:18:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T00:18:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:41 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: purged_snaps scrub starts 2024-04-25T00:18:41.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: purged_snaps scrub ok 2024-04-25T00:18:41.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:18:41.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.460 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: osd.11 [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] boot 2024-04-25T00:18:41.460 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T00:18:41.460 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:41 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:18:41.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:18:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: Detected new or changed devices on smithi122 2024-04-25T00:18:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:42 smithi122 ceph-mon[32959]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:18:42.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: Detected new or changed devices on smithi122 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:42.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:42 smithi071 ceph-mon[32667]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: Detected new or changed devices on smithi122 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:42.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:42 smithi045 ceph-mon[29194]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:18:43.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 -- 172.21.15.45:0/1795647468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c101190 msgr2=0x7ff34c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 --2- 172.21.15.45:0/1795647468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c101190 0x7ff34c1015b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0099c0 tx=0x7ff33c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 -- 172.21.15.45:0/1795647468 shutdown_connections 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 --2- 172.21.15.45:0/1795647468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 0x7ff34c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 --2- 172.21.15.45:0/1795647468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c101190 0x7ff34c1015b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 --2- 172.21.15.45:0/1795647468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c102390 0x7ff34c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 -- 172.21.15.45:0/1795647468 >> 172.21.15.45:0/1795647468 conn(0x7ff34c0fc730 msgr2=0x7ff34c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 -- 172.21.15.45:0/1795647468 shutdown_connections 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.772+0000 7ff3527f9700 1 -- 172.21.15.45:0/1795647468 wait complete. 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 Processor -- start 2024-04-25T00:18:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 -- start start 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 0x7ff34c1050d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 0x7ff34c103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10cd50 con 0x7ff34c102390 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10cbd0 con 0x7ff34c101190 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff3527f9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c10ced0 con 0x7ff34c102d30 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff34b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff34b7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41616/0 (socket says 172.21.15.45:41616) 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff34b7fe700 1 -- 172.21.15.45:0/4082383536 learned_addr learned my addr 172.21.15.45:0/4082383536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:43.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.773+0000 7ff34bfff700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 0x7ff34c1050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff350d96700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 0x7ff34c103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 -- 172.21.15.45:0/4082383536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 msgr2=0x7ff34c103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 0x7ff34c103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 -- 172.21.15.45:0/4082383536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 msgr2=0x7ff34c1050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 0x7ff34c1050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 -- 172.21.15.45:0/4082383536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c104580 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34bfff700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 0x7ff34c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff34b7fe700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff34000c920 tx=0x7ff34000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340007b00 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff340004d10 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1a2650 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34000f450 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.774+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a2bb0 con 0x7ff34c102390 2024-04-25T00:18:43.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.775+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c068f20 con 0x7ff34c102390 2024-04-25T00:18:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.776+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff340004750 con 0x7ff34c102390 2024-04-25T00:18:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.777+0000 7ff3497fa700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 0x7ff3340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.777+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3400cebe0 con 0x7ff34c102390 2024-04-25T00:18:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.777+0000 7ff34bfff700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 0x7ff3340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.777+0000 7ff34bfff700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 0x7ff3340a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c003960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.781+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff340014030 con 0x7ff34c102390 2024-04-25T00:18:43.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.990+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff34c1a3430 con 0x7ff34c102390 2024-04-25T00:18:43.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.990+0000 7ff3497fa700 1 -- 172.21.15.45:0/4082383536 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff34009bfe0 con 0x7ff34c102390 2024-04-25T00:18:43.991 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 msgr2=0x7ff3340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 0x7ff3340a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c003960 comp rx=0 tx=0).stop 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 msgr2=0x7ff34c103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff34000c920 tx=0x7ff34000cce0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 shutdown_connections 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff34c102d30 0x7ff34c103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff3340a2ca0 0x7ff3340a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff34c101190 0x7ff34c1050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 --2- 172.21.15.45:0/4082383536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff34c102390 0x7ff34c103770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 >> 172.21.15.45:0/4082383536 conn(0x7ff34c0fc730 msgr2=0x7ff34c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 shutdown_connections 2024-04-25T00:18:43.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:43.992+0000 7ff3527f9700 1 -- 172.21.15.45:0/4082383536 wait complete. 2024-04-25T00:18:44.520 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:44 smithi045 ceph-mon[29194]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:44.521 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:44 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4082383536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:44.551 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714004320,"num_in_osds":12,"osd_in_since":1714004300,"num_remapped_pgs":0} 2024-04-25T00:18:44.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:18:44.800 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:44.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:44 smithi122 ceph-mon[32959]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:44.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:44 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4082383536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:44.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:44 smithi071 ceph-mon[32667]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:44.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:44 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4082383536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:18:46.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- 172.21.15.45:0/3672744910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c1013c0 msgr2=0x7fdc4c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:46.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 --2- 172.21.15.45:0/3672744910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c1017a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0099c0 tx=0x7fdc3c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- 172.21.15.45:0/3672744910 shutdown_connections 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 --2- 172.21.15.45:0/3672744910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c106450 0x7fdc4c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 --2- 172.21.15.45:0/3672744910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c1017a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 --2- 172.21.15.45:0/3672744910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c101d70 0x7fdc4c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- 172.21.15.45:0/3672744910 >> 172.21.15.45:0/3672744910 conn(0x7fdc4c0780e0 msgr2=0x7fdc4c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- 172.21.15.45:0/3672744910 shutdown_connections 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- 172.21.15.45:0/3672744910 wait complete. 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 Processor -- start 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.432+0000 7fdc529ad700 1 -- start start 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c101d70 0x7fdc4c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c111460 con 0x7fdc4c106450 2024-04-25T00:18:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1112e0 con 0x7fdc4c1013c0 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc529ad700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1115e0 con 0x7fdc4c101d70 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41634/0 (socket says 172.21.15.45:41634) 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 -- 172.21.15.45:0/1656742960 learned_addr learned my addr 172.21.15.45:0/1656742960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc4b7fe700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c101d70 0x7fdc4c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc4bfff700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 -- 172.21.15.45:0/1656742960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c101d70 msgr2=0x7fdc4c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c101d70 0x7fdc4c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 -- 172.21.15.45:0/1656742960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c1013c0 msgr2=0x7fdc4c199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 -- 172.21.15.45:0/1656742960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c1a4e40 con 0x7fdc4c106450 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.433+0000 7fdc50f4a700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc44007ee0 tx=0x7fdc4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.434+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400dbd0 con 0x7fdc4c106450 2024-04-25T00:18:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.434+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c1a5070 con 0x7fdc4c106450 2024-04-25T00:18:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.434+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4400dd70 con 0x7fdc4c106450 2024-04-25T00:18:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.434+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44005a60 con 0x7fdc4c106450 2024-04-25T00:18:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.434+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c1a55d0 con 0x7fdc4c106450 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.435+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c068f20 con 0x7fdc4c106450 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.436+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc4401a020 con 0x7fdc4c106450 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.436+0000 7fdc497fa700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 0x7fdc340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.437+0000 7fdc4bfff700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 0x7fdc340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.437+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdc440ce2b0 con 0x7fdc4c106450 2024-04-25T00:18:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.437+0000 7fdc4bfff700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 0x7fdc340a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.441+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc44010070 con 0x7fdc4c106450 2024-04-25T00:18:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.650+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc4c02dcf0 con 0x7fdc4c106450 2024-04-25T00:18:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.651+0000 7fdc497fa700 1 -- 172.21.15.45:0/1656742960 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18444 (secure 0 0 0) 0x7fdc4409b610 con 0x7fdc4c106450 2024-04-25T00:18:46.652 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:46.652 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:18:42.079045+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 msgr2=0x7fdc340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 0x7fdc340a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 msgr2=0x7fdc4c1a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc44007ee0 tx=0x7fdc4400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 shutdown_connections 2024-04-25T00:18:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdc340a2c20 0x7fdc340a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4c106450 0x7fdc4c1a4900 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc4c1013c0 0x7fdc4c199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 --2- 172.21.15.45:0/1656742960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc4c101d70 0x7fdc4c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 >> 172.21.15.45:0/1656742960 conn(0x7fdc4c0780e0 msgr2=0x7fdc4c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 shutdown_connections 2024-04-25T00:18:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:46.653+0000 7fdc529ad700 1 -- 172.21.15.45:0/1656742960 wait complete. 2024-04-25T00:18:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:46 smithi122 ceph-mon[32959]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:46.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:46 smithi071 ceph-mon[32667]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:46.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:46 smithi045 ceph-mon[29194]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:47.318 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:47.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T00:18:47.566 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:47.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1656742960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:47.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1656742960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:47.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1656742960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:48.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:48 smithi045 ceph-mon[29194]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:48 smithi122 ceph-mon[32959]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:48.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:48 smithi071 ceph-mon[32667]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 -- 172.21.15.45:0/522507336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406ac40 msgr2=0x7fdda406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 --2- 172.21.15.45:0/522507336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406ac40 0x7fdda406b020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdda00099c0 tx=0x7fdda0009cd0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 -- 172.21.15.45:0/522507336 shutdown_connections 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 --2- 172.21.15.45:0/522507336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda4107590 0x7fdda410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 --2- 172.21.15.45:0/522507336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406ac40 0x7fdda406b020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 --2- 172.21.15.45:0/522507336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda406b5f0 0x7fdda4106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.273+0000 7fddabc16700 1 -- 172.21.15.45:0/522507336 >> 172.21.15.45:0/522507336 conn(0x7fdda40781d0 msgr2=0x7fdda40785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- 172.21.15.45:0/522507336 shutdown_connections 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- 172.21.15.45:0/522507336 wait complete. 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 Processor -- start 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- start start 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda406ac40 0x7fdda419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda4107590 0x7fdda419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4112660 con 0x7fdda406b5f0 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41124e0 con 0x7fdda406ac40 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.274+0000 7fddabc16700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41127e0 con 0x7fdda4107590 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41642/0 (socket says 172.21.15.45:41642) 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 -- 172.21.15.45:0/310696423 learned_addr learned my addr 172.21.15.45:0/310696423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:49.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fddaa1b3700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda4107590 0x7fdda419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda99b2700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda406ac40 0x7fdda419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 -- 172.21.15.45:0/310696423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda4107590 msgr2=0x7fdda419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda4107590 0x7fdda419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 -- 172.21.15.45:0/310696423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda406ac40 msgr2=0x7fdda419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda406ac40 0x7fdda419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 -- 172.21.15.45:0/310696423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda419fd60 con 0x7fdda406b5f0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdda91b1700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d950 tx=0x7fdd9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94017070 con 0x7fdda406b5f0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd94005020 con 0x7fdda406b5f0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda41a0050 con 0x7fdda406b5f0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400fbc0 con 0x7fdda406b5f0 2024-04-25T00:18:49.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.275+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a78d0 con 0x7fdda406b5f0 2024-04-25T00:18:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.276+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda4068f20 con 0x7fdda406b5f0 2024-04-25T00:18:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.278+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdd9400fd60 con 0x7fdda406b5f0 2024-04-25T00:18:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.279+0000 7fdd9affd700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 0x7fdd900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.279+0000 7fdda99b2700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 0x7fdd900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.279+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdd940ce740 con 0x7fdda406b5f0 2024-04-25T00:18:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.279+0000 7fdda99b2700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 0x7fdd900a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdda0000c00 tx=0x7fdda0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.283+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd9409bb40 con 0x7fdda406b5f0 2024-04-25T00:18:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.488+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdda41a7aa0 con 0x7fdda406b5f0 2024-04-25T00:18:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.489+0000 7fdd9affd700 1 -- 172.21.15.45:0/310696423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fdd9409b440 con 0x7fdda406b5f0 2024-04-25T00:18:49.489 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:18:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 msgr2=0x7fdd900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 0x7fdd900a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdda0000c00 tx=0x7fdda0005fb0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 msgr2=0x7fdda419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d950 tx=0x7fdd9400dc60 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 shutdown_connections 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdda4107590 0x7fdda419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fdd900a2c20 0x7fdd900a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda406ac40 0x7fdda419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 --2- 172.21.15.45:0/310696423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdda406b5f0 0x7fdda419b290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 >> 172.21.15.45:0/310696423 conn(0x7fdda40781d0 msgr2=0x7fdda40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 shutdown_connections 2024-04-25T00:18:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:49.491+0000 7fddabc16700 1 -- 172.21.15.45:0/310696423 wait complete. 2024-04-25T00:18:49.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:49 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/310696423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:49.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:49 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/310696423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:49.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:49 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/310696423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.b on smithi071 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.e on smithi071 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.h on smithi071 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.k on smithi071 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.c on smithi122 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.f on smithi122 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.i on smithi122 2024-04-25T00:18:50.239 INFO:tasks.cephadm:Adding mds.l on smithi122 2024-04-25T00:18:50.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l' 2024-04-25T00:18:50.490 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:18:50.774 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:50 smithi122 ceph-mon[32959]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:50.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:50 smithi045 ceph-mon[29194]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:50.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:50 smithi071 ceph-mon[32667]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 -- 172.21.15.122:0/1465985542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560107f80 msgr2=0x7ff56010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 --2- 172.21.15.122:0/1465985542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560107f80 0x7ff56010a3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009990 tx=0x7ff55c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 -- 172.21.15.122:0/1465985542 shutdown_connections 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 --2- 172.21.15.122:0/1465985542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560107f80 0x7ff56010a3f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 --2- 172.21.15.122:0/1465985542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560102d20 0x7ff560105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 --2- 172.21.15.122:0/1465985542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 0x7ff560107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 -- 172.21.15.122:0/1465985542 >> 172.21.15.122:0/1465985542 conn(0x7ff5600fc730 msgr2=0x7ff5600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 -- 172.21.15.122:0/1465985542 shutdown_connections 2024-04-25T00:18:52.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.234+0000 7ff5675f2700 1 -- 172.21.15.122:0/1465985542 wait complete. 2024-04-25T00:18:52.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 Processor -- start 2024-04-25T00:18:52.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 -- start start 2024-04-25T00:18:52.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:52.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 0x7ff560198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 0x7ff56019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010df40 con 0x7ff560107f80 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010ddc0 con 0x7ff560105650 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff5675f2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010e0c0 con 0x7ff560102d20 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50886/0 (socket says 172.21.15.122:50886) 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 -- 172.21.15.122:0/2711280754 learned_addr learned my addr 172.21.15.122:0/2711280754 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff565b8f700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 0x7ff56019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff564b8d700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 0x7ff560198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 -- 172.21.15.122:0/2711280754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 msgr2=0x7ff560198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 0x7ff560198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 -- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 msgr2=0x7ff56019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 0x7ff56019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 -- 172.21.15.122:0/2711280754 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a0270 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff565b8f700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 0x7ff56019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.235+0000 7ff56538e700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff55800ba80 tx=0x7ff55800be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.236+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558019070 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.236+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a0560 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.236+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff558014440 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.236+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5580075d0 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.236+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a0b70 con 0x7ff560102d20 2024-04-25T00:18:52.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.237+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560068f20 con 0x7ff560102d20 2024-04-25T00:18:52.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.238+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff55801c070 con 0x7ff560102d20 2024-04-25T00:18:52.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.238+0000 7ff5567fc700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 0x7ff54c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:52.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.239+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5580ce040 con 0x7ff560102d20 2024-04-25T00:18:52.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.239+0000 7ff564b8d700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 0x7ff54c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:52.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.239+0000 7ff564b8d700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 0x7ff54c0a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:52.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.243+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff55809b440 con 0x7ff560102d20 2024-04-25T00:18:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.454+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}) -- 0x7ff560062770 con 0x7ff54c0a2c20 2024-04-25T00:18:52.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.462+0000 7ff5567fc700 1 -- 172.21.15.122:0/2711280754 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff560062770 con 0x7ff54c0a2c20 2024-04-25T00:18:52.462 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mds.all update... 2024-04-25T00:18:52.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 msgr2=0x7ff54c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:52.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 0x7ff54c0a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 msgr2=0x7ff560198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff55800ba80 tx=0x7ff55800be40 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 shutdown_connections 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff560102d20 0x7ff560198610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff54c0a2c20 0x7ff54c0a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff560107f80 0x7ff56019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 --2- 172.21.15.122:0/2711280754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff560105650 0x7ff560198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 >> 172.21.15.122:0/2711280754 conn(0x7ff5600fc730 msgr2=0x7ff560103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 shutdown_connections 2024-04-25T00:18:52.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-25T00:18:52.464+0000 7ff5675f2700 1 -- 172.21.15.122:0/2711280754 wait complete. 2024-04-25T00:18:52.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:52 smithi122 ceph-mon[32959]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:52.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:52.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:52 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:52.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:52 smithi071 ceph-mon[32667]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:52.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:52.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:52 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:52.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:52 smithi045 ceph-mon[29194]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:52.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:52.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:52 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:53.173 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T00:18:53.176 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T00:18:53.219 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T00:18:53.262 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T00:18:53.305 DEBUG:teuthology.orchestra.run.smithi071:mds.b> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T00:18:53.308 DEBUG:teuthology.orchestra.run.smithi071:mds.e> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T00:18:53.351 DEBUG:teuthology.orchestra.run.smithi071:mds.h> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T00:18:53.394 DEBUG:teuthology.orchestra.run.smithi071:mds.k> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T00:18:53.437 DEBUG:teuthology.orchestra.run.smithi122:mds.c> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T00:18:53.439 DEBUG:teuthology.orchestra.run.smithi122:mds.f> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T00:18:53.482 DEBUG:teuthology.orchestra.run.smithi122:mds.i> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T00:18:53.525 DEBUG:teuthology.orchestra.run.smithi122:mds.l> sudo journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T00:18:53.568 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T00:18:53.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-25T00:18:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:53 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- 172.21.15.45:0/876370921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4102250 msgr2=0x7fb1e4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/876370921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4102250 0x7fb1e4108ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009ca0 tx=0x7fb1dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- 172.21.15.45:0/876370921 shutdown_connections 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/876370921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4102250 0x7fb1e4108ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/876370921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e4101890 0x7fb1e4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/876370921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e41006a0 0x7fb1e4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- 172.21.15.45:0/876370921 >> 172.21.15.45:0/876370921 conn(0x7fb1e40fbc30 msgr2=0x7fb1e40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- 172.21.15.45:0/876370921 shutdown_connections 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- 172.21.15.45:0/876370921 wait complete. 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 Processor -- start 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.682+0000 7fb1ebc15640 1 -- start start 2024-04-25T00:18:53.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e41006a0 0x7fb1e410b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e4102250 0x7fb1e4109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410d440 con 0x7fb1e4101890 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410d2c0 con 0x7fb1e4102250 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410d5c0 con 0x7fb1e41006a0 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41660/0 (socket says 172.21.15.45:41660) 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 -- 172.21.15.45:0/1172716130 learned_addr learned my addr 172.21.15.45:0/1172716130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e99b1640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e41006a0 0x7fb1e410b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ea1b2640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e4102250 0x7fb1e4109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 -- 172.21.15.45:0/1172716130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e41006a0 msgr2=0x7fb1e410b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e41006a0 0x7fb1e410b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 -- 172.21.15.45:0/1172716130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e4102250 msgr2=0x7fb1e4109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e4102250 0x7fb1e4109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 -- 172.21.15.45:0/1172716130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e410a430 con 0x7fb1e4101890 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1e91b0640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d950 tx=0x7fb1d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8017030 con 0x7fb1e4101890 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8004020 con 0x7fb1e4101890 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e41a1bd0 con 0x7fb1e4101890 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.683+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80047a0 con 0x7fb1e4101890 2024-04-25T00:18:53.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.684+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e41a2120 con 0x7fb1e4101890 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.685+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e4107960 con 0x7fb1e4101890 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.686+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb1d80041d0 con 0x7fb1e4101890 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.686+0000 7fb1d6ffd640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.686+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb1d80cebc0 con 0x7fb1e4101890 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.686+0000 7fb1e99b1640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.687+0000 7fb1e99b1640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 0x7fb1d00a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1e41016f0 tx=0x7fb1e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:53.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.690+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1d809bfc0 con 0x7fb1e4101890 2024-04-25T00:18:53.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:53.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-25T00:18:53.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:53.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:53.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:53.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:53.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:53 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.875+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1e402d340 con 0x7fb1e4101890 2024-04-25T00:18:53.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.876+0000 7fb1d6ffd640 1 -- 172.21.15.45:0/1172716130 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18444 (secure 0 0 0) 0x7fb1d809b8c0 con 0x7fb1e4101890 2024-04-25T00:18:53.877 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:53.877 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:18:42.079045+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 msgr2=0x7fb1d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 0x7fb1d00a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1e41016f0 tx=0x7fb1e0005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 msgr2=0x7fb1e4109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d950 tx=0x7fb1d800de10 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 shutdown_connections 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1d00a2b90 0x7fb1d00a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1e4102250 0x7fb1e4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e4101890 0x7fb1e4109660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 --2- 172.21.15.45:0/1172716130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e41006a0 0x7fb1e410b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 >> 172.21.15.45:0/1172716130 conn(0x7fb1e40fbc30 msgr2=0x7fb1e4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 shutdown_connections 2024-04-25T00:18:53.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.878+0000 7fb1ebc15640 1 -- 172.21.15.45:0/1172716130 wait complete. 2024-04-25T00:18:53.888 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:53.888 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:53.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:53 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:53.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:53.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:53.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- 172.21.15.45:0/1454731543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 msgr2=0x7f77dc108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/1454731543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc108ac0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009ca0 tx=0x7f77d402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- 172.21.15.45:0/1454731543 shutdown_connections 2024-04-25T00:18:53.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/1454731543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc108ac0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/1454731543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/1454731543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 0x7f77dc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- 172.21.15.45:0/1454731543 >> 172.21.15.45:0/1454731543 conn(0x7f77dc0fbbf0 msgr2=0x7f77dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- 172.21.15.45:0/1454731543 shutdown_connections 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- 172.21.15.45:0/1454731543 wait complete. 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 Processor -- start 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.995+0000 7f77e0aa5640 1 -- start start 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 0x7f77dc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc10c2e0 con 0x7f77dc102230 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc10c160 con 0x7f77dc101870 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77e0aa5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc10c460 con 0x7f77dc100680 2024-04-25T00:18:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77dad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77dad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44426/0 (socket says 172.21.15.45:44426) 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 0x7f77dc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77dad9d640 1 -- 172.21.15.45:0/4271295299 learned_addr learned my addr 172.21.15.45:0/4271295299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 -- 172.21.15.45:0/4271295299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 msgr2=0x7f77dc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 0x7f77dc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 -- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 msgr2=0x7f77dc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 -- 172.21.15.45:0/4271295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc19f660 con 0x7f77dc101870 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77dad9d640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77d9d9b640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc197f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77d000d6d0 tx=0x7f77d000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.996+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0017070 con 0x7f77dc101870 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.997+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0009820 con 0x7f77dc101870 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.997+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc19f950 con 0x7f77dc101870 2024-04-25T00:18:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.997+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0004d00 con 0x7f77dc101870 2024-04-25T00:18:54.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.997+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc19fea0 con 0x7f77dc101870 2024-04-25T00:18:54.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.998+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc1a0130 con 0x7f77dc101870 2024-04-25T00:18:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.999+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f77d00099d0 con 0x7f77dc101870 2024-04-25T00:18:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.999+0000 7f77cb7fe640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 0x7f77c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:53.999+0000 7f77da59c640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 0x7f77c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.000+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f77d00ce700 con 0x7f77dc101870 2024-04-25T00:18:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.000+0000 7f77da59c640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 0x7f77c40a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77dc1016d0 tx=0x7f77cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.003+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77d009b410 con 0x7f77dc101870 2024-04-25T00:18:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.191+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77dc1a0300 con 0x7f77dc101870 2024-04-25T00:18:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.191+0000 7f77cb7fe640 1 -- 172.21.15.45:0/4271295299 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f77d009f710 con 0x7f77dc101870 2024-04-25T00:18:54.192 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 msgr2=0x7f77c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 0x7f77c40a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77dc1016d0 tx=0x7f77cc00b390 comp rx=0 tx=0).stop 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 msgr2=0x7f77dc197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc197f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77d000d6d0 tx=0x7f77d000db90 comp rx=0 tx=0).stop 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 shutdown_connections 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f77c40a2b90 0x7f77c40a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77dc102230 0x7f77dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77dc101870 0x7f77dc197f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 --2- 172.21.15.45:0/4271295299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77dc100680 0x7f77dc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 >> 172.21.15.45:0/4271295299 conn(0x7f77dc0fbbf0 msgr2=0x7f77dc0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 shutdown_connections 2024-04-25T00:18:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.193+0000 7f77e0aa5640 1 -- 172.21.15.45:0/4271295299 wait complete. 2024-04-25T00:18:54.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 -- 172.21.15.45:0/3709886745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc401006a0 msgr2=0x7fcc40100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 --2- 172.21.15.45:0/3709886745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc401006a0 0x7fcc40100ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009900 tx=0x7fcc3802a410 comp rx=0 tx=0).stop 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 -- 172.21.15.45:0/3709886745 shutdown_connections 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 --2- 172.21.15.45:0/3709886745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc40102250 0x7fcc40108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 --2- 172.21.15.45:0/3709886745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40101890 0x7fcc40101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 --2- 172.21.15.45:0/3709886745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc401006a0 0x7fcc40100ac0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 -- 172.21.15.45:0/3709886745 >> 172.21.15.45:0/3709886745 conn(0x7fcc400fbc30 msgr2=0x7fcc400fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 -- 172.21.15.45:0/3709886745 shutdown_connections 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 -- 172.21.15.45:0/3709886745 wait complete. 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.306+0000 7fcc487a5640 1 Processor -- start 2024-04-25T00:18:54.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- start start 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc401006a0 0x7fcc40197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 0x7fcc4019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010d470 con 0x7fcc40101890 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010d2f0 con 0x7fcc40102250 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4010d5f0 con 0x7fcc401006a0 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44440/0 (socket says 172.21.15.45:44440) 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 -- 172.21.15.45:0/2042578065 learned_addr learned my addr 172.21.15.45:0/2042578065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc46541640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc401006a0 0x7fcc40197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc46d42640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 0x7fcc4019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 -- 172.21.15.45:0/2042578065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc401006a0 msgr2=0x7fcc40197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc401006a0 0x7fcc40197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 -- 172.21.15.45:0/2042578065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 msgr2=0x7fcc4019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 0x7fcc4019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 -- 172.21.15.45:0/2042578065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4019f6a0 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc46d42640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 0x7fcc4019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc45d40640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c920 tx=0x7fcc3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34007c70 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc34007e10 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4019f990 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34018890 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.307+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4019ff60 con 0x7fcc40101890 2024-04-25T00:18:54.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.308+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc401079c0 con 0x7fcc40101890 2024-04-25T00:18:54.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.309+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc34018a50 con 0x7fcc40101890 2024-04-25T00:18:54.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.310+0000 7fcc3f7fe640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:54.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.310+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcc340cefd0 con 0x7fcc40101890 2024-04-25T00:18:54.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.310+0000 7fcc46541640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:54.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.310+0000 7fcc46541640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 0x7fcc280a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc38006010 tx=0x7fcc38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:54.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.314+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc34014030 con 0x7fcc40101890 2024-04-25T00:18:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.535+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc4002d3c0 con 0x7fcc40101890 2024-04-25T00:18:54.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.535+0000 7fcc3f7fe640 1 -- 172.21.15.45:0/2042578065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fcc3409c460 con 0x7fcc40101890 2024-04-25T00:18:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:54.536 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T00:18:54.537 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-04-25T00:18:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 msgr2=0x7fcc280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 0x7fcc280a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc38006010 tx=0x7fcc38000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 msgr2=0x7fcc40197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c920 tx=0x7fcc3400cde0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 shutdown_connections 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.537+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcc280a2b90 0x7fcc280a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc40102250 0x7fcc4019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc40101890 0x7fcc40197fc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 --2- 172.21.15.45:0/2042578065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc401006a0 0x7fcc40197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 >> 172.21.15.45:0/2042578065 conn(0x7fcc400fbc30 msgr2=0x7fcc40104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 shutdown_connections 2024-04-25T00:18:54.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:54.538+0000 7fcc487a5640 1 -- 172.21.15.45:0/2042578065 wait complete. 2024-04-25T00:18:54.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:54 smithi122 ceph-mon[32959]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:54.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:54 smithi122 ceph-mon[32959]: Deploying daemon mds.l on smithi122 2024-04-25T00:18:54.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:54 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1172716130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:54.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:54 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4271295299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:54.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:54 smithi071 ceph-mon[32667]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:54.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:54 smithi071 ceph-mon[32667]: Deploying daemon mds.l on smithi122 2024-04-25T00:18:54.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:54 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1172716130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:54.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:54 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4271295299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:54 smithi045 ceph-mon[29194]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:54 smithi045 ceph-mon[29194]: Deploying daemon mds.l on smithi122 2024-04-25T00:18:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:54 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1172716130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:54 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4271295299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:55.539 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 systemd[1]: Starting Ceph mds.l for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:18:55.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:55 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2042578065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:55.806 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 podman[62879]: 2024-04-25 00:18:55.533317557 +0000 UTC m=+0.018575491 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:18:55.806 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 podman[62879]: 2024-04-25 00:18:55.666534037 +0000 UTC m=+0.151792000 container create 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:18:55.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:55 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2042578065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:55.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:55 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2042578065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 podman[62879]: 2024-04-25 00:18:55.936672926 +0000 UTC m=+0.421930847 container init 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 podman[62879]: 2024-04-25 00:18:55.939357671 +0000 UTC m=+0.424615593 container start 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 ceph-mds[62898]: -- 172.21.15.122:0/2908335709 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603e6aa7a40 con 0x5603e6a7b000 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: starting mds.l at 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:56 smithi122 bash[62879]: 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b 2024-04-25T00:18:56.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:18:56 smithi122 systemd[1]: Started Ceph mds.l for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:18:56.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:56.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 -- 172.21.15.45:0/2097338289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 msgr2=0x7fcee4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2097338289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee4108ac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009ca0 tx=0x7fcedc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 -- 172.21.15.45:0/2097338289 shutdown_connections 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2097338289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee4108ac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2097338289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2097338289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 -- 172.21.15.45:0/2097338289 >> 172.21.15.45:0/2097338289 conn(0x7fcee40fbbf0 msgr2=0x7fcee40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 -- 172.21.15.45:0/2097338289 shutdown_connections 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.664+0000 7fceeb14f640 1 -- 172.21.15.45:0/2097338289 wait complete. 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 Processor -- start 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 -- start start 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410c300 con 0x7fcee4102230 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410c180 con 0x7fcee4100680 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fceeb14f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410c480 con 0x7fcee4101870 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee96ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee96ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44452/0 (socket says 172.21.15.45:44452) 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee96ec640 1 -- 172.21.15.45:0/2888384298 learned_addr learned my addr 172.21.15.45:0/2888384298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee3fff640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 -- 172.21.15.45:0/2888384298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 msgr2=0x7fcee419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 -- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 msgr2=0x7fcee41a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee8eeb640 1 -- 172.21.15.45:0/2888384298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee41a18d0 con 0x7fcee4100680 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee3fff640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.665+0000 7fcee96ec640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fcee8eeb640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4199c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fced400e9c0 tx=0x7fced400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced400ccc0 con 0x7fcee4100680 2024-04-25T00:18:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee41a1bc0 con 0x7fcee4100680 2024-04-25T00:18:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced40035f0 con 0x7fcee4100680 2024-04-25T00:18:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced400ce60 con 0x7fcee4100680 2024-04-25T00:18:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.666+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41a2110 con 0x7fcee4100680 2024-04-25T00:18:56.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.667+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee41a2340 con 0x7fcee4100680 2024-04-25T00:18:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.668+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fced40037a0 con 0x7fcee4100680 2024-04-25T00:18:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.669+0000 7fcee1ffb640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 0x7fcecc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.669+0000 7fcee3fff640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 0x7fcecc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.669+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fced40cf080 con 0x7fcee4100680 2024-04-25T00:18:56.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.669+0000 7fcee3fff640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 0x7fcecc0a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcee419b190 tx=0x7fced8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:56.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.673+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fced409c3e0 con 0x7fcee4100680 2024-04-25T00:18:56.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.897+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcee41a2510 con 0x7fcee4100680 2024-04-25T00:18:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.897+0000 7fcee1ffb640 1 -- 172.21.15.45:0/2888384298 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fced409bce0 con 0x7fcee4100680 2024-04-25T00:18:56.898 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:56.898 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:18:56.898 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 msgr2=0x7fcecc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 0x7fcecc0a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcee419b190 tx=0x7fced8005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 msgr2=0x7fcee4199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4199c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fced400e9c0 tx=0x7fced400ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 shutdown_connections 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcecc0a2b90 0x7fcecc0a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcee4102230 0x7fcee41a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcee4101870 0x7fcee419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 --2- 172.21.15.45:0/2888384298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee4100680 0x7fcee4199c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 >> 172.21.15.45:0/2888384298 conn(0x7fcee40fbbf0 msgr2=0x7fcee40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 shutdown_connections 2024-04-25T00:18:56.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:56.900+0000 7fceeb14f640 1 -- 172.21.15.45:0/2888384298 wait complete. 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6832/3066271880,v1:172.21.15.122:6833/3066271880] up:boot 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: fsmap 1 up:standby 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.032 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.033 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:57.033 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:57.033 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:57.033 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: Deploying daemon mds.k on smithi071 2024-04-25T00:18:57.033 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2888384298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6832/3066271880,v1:172.21.15.122:6833/3066271880] up:boot 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: fsmap 1 up:standby 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: Deploying daemon mds.k on smithi071 2024-04-25T00:18:57.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2888384298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6832/3066271880,v1:172.21.15.122:6833/3066271880] up:boot 2024-04-25T00:18:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: fsmap 1 up:standby 2024-04-25T00:18:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:18:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: Deploying daemon mds.k on smithi071 2024-04-25T00:18:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2888384298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:58.566 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 systemd[1]: Starting Ceph mds.k for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:18:58.832 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 podman[61926]: 2024-04-25 00:18:58.46868145 +0000 UTC m=+0.017379521 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:18:58.832 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 podman[61926]: 2024-04-25 00:18:58.578286143 +0000 UTC m=+0.126984204 container create 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:18:58.833 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 podman[61926]: 2024-04-25 00:18:58.806475982 +0000 UTC m=+0.355174042 container init 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-25T00:18:58.833 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 podman[61926]: 2024-04-25 00:18:58.809073836 +0000 UTC m=+0.357771900 container start 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T00:18:58.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 -- 172.21.15.45:0/530960267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c102240 msgr2=0x7efd3c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 --2- 172.21.15.45:0/530960267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c102240 0x7efd3c108ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd34009ca0 tx=0x7efd3402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 -- 172.21.15.45:0/530960267 shutdown_connections 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 --2- 172.21.15.45:0/530960267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c102240 0x7efd3c108ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 --2- 172.21.15.45:0/530960267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c101880 0x7efd3c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 --2- 172.21.15.45:0/530960267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c100690 0x7efd3c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 -- 172.21.15.45:0/530960267 >> 172.21.15.45:0/530960267 conn(0x7efd3c0fbc20 msgr2=0x7efd3c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 -- 172.21.15.45:0/530960267 shutdown_connections 2024-04-25T00:18:59.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.031+0000 7efd42f7e640 1 -- 172.21.15.45:0/530960267 wait complete. 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 Processor -- start 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 -- start start 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 0x7efd3c06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 0x7efd3c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10c310 con 0x7efd3c101880 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10c190 con 0x7efd3c102240 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd42f7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10c490 con 0x7efd3c100690 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44478/0 (socket says 172.21.15.45:44478) 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 -- 172.21.15.45:0/1710313274 learned_addr learned my addr 172.21.15.45:0/1710313274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd4151b640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 0x7efd3c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd40d1a640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 0x7efd3c06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 -- 172.21.15.45:0/1710313274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 msgr2=0x7efd3c06a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 0x7efd3c06a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 -- 172.21.15.45:0/1710313274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 msgr2=0x7efd3c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 0x7efd3c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd3bfff640 1 -- 172.21.15.45:0/1710313274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1a18c0 con 0x7efd3c101880 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd4151b640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 0x7efd3c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.032+0000 7efd40d1a640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 0x7efd3c06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd3bfff640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d770 tx=0x7efd2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28017070 con 0x7efd3c101880 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd28005500 con 0x7efd3c101880 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c1a1b50 con 0x7efd3c101880 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28004e00 con 0x7efd3c101880 2024-04-25T00:18:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.033+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c1a20a0 con 0x7efd3c101880 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.034+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c1079b0 con 0x7efd3c101880 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.035+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd28004fa0 con 0x7efd3c101880 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.036+0000 7efd39ffb640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 0x7efd2c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.036+0000 7efd40d1a640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 0x7efd2c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.036+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efd280ce600 con 0x7efd3c101880 2024-04-25T00:18:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.036+0000 7efd40d1a640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 0x7efd2c0a5100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd3c1016e0 tx=0x7efd30005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:59.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.041+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd2809ba00 con 0x7efd3c101880 2024-04-25T00:18:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:58 smithi071 ceph-mon[32667]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:58 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:58 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:59.119 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 ceph-mds[61945]: -- 172.21.15.71:0/87108638 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608ff0cba40 con 0x5608ff09f000 2024-04-25T00:18:59.119 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: starting mds.k at 2024-04-25T00:18:59.119 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 bash[61926]: 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 2024-04-25T00:18:59.119 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:18:58 smithi071 systemd[1]: Started Ceph mds.k for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:18:59.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:58 smithi045 ceph-mon[29194]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:59.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:58 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:59.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:58 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.267+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd3c02d3b0 con 0x7efd3c101880 2024-04-25T00:18:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.268+0000 7efd39ffb640 1 -- 172.21.15.45:0/1710313274 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7efd2809b300 con 0x7efd3c101880 2024-04-25T00:18:59.268 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:18:59.269 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:18:59.269 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 msgr2=0x7efd2c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 0x7efd2c0a5100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd3c1016e0 tx=0x7efd30005e70 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 msgr2=0x7efd3c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d770 tx=0x7efd2800dc30 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 shutdown_connections 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efd2c0a2c40 0x7efd2c0a5100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c102240 0x7efd3c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd3c101880 0x7efd3c19eb80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 --2- 172.21.15.45:0/1710313274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c100690 0x7efd3c06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 >> 172.21.15.45:0/1710313274 conn(0x7efd3c0fbc20 msgr2=0x7efd3c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 shutdown_connections 2024-04-25T00:18:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:18:59.270+0000 7efd42f7e640 1 -- 172.21.15.45:0/1710313274 wait complete. 2024-04-25T00:18:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:58 smithi122 ceph-mon[32959]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:18:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:58 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:18:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:58 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6832/1511686553,v1:172.21.15.71:6833/1511686553] up:boot 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: fsmap 2 up:standby 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:00.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: Deploying daemon mds.j on smithi045 2024-04-25T00:19:00.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:18:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1710313274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6832/1511686553,v1:172.21.15.71:6833/1511686553] up:boot 2024-04-25T00:19:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: fsmap 2 up:standby 2024-04-25T00:19:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:19:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: Deploying daemon mds.j on smithi045 2024-04-25T00:19:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:18:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1710313274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6832/1511686553,v1:172.21.15.71:6833/1511686553] up:boot 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: fsmap 2 up:standby 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: Deploying daemon mds.j on smithi045 2024-04-25T00:19:00.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:18:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1710313274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:01 smithi045 ceph-mon[29194]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:01.233 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:01 smithi071 ceph-mon[32667]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:01.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:01.305 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:01 smithi122 ceph-mon[32959]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- 172.21.15.45:0/3079294017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc8103360 msgr2=0x7f6cc8109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/3079294017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc8103360 0x7f6cc8109bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009ca0 tx=0x7f6cc002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- 172.21.15.45:0/3079294017 shutdown_connections 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/3079294017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc8103360 0x7f6cc8109bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/3079294017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc81029a0 0x7f6cc8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/3079294017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81017b0 0x7f6cc8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- 172.21.15.45:0/3079294017 >> 172.21.15.45:0/3079294017 conn(0x7f6cc80fcd40 msgr2=0x7f6cc80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- 172.21.15.45:0/3079294017 shutdown_connections 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- 172.21.15.45:0/3079294017 wait complete. 2024-04-25T00:19:01.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 Processor -- start 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.397+0000 7f6ccfae1640 1 -- start start 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81029a0 0x7f6cc8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc8103360 0x7f6cc819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc810d430 con 0x7f6cc81017b0 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc810d2b0 con 0x7f6cc81029a0 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc810d5b0 con 0x7f6cc8103360 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44484/0 (socket says 172.21.15.45:44484) 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 -- 172.21.15.45:0/724373038 learned_addr learned my addr 172.21.15.45:0/724373038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd07c640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81029a0 0x7f6cc8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6cce07e640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc8103360 0x7f6cc819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 -- 172.21.15.45:0/724373038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc8103360 msgr2=0x7f6cc819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc8103360 0x7f6cc819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 -- 172.21.15.45:0/724373038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81029a0 msgr2=0x7f6cc8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:01.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81029a0 0x7f6cc8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 -- 172.21.15.45:0/724373038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc819f6c0 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccd87d640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e9c0 tx=0x7f6cc400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400cd80 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc819f9b0 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc40035f0 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4010690 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.398+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc819ff50 con 0x7f6cc81017b0 2024-04-25T00:19:01.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.399+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc8108ba0 con 0x7f6cc81017b0 2024-04-25T00:19:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.401+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc40037a0 con 0x7f6cc81017b0 2024-04-25T00:19:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.401+0000 7f6cbeffd640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 0x7f6cb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.401+0000 7f6ccd07c640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 0x7f6cb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.401+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6cc40cf020 con 0x7f6cc81017b0 2024-04-25T00:19:01.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.402+0000 7f6ccd07c640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 0x7f6cb40a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8198fb0 tx=0x7f6cb800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:01.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.406+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6cc409c420 con 0x7f6cc81017b0 2024-04-25T00:19:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.638+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cc8062340 con 0x7f6cc81017b0 2024-04-25T00:19:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.639+0000 7f6cbeffd640 1 -- 172.21.15.45:0/724373038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f6cc409bd20 con 0x7f6cc81017b0 2024-04-25T00:19:01.640 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:01.641 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:19:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-25T00:19:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 msgr2=0x7f6cb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 0x7f6cb40a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8198fb0 tx=0x7f6cb800b390 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 msgr2=0x7f6cc8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e9c0 tx=0x7f6cc400ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 shutdown_connections 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6cb40a2b90 0x7f6cb40a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cc8103360 0x7f6cc819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc81029a0 0x7f6cc8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 --2- 172.21.15.45:0/724373038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cc81017b0 0x7f6cc8197aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 >> 172.21.15.45:0/724373038 conn(0x7f6cc80fcd40 msgr2=0x7f6cc8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 shutdown_connections 2024-04-25T00:19:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:01.641+0000 7f6ccfae1640 1 -- 172.21.15.45:0/724373038 wait complete. 2024-04-25T00:19:01.899 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:19:01 smithi045 systemd[1]: Started Ceph mds.j for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/724373038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:02 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:02.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/724373038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:02 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/724373038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:02.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:02 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:03 smithi045 ceph-mon[29194]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:03 smithi045 ceph-mon[29194]: Deploying daemon mds.d on smithi045 2024-04-25T00:19:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:03 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6834/3465528843,v1:172.21.15.45:6835/3465528843] up:boot 2024-04-25T00:19:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:03 smithi045 ceph-mon[29194]: fsmap 3 up:standby 2024-04-25T00:19:03.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:03 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:19:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:03 smithi122 ceph-mon[32959]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:03 smithi122 ceph-mon[32959]: Deploying daemon mds.d on smithi045 2024-04-25T00:19:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:03 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6834/3465528843,v1:172.21.15.45:6835/3465528843] up:boot 2024-04-25T00:19:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:03 smithi122 ceph-mon[32959]: fsmap 3 up:standby 2024-04-25T00:19:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:03 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:19:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:03 smithi071 ceph-mon[32667]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:03 smithi071 ceph-mon[32667]: Deploying daemon mds.d on smithi045 2024-04-25T00:19:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:03 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6834/3465528843,v1:172.21.15.45:6835/3465528843] up:boot 2024-04-25T00:19:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:03 smithi071 ceph-mon[32667]: fsmap 3 up:standby 2024-04-25T00:19:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:03 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:19:03.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:03.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 -- 172.21.15.45:0/184065202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc10b0b0 msgr2=0x7f12cc10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:03.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 --2- 172.21.15.45:0/184065202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc10b0b0 0x7f12cc10d540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009ca0 tx=0x7f12c402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 -- 172.21.15.45:0/184065202 shutdown_connections 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 --2- 172.21.15.45:0/184065202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10da80 0x7f12cc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 --2- 172.21.15.45:0/184065202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc10b0b0 0x7f12cc10d540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 --2- 172.21.15.45:0/184065202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc108740 0x7f12cc10ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.780+0000 7f12d3031640 1 -- 172.21.15.45:0/184065202 >> 172.21.15.45:0/184065202 conn(0x7f12cc06e5e0 msgr2=0x7f12cc070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- 172.21.15.45:0/184065202 shutdown_connections 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- 172.21.15.45:0/184065202 wait complete. 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 Processor -- start 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- start start 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 0x7f12cc1a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc10da80 0x7f12cc1adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc11c1f0 con 0x7f12cc108740 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc11c070 con 0x7f12cc10b0b0 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d3031640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc11c370 con 0x7f12cc10da80 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d0dcd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.781+0000 7f12d0dcd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39058/0 (socket says 172.21.15.45:39058) 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 -- 172.21.15.45:0/1612530939 learned_addr learned my addr 172.21.15.45:0/1612530939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d15ce640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc10da80 0x7f12cc1adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12cbfff640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 0x7f12cc1a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 -- 172.21.15.45:0/1612530939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc10da80 msgr2=0x7f12cc1adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc10da80 0x7f12cc1adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 -- 172.21.15.45:0/1612530939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 msgr2=0x7f12cc1a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 0x7f12cc1a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 -- 172.21.15.45:0/1612530939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1ae510 con 0x7f12cc108740 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12cbfff640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 0x7f12cc1a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d0dcd640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00e9f0 tx=0x7f12bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00cd80 con 0x7f12cc108740 2024-04-25T00:19:03.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc010910 con 0x7f12cc108740 2024-04-25T00:19:03.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1ae800 con 0x7f12cc108740 2024-04-25T00:19:03.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc018990 con 0x7f12cc108740 2024-04-25T00:19:03.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.782+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1aed70 con 0x7f12cc108740 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.784+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc068560 con 0x7f12cc108740 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.785+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12bc010ac0 con 0x7f12cc108740 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.785+0000 7f12c9ffb640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 0x7f12b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.785+0000 7f12cbfff640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 0x7f12b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.785+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f12bc0ceec0 con 0x7f12cc108740 2024-04-25T00:19:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.786+0000 7f12cbfff640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 0x7f12b80a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006010 tx=0x7f12c4031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:03.791+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f12bc09c2c0 con 0x7f12cc108740 2024-04-25T00:19:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.041+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12cc0710e0 con 0x7f12cc108740 2024-04-25T00:19:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.041+0000 7f12c9ffb640 1 -- 172.21.15.45:0/1612530939 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f12cc0710e0 con 0x7f12cc108740 2024-04-25T00:19:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:04.043 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:19:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 msgr2=0x7f12b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 0x7f12b80a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006010 tx=0x7f12c4031bb0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 msgr2=0x7f12cc1a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00e9f0 tx=0x7f12bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 shutdown_connections 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f12b80a2c10 0x7f12b80a50d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12cc10da80 0x7f12cc1adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12cc10b0b0 0x7f12cc1a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 --2- 172.21.15.45:0/1612530939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12cc108740 0x7f12cc1a68f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 >> 172.21.15.45:0/1612530939 conn(0x7f12cc06e5e0 msgr2=0x7f12cc108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 shutdown_connections 2024-04-25T00:19:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:04.043+0000 7f12d3031640 1 -- 172.21.15.45:0/1612530939 wait complete. 2024-04-25T00:19:04.366 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 systemd[1]: Starting Ceph mds.d for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:04.626 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 podman[67951]: 2024-04-25 00:19:04.269437156 +0000 UTC m=+0.018775032 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:04.626 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 podman[67951]: 2024-04-25 00:19:04.389941327 +0000 UTC m=+0.139279236 container create 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T00:19:04.627 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 podman[67951]: 2024-04-25 00:19:04.626566765 +0000 UTC m=+0.375904632 container init 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:19:04.899 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 podman[67951]: 2024-04-25 00:19:04.629255479 +0000 UTC m=+0.378593354 container start 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:19:04.899 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 ceph-mds[67970]: -- 172.21.15.45:0/487500846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561bc799a40 con 0x5561bc76d000 2024-04-25T00:19:04.899 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d[67966]: starting mds.d at 2024-04-25T00:19:04.899 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 bash[67951]: 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 2024-04-25T00:19:04.900 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:19:04 smithi045 systemd[1]: Started Ceph mds.d for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1612530939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:05.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:05 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1612530939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:05 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1612530939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:05 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:06.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/2511999706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc103360 msgr2=0x7f4bcc109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/2511999706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc103360 0x7f4bcc109bf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009ca0 tx=0x7f4bc402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/2511999706 shutdown_connections 2024-04-25T00:19:06.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/2511999706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc103360 0x7f4bcc109bf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/2511999706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 0x7f4bcc102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/2511999706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/2511999706 >> 172.21.15.45:0/2511999706 conn(0x7f4bcc0fcd40 msgr2=0x7f4bcc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/2511999706 shutdown_connections 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.174+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/2511999706 wait complete. 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 Processor -- start 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 -- start start 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 0x7f4bcc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc103360 0x7f4bcc1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc10d410 con 0x7f4bcc1017b0 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc10d290 con 0x7f4bcc103360 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bd1eb3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc10d590 con 0x7f4bcc1029a0 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39108/0 (socket says 172.21.15.45:39108) 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bcb7fe640 1 -- 172.21.15.45:0/1619831298 learned_addr learned my addr 172.21.15.45:0/1619831298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bcbfff640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc103360 0x7f4bcc1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.175+0000 7f4bcaffd640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 0x7f4bcc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 -- 172.21.15.45:0/1619831298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 msgr2=0x7f4bcc19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 0x7f4bcc19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 -- 172.21.15.45:0/1619831298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc103360 msgr2=0x7f4bcc1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc103360 0x7f4bcc1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 -- 172.21.15.45:0/1619831298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc1a17e0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bcb7fe640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c970 tx=0x7f4bbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004220 con 0x7f4bcc1017b0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc0043c0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc1a1ad0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00f450 con 0x7f4bcc1017b0 2024-04-25T00:19:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.176+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a2020 con 0x7f4bcc1017b0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.177+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bcc1a21f0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.178+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4bbc0079b0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.179+0000 7f4bc8ff9640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 0x7f4bb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.179+0000 7f4bcaffd640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 0x7f4bb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.179+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4bbc0ce9c0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.179+0000 7f4bcaffd640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 0x7f4bb40a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005e10 tx=0x7f4bc0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:06.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.183+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4bbc014030 con 0x7f4bcc1017b0 2024-04-25T00:19:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:06 smithi122 ceph-mon[32959]: Deploying daemon mds.e on smithi071 2024-04-25T00:19:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6836/2745663812,v1:172.21.15.45:6837/2745663812] up:boot 2024-04-25T00:19:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:06 smithi122 ceph-mon[32959]: fsmap 4 up:standby 2024-04-25T00:19:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:06 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:19:06.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:06 smithi071 ceph-mon[32667]: Deploying daemon mds.e on smithi071 2024-04-25T00:19:06.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6836/2745663812,v1:172.21.15.45:6837/2745663812] up:boot 2024-04-25T00:19:06.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:06 smithi071 ceph-mon[32667]: fsmap 4 up:standby 2024-04-25T00:19:06.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:06 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:19:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:06 smithi045 ceph-mon[29194]: Deploying daemon mds.e on smithi071 2024-04-25T00:19:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6836/2745663812,v1:172.21.15.45:6837/2745663812] up:boot 2024-04-25T00:19:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:06 smithi045 ceph-mon[29194]: fsmap 4 up:standby 2024-04-25T00:19:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:06 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:19:06.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.413+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bcc1a23c0 con 0x7f4bcc1017b0 2024-04-25T00:19:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.413+0000 7f4bc8ff9640 1 -- 172.21.15.45:0/1619831298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f4bbc09bd20 con 0x7f4bcc1017b0 2024-04-25T00:19:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:06.414 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:19:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-04-25T00:19:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 msgr2=0x7f4bb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 0x7f4bb40a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005e10 tx=0x7f4bc0005da0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 msgr2=0x7f4bcc199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c970 tx=0x7f4bbc00ce30 comp rx=0 tx=0).stop 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 shutdown_connections 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4bb40a2b90 0x7f4bb40a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bcc103360 0x7f4bcc1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bcc1029a0 0x7f4bcc19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 --2- 172.21.15.45:0/1619831298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bcc1017b0 0x7f4bcc199bc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.415+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 >> 172.21.15.45:0/1619831298 conn(0x7f4bcc0fcd40 msgr2=0x7f4bcc073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.416+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 shutdown_connections 2024-04-25T00:19:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:06.416+0000 7f4bd1eb3640 1 -- 172.21.15.45:0/1619831298 wait complete. 2024-04-25T00:19:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:07 smithi122 ceph-mon[32959]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:07 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1619831298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:07 smithi071 ceph-mon[32667]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:07 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1619831298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.369 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 systemd[1]: Starting Ceph mds.e for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:07 smithi045 ceph-mon[29194]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:07 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1619831298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:07.663 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 podman[62248]: 2024-04-25 00:19:07.304135522 +0000 UTC m=+0.017843363 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:07.663 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 podman[62248]: 2024-04-25 00:19:07.418993269 +0000 UTC m=+0.132701138 container create 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 podman[62248]: 2024-04-25 00:19:07.663736298 +0000 UTC m=+0.377444125 container init 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 podman[62248]: 2024-04-25 00:19:07.666219429 +0000 UTC m=+0.379927255 container start 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 ceph-mds[62267]: -- 172.21.15.71:0/334781398 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562498003a40 con 0x562497fd6800 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: starting mds.e at 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 bash[62248]: 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e 2024-04-25T00:19:08.119 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:19:07 smithi071 systemd[1]: Started Ceph mds.e for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:08.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- 172.21.15.45:0/3769417391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 msgr2=0x7f9034109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 --2- 172.21.15.45:0/3769417391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f9034109910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f902c009ca0 tx=0x7f902c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- 172.21.15.45:0/3769417391 shutdown_connections 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 --2- 172.21.15.45:0/3769417391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f9034109910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 --2- 172.21.15.45:0/3769417391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 0x7f9034106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 --2- 172.21.15.45:0/3769417391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 0x7f90341046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- 172.21.15.45:0/3769417391 >> 172.21.15.45:0/3769417391 conn(0x7f90340fbbf0 msgr2=0x7f90340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- 172.21.15.45:0/3769417391 shutdown_connections 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- 172.21.15.45:0/3769417391 wait complete. 2024-04-25T00:19:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 Processor -- start 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.541+0000 7f9039f16640 1 -- start start 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 0x7f9034197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 0x7f9034197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f903410d410 con 0x7f9034107520 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f903410d290 con 0x7f9034104bf0 2024-04-25T00:19:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f903410d590 con 0x7f90341022c0 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39120/0 (socket says 172.21.15.45:39120) 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 -- 172.21.15.45:0/3131363562 learned_addr learned my addr 172.21.15.45:0/3131363562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f90337fe640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 0x7f9034197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9032ffd640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 0x7f9034197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 -- 172.21.15.45:0/3131363562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 msgr2=0x7f9034197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 0x7f9034197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 -- 172.21.15.45:0/3131363562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 msgr2=0x7f9034197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 0x7f9034197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 -- 172.21.15.45:0/3131363562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903419f680 con 0x7f9034107520 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9033fff640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f902c005e30 tx=0x7f902c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c003c30 con 0x7f9034107520 2024-04-25T00:19:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902c003dd0 con 0x7f9034107520 2024-04-25T00:19:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903419f910 con 0x7f9034107520 2024-04-25T00:19:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.542+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c02fde0 con 0x7f9034107520 2024-04-25T00:19:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.543+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903419fe60 con 0x7f9034107520 2024-04-25T00:19:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.544+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90340fd6e0 con 0x7f9034107520 2024-04-25T00:19:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.545+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f902c004050 con 0x7f9034107520 2024-04-25T00:19:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.545+0000 7f9030ff9640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 0x7f901c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.545+0000 7f90337fe640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 0x7f901c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.545+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f902c0f01e0 con 0x7f9034107520 2024-04-25T00:19:08.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.546+0000 7f90337fe640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 0x7f901c0a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9024005d00 tx=0x7f902400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.549+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f902c0bd5e0 con 0x7f9034107520 2024-04-25T00:19:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.773+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f903402d330 con 0x7f9034107520 2024-04-25T00:19:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.774+0000 7f9030ff9640 1 -- 172.21.15.45:0/3131363562 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f903402d330 con 0x7f9034107520 2024-04-25T00:19:08.775 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:08.775 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T00:19:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 msgr2=0x7f901c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 0x7f901c0a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9024005d00 tx=0x7f902400b410 comp rx=0 tx=0).stop 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 msgr2=0x7f903419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f902c005e30 tx=0x7f902c031ee0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 shutdown_connections 2024-04-25T00:19:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f901c0a2b90 0x7f901c0a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9034107520 0x7f903419f010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9034104bf0 0x7f9034197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 --2- 172.21.15.45:0/3131363562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90341022c0 0x7f9034197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 >> 172.21.15.45:0/3131363562 conn(0x7f90340fbbf0 msgr2=0x7f9034102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 shutdown_connections 2024-04-25T00:19:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:08.776+0000 7f9039f16640 1 -- 172.21.15.45:0/3131363562 wait complete. 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:09.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: Deploying daemon mds.f on smithi122 2024-04-25T00:19:09.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6834/741071549,v1:172.21.15.71:6835/741071549] up:boot 2024-04-25T00:19:09.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: fsmap 5 up:standby 2024-04-25T00:19:09.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:09.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:08 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3131363562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: Deploying daemon mds.f on smithi122 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6834/741071549,v1:172.21.15.71:6835/741071549] up:boot 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: fsmap 5 up:standby 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:08 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3131363562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: Deploying daemon mds.f on smithi122 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6834/741071549,v1:172.21.15.71:6835/741071549] up:boot 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: fsmap 5 up:standby 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:19:09.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:08 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3131363562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:10.056 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:09 smithi122 systemd[1]: Starting Ceph mds.f for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:10.411 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 podman[63195]: 2024-04-25 00:19:10.039297024 +0000 UTC m=+0.018169412 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:10.413 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 podman[63195]: 2024-04-25 00:19:10.157127726 +0000 UTC m=+0.136000096 container create e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:19:10.413 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 podman[63195]: 2024-04-25 00:19:10.410525717 +0000 UTC m=+0.389398090 container init e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:19:10.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:10.802 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 podman[63195]: 2024-04-25 00:19:10.413704318 +0000 UTC m=+0.392576689 container start e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:10.802 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mds[63215]: -- 172.21.15.122:0/2494315947 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557de94e9a40 con 0x557de94bd000 2024-04-25T00:19:10.802 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: starting mds.f at 2024-04-25T00:19:10.802 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 bash[63195]: e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc 2024-04-25T00:19:10.802 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:19:10 smithi122 systemd[1]: Started Ceph mds.f for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 -- 172.21.15.45:0/277199458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0102250 msgr2=0x7f38a0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 --2- 172.21.15.45:0/277199458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0102250 0x7f38a0108ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3898009ca0 tx=0x7f389802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 -- 172.21.15.45:0/277199458 shutdown_connections 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 --2- 172.21.15.45:0/277199458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0102250 0x7f38a0108ae0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 --2- 172.21.15.45:0/277199458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a0101890 0x7f38a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 --2- 172.21.15.45:0/277199458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a01006a0 0x7f38a0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 -- 172.21.15.45:0/277199458 >> 172.21.15.45:0/277199458 conn(0x7f38a00fbc30 msgr2=0x7f38a00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:10.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 -- 172.21.15.45:0/277199458 shutdown_connections 2024-04-25T00:19:10.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.898+0000 7f38a4a38640 1 -- 172.21.15.45:0/277199458 wait complete. 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:10.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:10 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 Processor -- start 2024-04-25T00:19:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 -- start start 2024-04-25T00:19:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 0x7f38a0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:10.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0102250 0x7f38a019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010c320 con 0x7f38a0101890 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010c1a0 con 0x7f38a0102250 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f38a4a38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010c4a0 con 0x7f38a01006a0 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f3897fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f3897fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39148/0 (socket says 172.21.15.45:39148) 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f3897fff640 1 -- 172.21.15.45:0/4190834514 learned_addr learned my addr 172.21.15.45:0/4190834514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.899+0000 7f389e59c640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 0x7f38a0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f389ed9d640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0102250 0x7f38a019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 -- 172.21.15.45:0/4190834514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 msgr2=0x7f38a0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 0x7f38a0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 -- 172.21.15.45:0/4190834514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0102250 msgr2=0x7f38a019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:10.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0102250 0x7f38a019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 -- 172.21.15.45:0/4190834514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019f670 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f389e59c640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 0x7f38a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f3897fff640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f388800e9f0 tx=0x7f388800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388800cd80 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019f960 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3888005500 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888010690 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.900+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a019feb0 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.901+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a01a0140 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.902+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38880046c0 con 0x7f38a0101890 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.903+0000 7f38977fe640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 0x7f388c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.903+0000 7f389e59c640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 0x7f388c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:10.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.903+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38880cef80 con 0x7f38a0101890 2024-04-25T00:19:10.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.903+0000 7f389e59c640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 0x7f388c0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38a01016f0 tx=0x7f389000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:10.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:10.907+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f388809c380 con 0x7f38a0101890 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:10 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:11.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:11.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:10 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.129+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38a01a0310 con 0x7f38a0101890 2024-04-25T00:19:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.130+0000 7f38977fe640 1 -- 172.21.15.45:0/4190834514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f388809bc80 con 0x7f38a0101890 2024-04-25T00:19:11.130 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:11.130 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:19:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 msgr2=0x7f388c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 0x7f388c0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38a01016f0 tx=0x7f389000b390 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 msgr2=0x7f38a0197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f388800e9f0 tx=0x7f388800edf0 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 shutdown_connections 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f388c0a2b90 0x7f388c0a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0102250 0x7f38a019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a0101890 0x7f38a0197f90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 --2- 172.21.15.45:0/4190834514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38a01006a0 0x7f38a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 >> 172.21.15.45:0/4190834514 conn(0x7f38a00fbc30 msgr2=0x7f38a00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 shutdown_connections 2024-04-25T00:19:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:11.132+0000 7f38a4a38640 1 -- 172.21.15.45:0/4190834514 wait complete. 2024-04-25T00:19:12.069 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:11 smithi122 ceph-mon[32959]: Deploying daemon mds.c on smithi122 2024-04-25T00:19:12.069 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:11 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:boot 2024-04-25T00:19:12.069 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:11 smithi122 ceph-mon[32959]: fsmap 6 up:standby 2024-04-25T00:19:12.069 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:11 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:12.069 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4190834514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:12.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:11 smithi071 ceph-mon[32667]: Deploying daemon mds.c on smithi122 2024-04-25T00:19:12.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:11 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:boot 2024-04-25T00:19:12.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:11 smithi071 ceph-mon[32667]: fsmap 6 up:standby 2024-04-25T00:19:12.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:11 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:12.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:11 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4190834514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:12.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:11 smithi045 ceph-mon[29194]: Deploying daemon mds.c on smithi122 2024-04-25T00:19:12.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:11 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:boot 2024-04-25T00:19:12.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:11 smithi045 ceph-mon[29194]: fsmap 6 up:standby 2024-04-25T00:19:12.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:11 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:19:12.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:11 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4190834514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:12 smithi122 ceph-mon[32959]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:13.056 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:12 smithi122 systemd[1]: Starting Ceph mds.c for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:13.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:12 smithi071 ceph-mon[32667]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:13.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:13.148 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:12 smithi045 ceph-mon[29194]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 -- 172.21.15.45:0/2315541072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 msgr2=0x7f587c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 --2- 172.21.15.45:0/2315541072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c100ac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5864009900 tx=0x7f586402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 -- 172.21.15.45:0/2315541072 shutdown_connections 2024-04-25T00:19:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 --2- 172.21.15.45:0/2315541072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 0x7f587c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 --2- 172.21.15.45:0/2315541072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 0x7f587c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 --2- 172.21.15.45:0/2315541072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c100ac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 -- 172.21.15.45:0/2315541072 >> 172.21.15.45:0/2315541072 conn(0x7f587c0fbc30 msgr2=0x7f587c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 -- 172.21.15.45:0/2315541072 shutdown_connections 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.258+0000 7f588266d640 1 -- 172.21.15.45:0/2315541072 wait complete. 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 Processor -- start 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 -- start start 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 0x7f587c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 0x7f587c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10c320 con 0x7f587c1006a0 2024-04-25T00:19:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10c1a0 con 0x7f587c101890 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f588266d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10c4a0 con 0x7f587c102250 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.259+0000 7f587bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39154/0 (socket says 172.21.15.45:39154) 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 -- 172.21.15.45:0/3216021405 learned_addr learned my addr 172.21.15.45:0/3216021405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f5880c0a640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 0x7f587c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587b7fe640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 0x7f587c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 -- 172.21.15.45:0/3216021405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 msgr2=0x7f587c19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 0x7f587c19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 -- 172.21.15.45:0/3216021405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 msgr2=0x7f587c196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 0x7f587c196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 -- 172.21.15.45:0/3216021405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c19e5a0 con 0x7f587c1006a0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f587bfff640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5864006010 tx=0x7f5864031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586403f030 con 0x7f587c1006a0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c19e830 con 0x7f587c1006a0 2024-04-25T00:19:13.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5864003ad0 con 0x7f587c1006a0 2024-04-25T00:19:13.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864035070 con 0x7f587c1006a0 2024-04-25T00:19:13.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.260+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c19ed20 con 0x7f587c1006a0 2024-04-25T00:19:13.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.261+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c107960 con 0x7f587c1006a0 2024-04-25T00:19:13.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.263+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5864042030 con 0x7f587c1006a0 2024-04-25T00:19:13.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.263+0000 7f58797fa640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 0x7f58680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:13.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.264+0000 7f587b7fe640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 0x7f58680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:13.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.264+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f58640ef120 con 0x7f587c1006a0 2024-04-25T00:19:13.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.264+0000 7f587b7fe640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 0x7f58680a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f586c005e10 tx=0x7f586c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:13.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.268+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58640bc520 con 0x7f587c1006a0 2024-04-25T00:19:13.359 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 podman[63501]: 2024-04-25 00:19:12.965117085 +0000 UTC m=+0.017917654 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:13.359 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 podman[63501]: 2024-04-25 00:19:13.097927826 +0000 UTC m=+0.150728360 container create 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:19:13.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.493+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f587c02d170 con 0x7f587c1006a0 2024-04-25T00:19:13.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.493+0000 7f58797fa640 1 -- 172.21.15.45:0/3216021405 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f58640bbe20 con 0x7f587c1006a0 2024-04-25T00:19:13.494 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:13.495 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:19:13.495 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-04-25T00:19:13.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.495+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 msgr2=0x7f58680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:13.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.495+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 0x7f58680a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f586c005e10 tx=0x7f586c005da0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.495+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 msgr2=0x7f587c196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.495+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5864006010 tx=0x7f5864031b60 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 shutdown_connections 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f58680a2b90 0x7f58680a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f587c102250 0x7f587c19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f587c101890 0x7f587c196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 --2- 172.21.15.45:0/3216021405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f587c1006a0 0x7f587c196980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 >> 172.21.15.45:0/3216021405 conn(0x7f587c0fbc30 msgr2=0x7f587c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 shutdown_connections 2024-04-25T00:19:13.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:13.496+0000 7f588266d640 1 -- 172.21.15.45:0/3216021405 wait complete. 2024-04-25T00:19:13.806 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 podman[63501]: 2024-04-25 00:19:13.359617699 +0000 UTC m=+0.412418233 container init 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:13.806 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 podman[63501]: 2024-04-25 00:19:13.362299782 +0000 UTC m=+0.415100325 container start 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T00:19:13.807 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 ceph-mds[63522]: -- 172.21.15.122:0/3611289251 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594fee55a40 con 0x5594fee28c00 2024-04-25T00:19:13.807 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: starting mds.c at 2024-04-25T00:19:13.807 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 bash[63501]: 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf 2024-04-25T00:19:13.807 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:19:13 smithi122 systemd[1]: Started Ceph mds.c for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3216021405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: Deploying daemon mds.h on smithi071 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:boot 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: fsmap 7 up:standby 2024-04-25T00:19:14.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:14 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:14.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:14.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3216021405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: Deploying daemon mds.h on smithi071 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:boot 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: fsmap 7 up:standby 2024-04-25T00:19:14.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:14 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3216021405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:14.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: Deploying daemon mds.h on smithi071 2024-04-25T00:19:14.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:boot 2024-04-25T00:19:14.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: fsmap 7 up:standby 2024-04-25T00:19:14.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:14 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:19:15.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 -- 172.21.15.45:0/3957610218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c1017b0 msgr2=0x7f4b4c101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 --2- 172.21.15.45:0/3957610218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c101bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009900 tx=0x7f4b3c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 -- 172.21.15.45:0/3957610218 shutdown_connections 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 --2- 172.21.15.45:0/3957610218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c103360 0x7f4b4c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 --2- 172.21.15.45:0/3957610218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1029a0 0x7f4b4c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 --2- 172.21.15.45:0/3957610218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c101bd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 -- 172.21.15.45:0/3957610218 >> 172.21.15.45:0/3957610218 conn(0x7f4b4c0fcd40 msgr2=0x7f4b4c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 -- 172.21.15.45:0/3957610218 shutdown_connections 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.624+0000 7f4b528ba640 1 -- 172.21.15.45:0/3957610218 wait complete. 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 Processor -- start 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 -- start start 2024-04-25T00:19:15.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c1029a0 0x7f4b4c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 0x7f4b4c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c10d430 con 0x7f4b4c103360 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c10d2b0 con 0x7f4b4c1017b0 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b528ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c10d5b0 con 0x7f4b4c1029a0 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b50e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 0x7f4b4c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:34628/0 (socket says 172.21.15.45:34628) 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b50e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 0x7f4b4c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40930/0 (socket says 172.21.15.45:40930) 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.625+0000 7f4b4bfff640 1 -- 172.21.15.45:0/2587344002 learned_addr learned my addr 172.21.15.45:0/2587344002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4b7fe640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c1029a0 0x7f4b4c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 -- 172.21.15.45:0/2587344002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c1029a0 msgr2=0x7f4b4c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c1029a0 0x7f4b4c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 -- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 msgr2=0x7f4b4c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 0x7f4b4c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 -- 172.21.15.45:0/2587344002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c19f6f0 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b4bfff640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c031610 tx=0x7f4b3c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c03f030 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c19f980 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c003ad0 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c035070 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.626+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c19fe70 con 0x7f4b4c1017b0 2024-04-25T00:19:15.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.627+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c108ad0 con 0x7f4b4c1017b0 2024-04-25T00:19:15.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.628+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b3c042030 con 0x7f4b4c1017b0 2024-04-25T00:19:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.629+0000 7f4b497fa640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.629+0000 7f4b4b7fe640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.629+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4b3c0ef530 con 0x7f4b4c1017b0 2024-04-25T00:19:15.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.629+0000 7f4b4b7fe640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 0x7f4b340a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c198fe0 tx=0x7f4b40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.633+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b3c0bc930 con 0x7f4b4c1017b0 2024-04-25T00:19:15.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.856+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b4c02d170 con 0x7f4b4c1017b0 2024-04-25T00:19:15.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.857+0000 7f4b497fa640 1 -- 172.21.15.45:0/2587344002 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f4b3c0bc230 con 0x7f4b4c1017b0 2024-04-25T00:19:15.857 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:15.857 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:19:15.858 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 msgr2=0x7f4b340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 0x7f4b340a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c198fe0 tx=0x7f4b40005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 msgr2=0x7f4b4c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c031610 tx=0x7f4b3c004270 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 shutdown_connections 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4b340a2b90 0x7f4b340a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b4c103360 0x7f4b4c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c1029a0 0x7f4b4c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 --2- 172.21.15.45:0/2587344002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b4c1017b0 0x7f4b4c197ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 >> 172.21.15.45:0/2587344002 conn(0x7f4b4c0fcd40 msgr2=0x7f4b4c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 shutdown_connections 2024-04-25T00:19:15.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:15.859+0000 7f4b528ba640 1 -- 172.21.15.45:0/2587344002 wait complete. 2024-04-25T00:19:15.868 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:15 smithi071 systemd[1]: Starting Ceph mds.h for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:16.204 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:15 smithi071 podman[62552]: 2024-04-25 00:19:15.855308389 +0000 UTC m=+0.017796848 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:16.204 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:15 smithi071 podman[62552]: 2024-04-25 00:19:15.967802938 +0000 UTC m=+0.130291392 container create 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T00:19:16.204 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 podman[62552]: 2024-04-25 00:19:16.204318575 +0000 UTC m=+0.366807024 container init 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T00:19:16.489 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 podman[62552]: 2024-04-25 00:19:16.206951029 +0000 UTC m=+0.369439479 container start 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T00:19:16.489 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mds[62571]: -- 172.21.15.71:0/1425026119 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562160a07a40 con 0x5621609dac00 2024-04-25T00:19:16.489 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: starting mds.h at 2024-04-25T00:19:16.489 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 bash[62552]: 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 2024-04-25T00:19:16.489 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:19:16 smithi071 systemd[1]: Started Ceph mds.h for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2587344002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:16 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2587344002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:16.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:16.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:16 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:16.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:16.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2587344002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:16.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:16.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:16.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:16 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:17.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:17 smithi122 ceph-mon[32959]: Deploying daemon mds.i on smithi122 2024-04-25T00:19:17.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:17 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:boot 2024-04-25T00:19:17.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:17 smithi122 ceph-mon[32959]: fsmap 8 up:standby 2024-04-25T00:19:17.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:17 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:17.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:17 smithi071 ceph-mon[32667]: Deploying daemon mds.i on smithi122 2024-04-25T00:19:17.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:17 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:boot 2024-04-25T00:19:17.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:17 smithi071 ceph-mon[32667]: fsmap 8 up:standby 2024-04-25T00:19:17.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:17 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:17.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:17.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:17 smithi045 ceph-mon[29194]: Deploying daemon mds.i on smithi122 2024-04-25T00:19:17.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:17 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:boot 2024-04-25T00:19:17.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:17 smithi045 ceph-mon[29194]: fsmap 8 up:standby 2024-04-25T00:19:17.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:17 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:19:17.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 -- 172.21.15.45:0/350580241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 msgr2=0x7efe84102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 --2- 172.21.15.45:0/350580241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84102e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe74009900 tx=0x7efe7402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 -- 172.21.15.45:0/350580241 shutdown_connections 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 --2- 172.21.15.45:0/350580241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 --2- 172.21.15.45:0/350580241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84102e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 --2- 172.21.15.45:0/350580241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 0x7efe84101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 -- 172.21.15.45:0/350580241 >> 172.21.15.45:0/350580241 conn(0x7efe840fcd40 msgr2=0x7efe840ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 -- 172.21.15.45:0/350580241 shutdown_connections 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 -- 172.21.15.45:0/350580241 wait complete. 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.986+0000 7efe8a810640 1 Processor -- start 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 -- start start 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 0x7efe840754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:17.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410d450 con 0x7efe841029a0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410d2d0 con 0x7efe841017b0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe8a810640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410d5d0 con 0x7efe84103360 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40958/0 (socket says 172.21.15.45:40958) 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 -- 172.21.15.45:0/2529715962 learned_addr learned my addr 172.21.15.45:0/2529715962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe88dad640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe83fff640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 0x7efe840754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 -- 172.21.15.45:0/2529715962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 msgr2=0x7efe84074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 -- 172.21.15.45:0/2529715962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 msgr2=0x7efe840754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 0x7efe840754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe837fe640 1 -- 172.21.15.45:0/2529715962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe84074900 con 0x7efe841029a0 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.987+0000 7efe88dad640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:17.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe837fe640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe74009230 tx=0x7efe74004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7403f030 con 0x7efe841029a0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe74003ce0 con 0x7efe841029a0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe841a1bb0 con 0x7efe841029a0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74035070 con 0x7efe841029a0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.988+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a2150 con 0x7efe841029a0 2024-04-25T00:19:17.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.989+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84108ad0 con 0x7efe841029a0 2024-04-25T00:19:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.990+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe74042030 con 0x7efe841029a0 2024-04-25T00:19:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.991+0000 7efe817fa640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 0x7efe700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.991+0000 7efe83fff640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 0x7efe700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.991+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efe740ef5f0 con 0x7efe841029a0 2024-04-25T00:19:17.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.991+0000 7efe83fff640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 0x7efe700a50d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe6c005fd0 tx=0x7efe6c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:17.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:17.996+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe740bc9f0 con 0x7efe841029a0 2024-04-25T00:19:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.221+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8402d170 con 0x7efe841029a0 2024-04-25T00:19:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.222+0000 7efe817fa640 1 -- 172.21.15.45:0/2529715962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7efe740bc2f0 con 0x7efe841029a0 2024-04-25T00:19:18.223 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:18.223 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:19:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-04-25T00:19:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 msgr2=0x7efe700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 0x7efe700a50d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efe6c005fd0 tx=0x7efe6c005850 comp rx=0 tx=0).stop 2024-04-25T00:19:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 msgr2=0x7efe84073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe74009230 tx=0x7efe74004360 comp rx=0 tx=0).stop 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 shutdown_connections 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe700a2c10 0x7efe700a50d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe84103360 0x7efe84074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe841029a0 0x7efe84073b30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 --2- 172.21.15.45:0/2529715962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe841017b0 0x7efe840754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.224+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 >> 172.21.15.45:0/2529715962 conn(0x7efe840fcd40 msgr2=0x7efe84105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.225+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 shutdown_connections 2024-04-25T00:19:18.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:18.225+0000 7efe8a810640 1 -- 172.21.15.45:0/2529715962 wait complete. 2024-04-25T00:19:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:18 smithi122 ceph-mon[32959]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:18 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2529715962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:18.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:18 smithi071 ceph-mon[32667]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:18.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:18 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2529715962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:18.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:18 smithi045 ceph-mon[29194]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:18.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2529715962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:18.930 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:18 smithi122 podman[63811]: 2024-04-25 00:19:18.928374827 +0000 UTC m=+0.364657982 container init c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:19:18.931 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:18 smithi122 podman[63811]: 2024-04-25 00:19:18.930849331 +0000 UTC m=+0.367132487 container start c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:19.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:18 smithi122 ceph-mds[63830]: -- 172.21.15.122:0/297061674 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604863c5a40 con 0x560486398c00 2024-04-25T00:19:19.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: starting mds.i at 2024-04-25T00:19:19.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:19 smithi122 bash[63811]: c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 2024-04-25T00:19:19.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:19:19 smithi122 systemd[1]: Started Ceph mds.i for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:20.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:20.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 -- 172.21.15.45:0/466317851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 msgr2=0x7fd524101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 --2- 172.21.15.45:0/466317851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524101d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd518009900 tx=0x7fd51802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 -- 172.21.15.45:0/466317851 shutdown_connections 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 --2- 172.21.15.45:0/466317851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 0x7fd524108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 --2- 172.21.15.45:0/466317851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524101d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 --2- 172.21.15.45:0/466317851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 0x7fd524100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 -- 172.21.15.45:0/466317851 >> 172.21.15.45:0/466317851 conn(0x7fd5240fbc30 msgr2=0x7fd5240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 -- 172.21.15.45:0/466317851 shutdown_connections 2024-04-25T00:19:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.342+0000 7fd52b75a640 1 -- 172.21.15.45:0/466317851 wait complete. 2024-04-25T00:19:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 Processor -- start 2024-04-25T00:19:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 -- start start 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 0x7fd524104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 0x7fd524103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd52410c320 con 0x7fd524101890 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd52410c1a0 con 0x7fd5241006a0 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd52b75a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd52410c4a0 con 0x7fd524102250 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40978/0 (socket says 172.21.15.45:40978) 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 -- 172.21.15.45:0/3685233693 learned_addr learned my addr 172.21.15.45:0/3685233693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd529cf7640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 0x7fd524103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd5294f6640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 0x7fd524104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 -- 172.21.15.45:0/3685233693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 msgr2=0x7fd524103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:20.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 0x7fd524103300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 -- 172.21.15.45:0/3685233693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 msgr2=0x7fd524104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 0x7fd524104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 -- 172.21.15.45:0/3685233693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd524103b90 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.343+0000 7fd528cf5640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd518009230 tx=0x7fd518004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.344+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51803f030 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.344+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd518003ce0 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.344+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5241a1b60 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.344+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518035070 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.344+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5241a20b0 con 0x7fd524101890 2024-04-25T00:19:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.345+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5241a2280 con 0x7fd524101890 2024-04-25T00:19:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.346+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd518042030 con 0x7fd524101890 2024-04-25T00:19:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.346+0000 7fd5227fc640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 0x7fd50c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.346+0000 7fd5294f6640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 0x7fd50c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.346+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd5180ef1b0 con 0x7fd524101890 2024-04-25T00:19:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.347+0000 7fd5294f6640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 0x7fd50c0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.351+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd5180bc5b0 con 0x7fd524101890 2024-04-25T00:19:20.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: Deploying daemon mds.g on smithi045 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:boot 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: fsmap 9 up:standby 2024-04-25T00:19:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:20 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: Deploying daemon mds.g on smithi045 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:boot 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: fsmap 9 up:standby 2024-04-25T00:19:20.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:20 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: Deploying daemon mds.g on smithi045 2024-04-25T00:19:20.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:boot 2024-04-25T00:19:20.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: fsmap 9 up:standby 2024-04-25T00:19:20.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:20 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:19:20.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.574+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5241a2450 con 0x7fd524101890 2024-04-25T00:19:20.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.574+0000 7fd5227fc640 1 -- 172.21.15.45:0/3685233693 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7fd5180bbeb0 con 0x7fd524101890 2024-04-25T00:19:20.575 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:20.575 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:19:20.575 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 msgr2=0x7fd50c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 0x7fd50c0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c005850 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 msgr2=0x7fd524102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd518009230 tx=0x7fd518004360 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 shutdown_connections 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd50c0a2b90 0x7fd50c0a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd524102250 0x7fd524103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524101890 0x7fd524102dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 --2- 172.21.15.45:0/3685233693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5241006a0 0x7fd524104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 >> 172.21.15.45:0/3685233693 conn(0x7fd5240fbc30 msgr2=0x7fd524069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 shutdown_connections 2024-04-25T00:19:20.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:20.577+0000 7fd52b75a640 1 -- 172.21.15.45:0/3685233693 wait complete. 2024-04-25T00:19:21.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:21 smithi045 ceph-mon[29194]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:21.289 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3685233693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:21.289 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 systemd[1]: Starting Ceph mds.g for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:21 smithi071 ceph-mon[32667]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3685233693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:21 smithi122 ceph-mon[32959]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3685233693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:21.644 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 podman[68549]: 2024-04-25 00:19:21.413118415 +0000 UTC m=+0.017914897 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:21.645 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 podman[68549]: 2024-04-25 00:19:21.53114207 +0000 UTC m=+0.135938540 container create eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 podman[68549]: 2024-04-25 00:19:21.800632299 +0000 UTC m=+0.405428770 container init eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 podman[68549]: 2024-04-25 00:19:21.803328161 +0000 UTC m=+0.408124639 container start eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 ceph-mds[68568]: -- 172.21.15.45:0/3251866347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596cede9a40 con 0x5596cedbcc00 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: starting mds.g at 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 bash[68549]: eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c 2024-04-25T00:19:21.899 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:19:21 smithi045 systemd[1]: Started Ceph mds.g for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:22.589 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- 172.21.15.45:0/147416847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c102280 msgr2=0x7f525c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 --2- 172.21.15.45:0/147416847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c102280 0x7f525c104670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f524c009900 tx=0x7f524c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- 172.21.15.45:0/147416847 shutdown_connections 2024-04-25T00:19:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 --2- 172.21.15.45:0/147416847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c1074e0 0x7f525c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 --2- 172.21.15.45:0/147416847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c104bb0 0x7f525c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 --2- 172.21.15.45:0/147416847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c102280 0x7f525c104670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- 172.21.15.45:0/147416847 >> 172.21.15.45:0/147416847 conn(0x7f525c0fbc30 msgr2=0x7f525c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- 172.21.15.45:0/147416847 shutdown_connections 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- 172.21.15.45:0/147416847 wait complete. 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 Processor -- start 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.695+0000 7f5261fc8640 1 -- start start 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c102280 0x7f525c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c104bb0 0x7f525c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10d440 con 0x7f525c1074e0 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10d2c0 con 0x7f525c104bb0 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f525c10d5c0 con 0x7f525c102280 2024-04-25T00:19:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41012/0 (socket says 172.21.15.45:41012) 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 -- 172.21.15.45:0/899378176 learned_addr learned my addr 172.21.15.45:0/899378176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525b7fe640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c102280 0x7f525c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525affd640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c104bb0 0x7f525c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 -- 172.21.15.45:0/899378176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c102280 msgr2=0x7f525c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c102280 0x7f525c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 -- 172.21.15.45:0/899378176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c104bb0 msgr2=0x7f525c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c104bb0 0x7f525c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 -- 172.21.15.45:0/899378176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c19f730 con 0x7f525c1074e0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f525bfff640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f525400eb10 tx=0x7f525400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254004b20 con 0x7f525c1074e0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5254004cc0 con 0x7f525c1074e0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c19fa20 con 0x7f525c1074e0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254010760 con 0x7f525c1074e0 2024-04-25T00:19:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.696+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c19ff20 con 0x7f525c1074e0 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.697+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c0fd720 con 0x7f525c1074e0 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.698+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5254005020 con 0x7f525c1074e0 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.699+0000 7f5258ff9640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.699+0000 7f525b7fe640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.699+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f52540cf1f0 con 0x7f525c1074e0 2024-04-25T00:19:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.699+0000 7f525b7fe640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 0x7f52440a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f524c0314e0 tx=0x7f524c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.703+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f525409c550 con 0x7f525c1074e0 2024-04-25T00:19:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.938+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f525c02d170 con 0x7f525c1074e0 2024-04-25T00:19:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.939+0000 7f5258ff9640 1 -- 172.21.15.45:0/899378176 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f525409be50 con 0x7f525c1074e0 2024-04-25T00:19:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:22.940 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:19:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 msgr2=0x7f52440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 0x7f52440a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f524c0314e0 tx=0x7f524c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 msgr2=0x7f525c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f525400eb10 tx=0x7f525400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 shutdown_connections 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f52440a2b90 0x7f52440a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f525c1074e0 0x7f525c19f0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f525c104bb0 0x7f525c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 --2- 172.21.15.45:0/899378176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f525c102280 0x7f525c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:22.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 >> 172.21.15.45:0/899378176 conn(0x7f525c0fbc30 msgr2=0x7f525c102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:22.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 shutdown_connections 2024-04-25T00:19:22.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:22.941+0000 7f5261fc8640 1 -- 172.21.15.45:0/899378176 wait complete. 2024-04-25T00:19:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: Deploying daemon mds.a on smithi045 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6838/2848612989,v1:172.21.15.45:6839/2848612989] up:boot 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: fsmap 10 up:standby 2024-04-25T00:19:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:22 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: Deploying daemon mds.a on smithi045 2024-04-25T00:19:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6838/2848612989,v1:172.21.15.45:6839/2848612989] up:boot 2024-04-25T00:19:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: fsmap 10 up:standby 2024-04-25T00:19:23.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:22 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: Deploying daemon mds.a on smithi045 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6838/2848612989,v1:172.21.15.45:6839/2848612989] up:boot 2024-04-25T00:19:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: fsmap 10 up:standby 2024-04-25T00:19:23.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:22 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:19:24.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:23 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/899378176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:24.227 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 systemd[1]: Starting Ceph mds.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:24.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:23 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/899378176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:24.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:23 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/899378176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:24.517 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 podman[68894]: 2024-04-25 00:19:24.294123002 +0000 UTC m=+0.019060294 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:24.517 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 podman[68894]: 2024-04-25 00:19:24.4035539 +0000 UTC m=+0.128491218 container create 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 podman[68894]: 2024-04-25 00:19:24.63960705 +0000 UTC m=+0.364544323 container init 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 podman[68894]: 2024-04-25 00:19:24.643186377 +0000 UTC m=+0.368123653 container start 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mds[68922]: -- 172.21.15.45:0/283741531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557964bfda40 con 0x557964bd0c00 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: starting mds.a at 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 bash[68894]: 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 2024-04-25T00:19:24.899 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:19:24 smithi045 systemd[1]: Started Ceph mds.a for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:24.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 -- 172.21.15.45:0/3398717952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c101870 msgr2=0x7fe79c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 --2- 172.21.15.45:0/3398717952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c101870 0x7fe79c101cf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009900 tx=0x7fe78c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 -- 172.21.15.45:0/3398717952 shutdown_connections 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 --2- 172.21.15.45:0/3398717952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c102230 0x7fe79c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 --2- 172.21.15.45:0/3398717952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c101870 0x7fe79c101cf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 --2- 172.21.15.45:0/3398717952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c100680 0x7fe79c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.060+0000 7fe7a109e640 1 -- 172.21.15.45:0/3398717952 >> 172.21.15.45:0/3398717952 conn(0x7fe79c0fbbf0 msgr2=0x7fe79c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.061+0000 7fe7a109e640 1 -- 172.21.15.45:0/3398717952 shutdown_connections 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.061+0000 7fe7a109e640 1 -- 172.21.15.45:0/3398717952 wait complete. 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.061+0000 7fe7a109e640 1 Processor -- start 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.061+0000 7fe7a109e640 1 -- start start 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 0x7fe79c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 0x7fe79c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:25.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10c2e0 con 0x7fe79c102230 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10c160 con 0x7fe79c101870 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe7a109e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10c460 con 0x7fe79c100680 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36308/0 (socket says 172.21.15.45:36308) 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 -- 172.21.15.45:0/1416894316 learned_addr learned my addr 172.21.15.45:0/1416894316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79a59c640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 0x7fe79c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79ad9d640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 0x7fe79c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 -- 172.21.15.45:0/1416894316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 msgr2=0x7fe79c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 0x7fe79c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 -- 172.21.15.45:0/1416894316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 msgr2=0x7fe79c102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 0x7fe79c102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 -- 172.21.15.45:0/1416894316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c103a50 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79ad9d640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 0x7fe79c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79a59c640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 0x7fe79c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe79b59e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe79400cad0 tx=0x7fe79400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.062+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79400ae20 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.063+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c1a1c20 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.063+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe794005020 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.063+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79400f6f0 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.063+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1a2140 con 0x7fe79c102230 2024-04-25T00:19:25.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.064+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c068560 con 0x7fe79c102230 2024-04-25T00:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.065+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe7940051d0 con 0x7fe79c102230 2024-04-25T00:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.065+0000 7fe783fff640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 0x7fe7840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.066+0000 7fe79ad9d640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 0x7fe7840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.066+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7940cefa0 con 0x7fe79c102230 2024-04-25T00:19:25.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.066+0000 7fe79ad9d640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 0x7fe7840a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe79c1016d0 tx=0x7fe790005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:25.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.069+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe794014030 con 0x7fe79c102230 2024-04-25T00:19:25.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:25.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:25.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:25.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:24 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.293+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79c1a2590 con 0x7fe79c102230 2024-04-25T00:19:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.294+0000 7fe783fff640 1 -- 172.21.15.45:0/1416894316 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7fe79409c3a0 con 0x7fe79c102230 2024-04-25T00:19:25.295 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:25.295 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:19:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 msgr2=0x7fe7840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 0x7fe7840a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe79c1016d0 tx=0x7fe790005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 msgr2=0x7fe79c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe79400cad0 tx=0x7fe79400cf90 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 shutdown_connections 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe7840a2b90 0x7fe7840a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79c102230 0x7fe79c103190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe79c101870 0x7fe79c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 --2- 172.21.15.45:0/1416894316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c100680 0x7fe79c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 >> 172.21.15.45:0/1416894316 conn(0x7fe79c0fbbf0 msgr2=0x7fe79c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 shutdown_connections 2024-04-25T00:19:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:25.296+0000 7fe7a109e640 1 -- 172.21.15.45:0/1416894316 wait complete. 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:24 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:19:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:24 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:25 smithi122 ceph-mon[32959]: Deploying daemon mds.b on smithi071 2024-04-25T00:19:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:25 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6840/3316648689,v1:172.21.15.45:6841/3316648689] up:boot 2024-04-25T00:19:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:25 smithi122 ceph-mon[32959]: fsmap 11 up:standby 2024-04-25T00:19:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:25 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:19:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:25 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1416894316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:25 smithi071 ceph-mon[32667]: Deploying daemon mds.b on smithi071 2024-04-25T00:19:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:25 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6840/3316648689,v1:172.21.15.45:6841/3316648689] up:boot 2024-04-25T00:19:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:25 smithi071 ceph-mon[32667]: fsmap 11 up:standby 2024-04-25T00:19:26.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:25 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:19:26.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:25 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1416894316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:26.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:25 smithi045 ceph-mon[29194]: Deploying daemon mds.b on smithi071 2024-04-25T00:19:26.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:25 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6840/3316648689,v1:172.21.15.45:6841/3316648689] up:boot 2024-04-25T00:19:26.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:25 smithi045 ceph-mon[29194]: fsmap 11 up:standby 2024-04-25T00:19:26.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:25 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:19:26.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:25 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1416894316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:27.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:26 smithi071 ceph-mon[32667]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:27.271 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 systemd[1]: Starting Ceph mds.b for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:19:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:26 smithi122 ceph-mon[32959]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:27.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:27.335 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:26 smithi045 ceph-mon[29194]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 -- 172.21.15.45:0/1986096712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c102280 msgr2=0x7f7e4c104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 --2- 172.21.15.45:0/1986096712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c102280 0x7f7e4c104670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009900 tx=0x7f7e3c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 -- 172.21.15.45:0/1986096712 shutdown_connections 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 --2- 172.21.15.45:0/1986096712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 0x7f7e4c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 --2- 172.21.15.45:0/1986096712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 --2- 172.21.15.45:0/1986096712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c102280 0x7f7e4c104670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 -- 172.21.15.45:0/1986096712 >> 172.21.15.45:0/1986096712 conn(0x7f7e4c0fbc30 msgr2=0x7f7e4c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 -- 172.21.15.45:0/1986096712 shutdown_connections 2024-04-25T00:19:27.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.423+0000 7f7e52064640 1 -- 172.21.15.45:0/1986096712 wait complete. 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 Processor -- start 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 -- start start 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e4c102280 0x7f7e4c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 0x7f7e4c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10d4c0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10d340 con 0x7f7e4c1074e0 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e52064640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4c10d640 con 0x7f7e4c102280 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36326/0 (socket says 172.21.15.45:36326) 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 -- 172.21.15.45:0/961265624 learned_addr learned my addr 172.21.15.45:0/961265624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4bfff640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 0x7f7e4c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 -- 172.21.15.45:0/961265624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e4c102280 msgr2=0x7f7e4c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e4c102280 0x7f7e4c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 -- 172.21.15.45:0/961265624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 msgr2=0x7f7e4c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 0x7f7e4c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.424+0000 7f7e4affd640 1 -- 172.21.15.45:0/961265624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4c19f780 con 0x7f7e4c104bb0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e4affd640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c970 tx=0x7f7e4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40007bb0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e40007d50 con 0x7f7e4c104bb0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4c19fa70 con 0x7f7e4c104bb0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400187a0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.425+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4c19fff0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.426+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4c0fd720 con 0x7f7e4c104bb0 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.427+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7e4000f450 con 0x7f7e4c104bb0 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.428+0000 7f7e48ff9640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 0x7f7e340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.428+0000 7f7e4b7fe640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 0x7f7e340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.428+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7e400cebe0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.428+0000 7f7e4b7fe640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 0x7f7e340a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0096c0 tx=0x7f7e3c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.432+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7e40014030 con 0x7f7e4c104bb0 2024-04-25T00:19:27.522 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 podman[62857]: 2024-04-25 00:19:27.174306168 +0000 UTC m=+0.018149230 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:19:27.522 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 podman[62857]: 2024-04-25 00:19:27.2910357 +0000 UTC m=+0.134878742 container create 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:19:27.522 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 podman[62857]: 2024-04-25 00:19:27.519386784 +0000 UTC m=+0.363229832 container init 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:19:27.522 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 podman[62857]: 2024-04-25 00:19:27.522082992 +0000 UTC m=+0.365926032 container start 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T00:19:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.656+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e4c02d170 con 0x7f7e4c104bb0 2024-04-25T00:19:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.657+0000 7f7e48ff9640 1 -- 172.21.15.45:0/961265624 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f7e4009bfe0 con 0x7f7e4c104bb0 2024-04-25T00:19:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:27.658 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:19:27.658 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-04-25T00:19:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 msgr2=0x7f7e340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 0x7f7e340a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0096c0 tx=0x7f7e3c031400 comp rx=0 tx=0).stop 2024-04-25T00:19:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 msgr2=0x7f7e4c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c970 tx=0x7f7e4000ce30 comp rx=0 tx=0).stop 2024-04-25T00:19:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 shutdown_connections 2024-04-25T00:19:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e340a2b90 0x7f7e340a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4c1074e0 0x7f7e4c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4c104bb0 0x7f7e4c1980a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 --2- 172.21.15.45:0/961265624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e4c102280 0x7f7e4c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 >> 172.21.15.45:0/961265624 conn(0x7f7e4c0fbc30 msgr2=0x7f7e4c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 shutdown_connections 2024-04-25T00:19:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:27.659+0000 7f7e52064640 1 -- 172.21.15.45:0/961265624 wait complete. 2024-04-25T00:19:27.868 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mds[62876]: -- 172.21.15.71:0/3908736746 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ddd685a40 con 0x563ddd658c00 2024-04-25T00:19:27.868 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: starting mds.b at 2024-04-25T00:19:27.868 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 bash[62857]: 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf 2024-04-25T00:19:27.869 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:27 smithi071 systemd[1]: Started Ceph mds.b for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/961265624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/961265624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/961265624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:28.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:28 smithi122 ceph-mon[32959]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:28 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:boot 2024-04-25T00:19:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:28 smithi122 ceph-mon[32959]: fsmap 12 up:standby 2024-04-25T00:19:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:28 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:28 smithi071 ceph-mon[32667]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:29.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:28 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:boot 2024-04-25T00:19:29.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:28 smithi071 ceph-mon[32667]: fsmap 12 up:standby 2024-04-25T00:19:29.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:28 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:28 smithi045 ceph-mon[29194]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:28 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:boot 2024-04-25T00:19:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:28 smithi045 ceph-mon[29194]: fsmap 12 up:standby 2024-04-25T00:19:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:28 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:19:29.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:29.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 -- 172.21.15.45:0/1653452639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0100680 msgr2=0x7f1ba0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/1653452639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0100680 0x7f1ba0100aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009ca0 tx=0x7f1b9002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 -- 172.21.15.45:0/1653452639 shutdown_connections 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/1653452639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0102230 0x7f1ba0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/1653452639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0101870 0x7f1ba0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.777+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/1653452639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0100680 0x7f1ba0100aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- 172.21.15.45:0/1653452639 >> 172.21.15.45:0/1653452639 conn(0x7f1ba00fbbf0 msgr2=0x7f1ba00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- 172.21.15.45:0/1653452639 shutdown_connections 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- 172.21.15.45:0/1653452639 wait complete. 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 Processor -- start 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- start start 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0101870 0x7f1ba0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c2d0 con 0x7f1ba0102230 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c150 con 0x7f1ba0100680 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.778+0000 7f1ba72ca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba010c450 con 0x7f1ba0101870 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5066640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36338/0 (socket says 172.21.15.45:36338) 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5066640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:59708/0 (socket says 172.21.15.45:59708) 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 -- 172.21.15.45:0/960673493 learned_addr learned my addr 172.21.15.45:0/960673493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba4865640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0101870 0x7f1ba0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 -- 172.21.15.45:0/960673493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0101870 msgr2=0x7f1ba0196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0101870 0x7f1ba0196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 -- 172.21.15.45:0/960673493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 msgr2=0x7f1ba0196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 -- 172.21.15.45:0/960673493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019e620 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5066640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba5867640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800e9b0 tx=0x7f1b9800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9800cd80 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba019e910 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b980035f0 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98010670 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.779+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba019ee60 con 0x7f1ba0102230 2024-04-25T00:19:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.780+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba0107a90 con 0x7f1ba0102230 2024-04-25T00:19:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.782+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1b980037a0 con 0x7f1ba0102230 2024-04-25T00:19:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.783+0000 7f1b9e7fc640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 0x7f1b880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.783+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1b980ceff0 con 0x7f1ba0102230 2024-04-25T00:19:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.783+0000 7f1ba5066640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 0x7f1b880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:29.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.783+0000 7f1ba5066640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 0x7f1b880a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006010 tx=0x7f1b90000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:29.786+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b9809c400 con 0x7f1ba0102230 2024-04-25T00:19:30.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.011+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ba002d320 con 0x7f1ba0102230 2024-04-25T00:19:30.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.012+0000 7f1b9e7fc640 1 -- 172.21.15.45:0/960673493 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f1b9809bd00 con 0x7f1ba0102230 2024-04-25T00:19:30.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:30.013 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T00:19:30.013 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-04-25T00:19:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 msgr2=0x7f1b880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 0x7f1b880a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1b90006010 tx=0x7f1b90000f40 comp rx=0 tx=0).stop 2024-04-25T00:19:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 msgr2=0x7f1ba019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800e9b0 tx=0x7f1b9800ee70 comp rx=0 tx=0).stop 2024-04-25T00:19:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 shutdown_connections 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1b880a2b90 0x7f1b880a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ba0102230 0x7f1ba019dfb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba0101870 0x7f1ba0196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.014+0000 7f1ba72ca640 1 --2- 172.21.15.45:0/960673493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0100680 0x7f1ba0196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.015+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 >> 172.21.15.45:0/960673493 conn(0x7f1ba00fbbf0 msgr2=0x7f1ba0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.015+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 shutdown_connections 2024-04-25T00:19:30.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.015+0000 7f1ba72ca640 1 -- 172.21.15.45:0/960673493 wait complete. 2024-04-25T00:19:30.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.129+0000 7effec838640 1 -- 172.21.15.45:0/1144000865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81017b0 msgr2=0x7effe8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.129+0000 7effec838640 1 --2- 172.21.15.45:0/1144000865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81017b0 0x7effe8101bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effd8009900 tx=0x7effd802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- 172.21.15.45:0/1144000865 shutdown_connections 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- 172.21.15.45:0/1144000865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- 172.21.15.45:0/1144000865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81029a0 0x7effe8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- 172.21.15.45:0/1144000865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81017b0 0x7effe8101bd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- 172.21.15.45:0/1144000865 >> 172.21.15.45:0/1144000865 conn(0x7effe80fcd40 msgr2=0x7effe80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- 172.21.15.45:0/1144000865 shutdown_connections 2024-04-25T00:19:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- 172.21.15.45:0/1144000865 wait complete. 2024-04-25T00:19:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 Processor -- start 2024-04-25T00:19:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- start start 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81029a0 0x7effe8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effe810d420 con 0x7effe81029a0 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effe810d2a0 con 0x7effe8103360 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.130+0000 7effec838640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effe810d5a0 con 0x7effe81017b0 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81029a0 0x7effe8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:44414/0 (socket says 172.21.15.45:44414) 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 -- 172.21.15.45:0/3031592155 learned_addr learned my addr 172.21.15.45:0/3031592155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe6d9d640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 -- 172.21.15.45:0/3031592155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 msgr2=0x7effe819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 -- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81029a0 msgr2=0x7effe8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81029a0 0x7effe8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 -- 172.21.15.45:0/3031592155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe819f710 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe6d9d640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effe659c640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7effd8031610 tx=0x7effd80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd803f030 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe819f9a0 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd8003ce0 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8035070 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.131+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe819fee0 con 0x7effe81017b0 2024-04-25T00:19:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.132+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe8108b70 con 0x7effe81017b0 2024-04-25T00:19:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.134+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7effd8042030 con 0x7effe81017b0 2024-04-25T00:19:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.134+0000 7effdf7fe640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 0x7effd40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.134+0000 7effe5d9b640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 0x7effd40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.135+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7effd80ef3e0 con 0x7effe81017b0 2024-04-25T00:19:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.135+0000 7effe5d9b640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 0x7effd40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effe8199000 tx=0x7effd000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.143+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7effd80bc170 con 0x7effe81017b0 2024-04-25T00:19:30.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.330+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effe802d370 con 0x7effe81017b0 2024-04-25T00:19:30.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.331+0000 7effdf7fe640 1 -- 172.21.15.45:0/3031592155 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18444 (secure 0 0 0) 0x7effd80c0470 con 0x7effe81017b0 2024-04-25T00:19:30.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:30.332 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:18:42.079045+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 msgr2=0x7effd40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 0x7effd40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effe8199000 tx=0x7effd000b390 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 msgr2=0x7effe8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7effd8031610 tx=0x7effd80043b0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 shutdown_connections 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7effd40a2b90 0x7effd40a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe8103360 0x7effe819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe81029a0 0x7effe8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 --2- 172.21.15.45:0/3031592155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe81017b0 0x7effe8197af0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 >> 172.21.15.45:0/3031592155 conn(0x7effe80fcd40 msgr2=0x7effe8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 shutdown_connections 2024-04-25T00:19:30.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.333+0000 7effec838640 1 -- 172.21.15.45:0/3031592155 wait complete. 2024-04-25T00:19:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:19:30.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:19:30.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- 172.21.15.45:0/3996330446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c104b50 msgr2=0x7f8f8c106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/3996330446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c104b50 0x7f8f8c106f40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009900 tx=0x7f8f8802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- 172.21.15.45:0/3996330446 shutdown_connections 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/3996330446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c107480 0x7f8f8c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/3996330446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c104b50 0x7f8f8c106f40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/3996330446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c102220 0x7f8f8c104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- 172.21.15.45:0/3996330446 >> 172.21.15.45:0/3996330446 conn(0x7f8f8c0fbc30 msgr2=0x7f8f8c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- 172.21.15.45:0/3996330446 shutdown_connections 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- 172.21.15.45:0/3996330446 wait complete. 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 Processor -- start 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 -- start start 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.465+0000 7f8f93b1d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c104b50 0x7f8f8c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 0x7f8f8c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10d420 con 0x7f8f8c102220 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10d2a0 con 0x7f8f8c107480 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c10d5a0 con 0x7f8f8c104b50 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36380/0 (socket says 172.21.15.45:36380) 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 -- 172.21.15.45:0/339228729 learned_addr learned my addr 172.21.15.45:0/339228729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f920ba640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 0x7f8f8c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f910b8640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c104b50 0x7f8f8c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 -- 172.21.15.45:0/339228729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c104b50 msgr2=0x7f8f8c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c104b50 0x7f8f8c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 -- 172.21.15.45:0/339228729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 msgr2=0x7f8f8c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 0x7f8f8c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 -- 172.21.15.45:0/339228729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19f770 con 0x7f8f8c102220 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f920ba640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 0x7f8f8c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f918b9640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9f0 tx=0x7f8f7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00cd80 con 0x7f8f8c102220 2024-04-25T00:19:30.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c19fa60 con 0x7f8f8c102220 2024-04-25T00:19:30.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c010910 con 0x7f8f8c102220 2024-04-25T00:19:30.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0188f0 con 0x7f8f8c102220 2024-04-25T00:19:30.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.466+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c19ffd0 con 0x7f8f8c102220 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.468+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c0fd720 con 0x7f8f8c102220 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.469+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f7c010ac0 con 0x7f8f8c102220 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.469+0000 7f8f82ffd640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 0x7f8f780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.469+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8f7c0cef80 con 0x7f8f8c102220 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.469+0000 7f8f910b8640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 0x7f8f780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.470+0000 7f8f910b8640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 0x7f8f780a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009230 tx=0x7f8f880315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.474+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f7c09bd10 con 0x7f8f8c102220 2024-04-25T00:19:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.668+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f8c02d390 con 0x7f8f8c102220 2024-04-25T00:19:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.669+0000 7f8f82ffd640 1 -- 172.21.15.45:0/339228729 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f8f7c0a0010 con 0x7f8f8c102220 2024-04-25T00:19:30.670 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:19:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.671+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 msgr2=0x7f8f780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.671+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 0x7f8f780a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009230 tx=0x7f8f880315b0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.671+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 msgr2=0x7f8f8c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.671+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00e9f0 tx=0x7f8f7c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 shutdown_connections 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8f780a2b90 0x7f8f780a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c107480 0x7f8f8c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c104b50 0x7f8f8c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 --2- 172.21.15.45:0/339228729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f8c102220 0x7f8f8c197b50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 >> 172.21.15.45:0/339228729 conn(0x7f8f8c0fbc30 msgr2=0x7f8f8c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 shutdown_connections 2024-04-25T00:19:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.672+0000 7f8f93b1d640 1 -- 172.21.15.45:0/339228729 wait complete. 2024-04-25T00:19:30.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:19:30.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 -- 172.21.15.45:0/619021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a01006a0 msgr2=0x7f34a0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 --2- 172.21.15.45:0/619021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a01006a0 0x7f34a0100ac0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3490009900 tx=0x7f349002a410 comp rx=0 tx=0).stop 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 -- 172.21.15.45:0/619021682 shutdown_connections 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 --2- 172.21.15.45:0/619021682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0102250 0x7f34a0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 --2- 172.21.15.45:0/619021682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a0101890 0x7f34a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 --2- 172.21.15.45:0/619021682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a01006a0 0x7f34a0100ac0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.789+0000 7f34a4e29640 1 -- 172.21.15.45:0/619021682 >> 172.21.15.45:0/619021682 conn(0x7f34a00fbc30 msgr2=0x7f34a00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.790+0000 7f34a4e29640 1 -- 172.21.15.45:0/619021682 shutdown_connections 2024-04-25T00:19:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.790+0000 7f34a4e29640 1 -- 172.21.15.45:0/619021682 wait complete. 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.790+0000 7f34a4e29640 1 Processor -- start 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 -- start start 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a01006a0 0x7f34a0196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 0x7f34a0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010c370 con 0x7f34a0102250 2024-04-25T00:19:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010c1f0 con 0x7f34a0101890 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f34a4e29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010c4f0 con 0x7f34a01006a0 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a01006a0 0x7f34a0196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 0x7f34a0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36406/0 (socket says 172.21.15.45:36406) 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 -- 172.21.15.45:0/1493134197 learned_addr learned my addr 172.21.15.45:0/1493134197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 -- 172.21.15.45:0/1493134197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a01006a0 msgr2=0x7f34a0196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a01006a0 0x7f34a0196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 -- 172.21.15.45:0/1493134197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 msgr2=0x7f34a0196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 0x7f34a0196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 -- 172.21.15.45:0/1493134197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a019e640 con 0x7f34a0102250 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349dd9b640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 0x7f34a0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.791+0000 7f349ed9d640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349800eaa0 tx=0x7f349800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.792+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004d20 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.792+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498005020 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.792+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980116d0 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.792+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a019e930 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.792+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a019eeb0 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.793+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a0107a90 con 0x7f34a0102250 2024-04-25T00:19:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.795+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f34980051d0 con 0x7f34a0102250 2024-04-25T00:19:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.795+0000 7f348f7fe640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 0x7f34880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.795+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f34980cf110 con 0x7f34a0102250 2024-04-25T00:19:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.795+0000 7f349e59c640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 0x7f34880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.796+0000 7f349e59c640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 0x7f34880a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3490005ab0 tx=0x7f3490000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.799+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3498016030 con 0x7f34a0102250 2024-04-25T00:19:30.894 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/960673493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.895 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:30 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3031592155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/960673493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:30 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3031592155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/960673493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:30.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:30 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3031592155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:30.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.989+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f34a0062340 con 0x7f34a0102250 2024-04-25T00:19:30.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.990+0000 7f348f7fe640 1 -- 172.21.15.45:0/1493134197 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f349809c4f0 con 0x7f34a0102250 2024-04-25T00:19:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:19:30.991 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T00:19:30.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.991+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 msgr2=0x7f34880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.991+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 0x7f34880a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3490005ab0 tx=0x7f3490000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.991+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 msgr2=0x7f34a019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349800eaa0 tx=0x7f349800eea0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 shutdown_connections 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34880a2b90 0x7f34880a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34a0102250 0x7f34a019dfd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a0101890 0x7f34a0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 --2- 172.21.15.45:0/1493134197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a01006a0 0x7f34a0196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 >> 172.21.15.45:0/1493134197 conn(0x7f34a00fbc30 msgr2=0x7f34a0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 shutdown_connections 2024-04-25T00:19:30.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:30.992+0000 7f34a4e29640 1 -- 172.21.15.45:0/1493134197 wait complete. 2024-04-25T00:19:31.003 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:19:31.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 -- 172.21.15.45:0/2824841371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8018102250 msgr2=0x7f8018108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 --2- 172.21.15.45:0/2824841371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8018102250 0x7f8018108ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8010009ca0 tx=0x7f801002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 -- 172.21.15.45:0/2824841371 shutdown_connections 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 --2- 172.21.15.45:0/2824841371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8018102250 0x7f8018108ae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 --2- 172.21.15.45:0/2824841371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 0x7f8018101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 --2- 172.21.15.45:0/2824841371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80181006a0 0x7f8018100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 -- 172.21.15.45:0/2824841371 >> 172.21.15.45:0/2824841371 conn(0x7f80180fbc30 msgr2=0x7f80180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 -- 172.21.15.45:0/2824841371 shutdown_connections 2024-04-25T00:19:31.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.111+0000 7f801db1d640 1 -- 172.21.15.45:0/2824841371 wait complete. 2024-04-25T00:19:31.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 Processor -- start 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 -- start start 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 0x7f801819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8018102250 0x7f80181a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c370 con 0x7f80181006a0 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c1f0 con 0x7f8018101890 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f801db1d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f801810c4f0 con 0x7f8018102250 2024-04-25T00:19:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f80177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f80177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36424/0 (socket says 172.21.15.45:36424) 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f80177fe640 1 -- 172.21.15.45:0/4284395526 learned_addr learned my addr 172.21.15.45:0/4284395526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f8016ffd640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 0x7f801819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.112+0000 7f8017fff640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8018102250 0x7f80181a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 -- 172.21.15.45:0/4284395526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8018102250 msgr2=0x7f80181a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8018102250 0x7f80181a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 -- 172.21.15.45:0/4284395526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 msgr2=0x7f801819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 0x7f801819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 -- 172.21.15.45:0/4284395526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80181a1910 con 0x7f80181006a0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f80177fe640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9c0 tx=0x7f800800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800cd80 con 0x7f80181006a0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181a1c00 con 0x7f80181006a0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.113+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a2150 con 0x7f80181006a0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.114+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80080035f0 con 0x7f80181006a0 2024-04-25T00:19:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.114+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008010690 con 0x7f80181006a0 2024-04-25T00:19:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.114+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80040052c0 con 0x7f80181006a0 2024-04-25T00:19:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.116+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80080037a0 con 0x7f80181006a0 2024-04-25T00:19:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.117+0000 7f8014ff9640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 0x7f80000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.117+0000 7f8016ffd640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 0x7f80000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.117+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80080cf040 con 0x7f80181006a0 2024-04-25T00:19:31.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.117+0000 7f8016ffd640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 0x7f80000a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f800c009bd0 tx=0x7f800c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:31.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.120+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f800809c440 con 0x7f80181006a0 2024-04-25T00:19:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.364+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f80040055b0 con 0x7f80181006a0 2024-04-25T00:19:31.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.852+0000 7f8014ff9640 1 -- 172.21.15.45:0/4284395526 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f800809bd40 con 0x7f80181006a0 2024-04-25T00:19:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T00:19:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 msgr2=0x7f80000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 0x7f80000a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f800c009bd0 tx=0x7f800c005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 msgr2=0x7f8018199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9c0 tx=0x7f800800ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 shutdown_connections 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f80000a2b90 0x7f80000a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8018102250 0x7f80181a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8018101890 0x7f801819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 --2- 172.21.15.45:0/4284395526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80181006a0 0x7f8018199d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 >> 172.21.15.45:0/4284395526 conn(0x7f80180fbc30 msgr2=0x7f8018069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 shutdown_connections 2024-04-25T00:19:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.855+0000 7f801db1d640 1 -- 172.21.15.45:0/4284395526 wait complete. 2024-04-25T00:19:31.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- 172.21.15.45:0/1280208644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b4102280 msgr2=0x7fa0b4104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/1280208644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b4102280 0x7fa0b4104670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009900 tx=0x7fa0a402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- 172.21.15.45:0/1280208644 shutdown_connections 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/1280208644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b41074e0 0x7fa0b41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/1280208644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/1280208644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b4102280 0x7fa0b4104670 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- 172.21.15.45:0/1280208644 >> 172.21.15.45:0/1280208644 conn(0x7fa0b40fbc30 msgr2=0x7fa0b40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- 172.21.15.45:0/1280208644 shutdown_connections 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- 172.21.15.45:0/1280208644 wait complete. 2024-04-25T00:19:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 Processor -- start 2024-04-25T00:19:31.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.981+0000 7fa0ba2da640 1 -- start start 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 0x7fa0b4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410d440 con 0x7fa0b41074e0 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410d2c0 con 0x7fa0b4102280 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410d5c0 con 0x7fa0b4104bb0 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36442/0 (socket says 172.21.15.45:36442) 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 -- 172.21.15.45:0/270666036 learned_addr learned my addr 172.21.15.45:0/270666036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b3fff640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 0x7fa0b4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b37fe640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 -- 172.21.15.45:0/270666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 msgr2=0x7fa0b4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 -- 172.21.15.45:0/270666036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 msgr2=0x7fa0b4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 0x7fa0b4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 -- 172.21.15.45:0/270666036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b419f6e0 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b37fe640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b3fff640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 0x7fa0b4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b8877640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00cce0 tx=0x7fa0ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00f450 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b419f9d0 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ac00fad0 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.982+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac018c10 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.983+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b419ff20 con 0x7fa0b41074e0 2024-04-25T00:19:31.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.983+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b40fd720 con 0x7fa0b41074e0 2024-04-25T00:19:31.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.985+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa0ac00f5f0 con 0x7fa0b41074e0 2024-04-25T00:19:31.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.986+0000 7fa0b17fa640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 0x7fa09c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:31.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.986+0000 7fa0b3fff640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 0x7fa09c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:31.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.986+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fa0ac0cf510 con 0x7fa0b41074e0 2024-04-25T00:19:31.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.986+0000 7fa0b3fff640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 0x7fa09c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000c00 tx=0x7fa0a40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:31.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:31.990+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa0ac014030 con 0x7fa0b41074e0 2024-04-25T00:19:32.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:31 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/339228729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:32.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:31 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1493134197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:19:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:31 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/339228729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:32.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:31 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1493134197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:19:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.175+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa0b4062340 con 0x7fa0b41074e0 2024-04-25T00:19:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:31 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/339228729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:32.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:31 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1493134197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:19:32.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.853+0000 7fa0b17fa640 1 -- 172.21.15.45:0/270666036 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7fa0ac09c830 con 0x7fa0b41074e0 2024-04-25T00:19:32.854 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T00:19:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 msgr2=0x7fa09c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 0x7fa09c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000c00 tx=0x7fa0a40095f0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 msgr2=0x7fa0b419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00cce0 tx=0x7fa0ac00a620 comp rx=0 tx=0).stop 2024-04-25T00:19:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 shutdown_connections 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa09c0a2b90 0x7fa09c0a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b41074e0 0x7fa0b419f070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0b4104bb0 0x7fa0b4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 --2- 172.21.15.45:0/270666036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b4102280 0x7fa0b4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 >> 172.21.15.45:0/270666036 conn(0x7fa0b40fbc30 msgr2=0x7fa0b4102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.857+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 shutdown_connections 2024-04-25T00:19:32.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.858+0000 7fa0ba2da640 1 -- 172.21.15.45:0/270666036 wait complete. 2024-04-25T00:19:32.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 -- 172.21.15.45:0/2795854338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 msgr2=0x7f84b4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 --2- 172.21.15.45:0/2795854338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4101d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009900 tx=0x7f84a802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 -- 172.21.15.45:0/2795854338 shutdown_connections 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 --2- 172.21.15.45:0/2795854338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b4102250 0x7f84b4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 --2- 172.21.15.45:0/2795854338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4101d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 --2- 172.21.15.45:0/2795854338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b41006a0 0x7f84b4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 -- 172.21.15.45:0/2795854338 >> 172.21.15.45:0/2795854338 conn(0x7f84b40fbc30 msgr2=0x7f84b40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 -- 172.21.15.45:0/2795854338 shutdown_connections 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 -- 172.21.15.45:0/2795854338 wait complete. 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.985+0000 7f84ba329640 1 Processor -- start 2024-04-25T00:19:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 -- start start 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b41006a0 0x7f84b41045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4102250 0x7f84b41031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410c300 con 0x7f84b4101890 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410c180 con 0x7f84b4102250 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84ba329640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410c480 con 0x7f84b41006a0 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 -- 172.21.15.45:0/1257639609 learned_addr learned my addr 172.21.15.45:0/1257639609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b88c6640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4102250 0x7f84b41031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b3fff640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b41006a0 0x7f84b41045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 -- 172.21.15.45:0/1257639609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b41006a0 msgr2=0x7f84b41045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b41006a0 0x7f84b41045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 -- 172.21.15.45:0/1257639609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4102250 msgr2=0x7f84b41031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4102250 0x7f84b41031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 -- 172.21.15.45:0/1257639609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b4103a70 con 0x7f84b4101890 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.986+0000 7f84b37fe640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009230 tx=0x7f84a8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.987+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a803f030 con 0x7f84b4101890 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.987+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a8003ce0 con 0x7f84b4101890 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.987+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a1c40 con 0x7f84b4101890 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.987+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8035070 con 0x7f84b4101890 2024-04-25T00:19:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.987+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a2100 con 0x7f84b4101890 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.988+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b41a22d0 con 0x7f84b4101890 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.989+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f84a8042030 con 0x7f84b4101890 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.990+0000 7f84b17fa640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 0x7f849c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.990+0000 7f84b3fff640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 0x7f849c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.990+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f84a80ef970 con 0x7f84b4101890 2024-04-25T00:19:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.990+0000 7f84b3fff640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 0x7f849c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005db0 tx=0x7f84a4005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:32.994+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84a80bca60 con 0x7f84b4101890 2024-04-25T00:19:33.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:19:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:19:33.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:32 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:19:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:19:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:19:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:32 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:19:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.220+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f84b41a2660 con 0x7f84b4101890 2024-04-25T00:19:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:19:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4284395526' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:19:33.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:32 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:19:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.862+0000 7f84b17fa640 1 -- 172.21.15.45:0/1257639609 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f84a80bc360 con 0x7f84b4101890 2024-04-25T00:19:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:19:33.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 msgr2=0x7f849c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:33.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 0x7f849c0a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005db0 tx=0x7f84a4005d40 comp rx=0 tx=0).stop 2024-04-25T00:19:33.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 msgr2=0x7f84b4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:33.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009230 tx=0x7f84a8004360 comp rx=0 tx=0).stop 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 shutdown_connections 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f849c0a2b90 0x7f849c0a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4102250 0x7f84b41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b4101890 0x7f84b4102c70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 --2- 172.21.15.45:0/1257639609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84b41006a0 0x7f84b41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 >> 172.21.15.45:0/1257639609 conn(0x7f84b40fbc30 msgr2=0x7f84b40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 shutdown_connections 2024-04-25T00:19:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:33.871+0000 7f84ba329640 1 -- 172.21.15.45:0/1257639609 wait complete. 2024-04-25T00:19:33.894 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T00:19:33.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 -- 172.21.15.45:0/1917627335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910102980 msgr2=0x7f1910102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 --2- 172.21.15.45:0/1917627335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910102980 0x7f1910102e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009900 tx=0x7f18f802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 -- 172.21.15.45:0/1917627335 shutdown_connections 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 --2- 172.21.15.45:0/1917627335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910103340 0x7f1910109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 --2- 172.21.15.45:0/1917627335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910102980 0x7f1910102e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 --2- 172.21.15.45:0/1917627335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910101790 0x7f1910101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 -- 172.21.15.45:0/1917627335 >> 172.21.15.45:0/1917627335 conn(0x7f19100fcd20 msgr2=0x7f19100ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 -- 172.21.15.45:0/1917627335 shutdown_connections 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.007+0000 7f1914f21640 1 -- 172.21.15.45:0/1917627335 wait complete. 2024-04-25T00:19:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 Processor -- start 2024-04-25T00:19:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 -- start start 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910101790 0x7f1910197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910102980 0x7f1910197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f191010d470 con 0x7f1910103340 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f191010d2f0 con 0x7f1910102980 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1914f21640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f191010d5f0 con 0x7f1910101790 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57100/0 (socket says 172.21.15.45:57100) 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 -- 172.21.15.45:0/3772322823 learned_addr learned my addr 172.21.15.45:0/3772322823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190e59c640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910101790 0x7f1910197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f1905d9b640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910102980 0x7f1910197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 -- 172.21.15.45:0/3772322823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910101790 msgr2=0x7f1910197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910101790 0x7f1910197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 -- 172.21.15.45:0/3772322823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910102980 msgr2=0x7f1910197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910102980 0x7f1910197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.008+0000 7f190ed9d640 1 -- 172.21.15.45:0/3772322823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191019f6d0 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f190ed9d640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f190800b970 tx=0x7f190800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908019070 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191019f9c0 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190800cb40 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908007610 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.009+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191019ffe0 con 0x7f1910103340 2024-04-25T00:19:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.010+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1910108b30 con 0x7f1910103340 2024-04-25T00:19:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.011+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f190801c070 con 0x7f1910103340 2024-04-25T00:19:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.012+0000 7f1907fff640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 0x7f18fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.012+0000 7f190e59c640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 0x7f18fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.012+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f19080ce460 con 0x7f1910103340 2024-04-25T00:19:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.012+0000 7f190e59c640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 0x7f18fc0a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19101027e0 tx=0x7f1900005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.016+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f190809aec0 con 0x7f1910103340 2024-04-25T00:19:34.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:33 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:19:34.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:33 smithi071 ceph-mon[32667]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:19:34.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:33 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:19:34.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:33 smithi071 ceph-mon[32667]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:19:34.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:33 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:19:34.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:33 smithi045 ceph-mon[29194]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:19:34.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:33 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:19:34.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:33 smithi045 ceph-mon[29194]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:19:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.212+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f191002d170 con 0x7f1910103340 2024-04-25T00:19:34.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:33 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/270666036' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:19:34.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:33 smithi122 ceph-mon[32959]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:19:34.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:33 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:19:34.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:33 smithi122 ceph-mon[32959]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:19:34.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.869+0000 7f1907fff640 1 -- 172.21.15.45:0/3772322823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f190809f1c0 con 0x7f1910103340 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 msgr2=0x7f18fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 0x7f18fc0a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19101027e0 tx=0x7f1900005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 msgr2=0x7f191019f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f190800b970 tx=0x7f190800be30 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 shutdown_connections 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18fc0a2b90 0x7f18fc0a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1910103340 0x7f191019f060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1910102980 0x7f1910197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 --2- 172.21.15.45:0/3772322823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1910101790 0x7f1910197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:34.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 >> 172.21.15.45:0/3772322823 conn(0x7f19100fcd20 msgr2=0x7f1910105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:34.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.873+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 shutdown_connections 2024-04-25T00:19:34.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:34.874+0000 7f1914f21640 1 -- 172.21.15.45:0/3772322823 wait complete. 2024-04-25T00:19:34.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 -- 172.21.15.45:0/3023531601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c102060 msgr2=0x7f0c0c104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 --2- 172.21.15.45:0/3023531601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c102060 0x7f0c0c104450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009900 tx=0x7f0bfc02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 -- 172.21.15.45:0/3023531601 shutdown_connections 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 --2- 172.21.15.45:0/3023531601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 --2- 172.21.15.45:0/3023531601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 0x7f0c0c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 --2- 172.21.15.45:0/3023531601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c102060 0x7f0c0c104450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 -- 172.21.15.45:0/3023531601 >> 172.21.15.45:0/3023531601 conn(0x7f0c0c0fba90 msgr2=0x7f0c0c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 -- 172.21.15.45:0/3023531601 shutdown_connections 2024-04-25T00:19:35.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.006+0000 7f0c11400640 1 -- 172.21.15.45:0/3023531601 wait complete. 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 Processor -- start 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 -- start start 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c102060 0x7f0c0c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 0x7f0c0c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10d250 con 0x7f0c0c1072c0 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10d0d0 con 0x7f0c0c102060 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c11400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10d3d0 con 0x7f0c0c104990 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57118/0 (socket says 172.21.15.45:57118) 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 -- 172.21.15.45:0/2162167533 learned_addr learned my addr 172.21.15.45:0/2162167533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0affd640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c102060 0x7f0c0c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c03fff640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 0x7f0c0c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 -- 172.21.15.45:0/2162167533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 msgr2=0x7f0c0c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 0x7f0c0c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 -- 172.21.15.45:0/2162167533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c102060 msgr2=0x7f0c0c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c102060 0x7f0c0c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.007+0000 7f0c0b7fe640 1 -- 172.21.15.45:0/2162167533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c19f540 con 0x7f0c0c1072c0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c0b7fe640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400eaa0 tx=0x7f0c0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400cdb0 con 0x7f0c0c1072c0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c19f830 con 0x7f0c0c1072c0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c04005020 con 0x7f0c0c1072c0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040116b0 con 0x7f0c0c1072c0 2024-04-25T00:19:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.008+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c19fd80 con 0x7f0c0c1072c0 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.009+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c0fd580 con 0x7f0c0c1072c0 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.010+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c04005560 con 0x7f0c0c1072c0 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.010+0000 7f0c08ff9640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 0x7f0bf80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.011+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f0c040cf240 con 0x7f0c0c1072c0 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.011+0000 7f0c0affd640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 0x7f0bf80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:35.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.011+0000 7f0c0affd640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 0x7f0bf80a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0314e0 tx=0x7f0bfc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.015+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0c04016030 con 0x7f0c0c1072c0 2024-04-25T00:19:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: pgmap v287: 97 pgs: 1 creating+activating, 15 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:19:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:34 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: pgmap v287: 97 pgs: 1 creating+activating, 15 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:19:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:34 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:19:35.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:35.201+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0c0c062340 con 0x7f0c0c1072c0 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: pgmap v287: 97 pgs: 1 creating+activating, 15 creating+peering, 1 active+clean, 80 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1257639609' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:19:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:19:35.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:34 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:active 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:35 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:active 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.287 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:35 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3772322823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] up:active 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:35 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:19:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.881+0000 7f0c08ff9640 1 -- 172.21.15.45:0/2162167533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f0c0409bb20 con 0x7f0c0c1072c0 2024-04-25T00:19:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2024-04-25T00:19:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 msgr2=0x7f0bf80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 0x7f0bf80a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0314e0 tx=0x7f0bfc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 msgr2=0x7f0c0c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400eaa0 tx=0x7f0c0400ef60 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 shutdown_connections 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0bf80a2b90 0x7f0bf80a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c0c1072c0 0x7f0c0c19eed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c0c104990 0x7f0c0c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 --2- 172.21.15.45:0/2162167533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c0c102060 0x7f0c0c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 >> 172.21.15.45:0/2162167533 conn(0x7f0c0c0fba90 msgr2=0x7f0c0c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 shutdown_connections 2024-04-25T00:19:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:36.885+0000 7f0c11400640 1 -- 172.21.15.45:0/2162167533 wait complete. 2024-04-25T00:19:36.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T00:19:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.014+0000 7fd89e4f5640 1 -- 172.21.15.45:0/4231698691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 msgr2=0x7fd8981045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.014+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/4231698691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd8981045f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 -- 172.21.15.45:0/4231698691 shutdown_connections 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/4231698691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898107460 0x7fd8981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/4231698691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898104b30 0x7fd898106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/4231698691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd8981045f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 -- 172.21.15.45:0/4231698691 >> 172.21.15.45:0/4231698691 conn(0x7fd8980fbbf0 msgr2=0x7fd8980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 -- 172.21.15.45:0/4231698691 shutdown_connections 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 -- 172.21.15.45:0/4231698691 wait complete. 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 Processor -- start 2024-04-25T00:19:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.015+0000 7fd89e4f5640 1 -- start start 2024-04-25T00:19:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898104b30 0x7fd898198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898107460 0x7fd89819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810d430 con 0x7fd898102200 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810d2b0 con 0x7fd898104b30 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd89810d5b0 con 0x7fd898107460 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57136/0 (socket says 172.21.15.45:57136) 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 -- 172.21.15.45:0/372060475 learned_addr learned my addr 172.21.15.45:0/372060475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd8977fe640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898104b30 0x7fd898198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89ca92640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898107460 0x7fd89819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 -- 172.21.15.45:0/372060475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898107460 msgr2=0x7fd89819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898107460 0x7fd89819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 -- 172.21.15.45:0/372060475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898104b30 msgr2=0x7fd898198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898104b30 0x7fd898198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 -- 172.21.15.45:0/372060475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89819f760 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd897fff640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd888009230 tx=0x7fd888031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88803f030 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89819f9f0 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd888003ad0 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.016+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89819ff30 con 0x7fd898102200 2024-04-25T00:19:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.018+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8980fd6e0 con 0x7fd898102200 2024-04-25T00:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.019+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd888042030 con 0x7fd898102200 2024-04-25T00:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.019+0000 7fd8957fa640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 0x7fd8800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.020+0000 7fd8977fe640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 0x7fd8800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.020+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fd8880efb90 con 0x7fd898102200 2024-04-25T00:19:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.020+0000 7fd8977fe640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 0x7fd8800a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd898199050 tx=0x7fd88c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:37.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.023+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd8880bc280 con 0x7fd898102200 2024-04-25T00:19:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:36 smithi045 ceph-mon[29194]: pgmap v290: 97 pgs: 1 creating+activating, 17 creating+peering, 30 active+clean, 49 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-25T00:19:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:36 smithi045 ceph-mon[29194]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:19:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:36 smithi045 ceph-mon[29194]: Cluster is now healthy 2024-04-25T00:19:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:36 smithi045 ceph-mon[29194]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:19:37.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.206+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fd89802d170 con 0x7fd898102200 2024-04-25T00:19:37.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:36 smithi122 ceph-mon[32959]: pgmap v290: 97 pgs: 1 creating+activating, 17 creating+peering, 30 active+clean, 49 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-25T00:19:37.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:36 smithi122 ceph-mon[32959]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:19:37.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:36 smithi122 ceph-mon[32959]: Cluster is now healthy 2024-04-25T00:19:37.208 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:36 smithi122 ceph-mon[32959]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:19:37.214 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:36 smithi071 ceph-mon[32667]: pgmap v290: 97 pgs: 1 creating+activating, 17 creating+peering, 30 active+clean, 49 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-25T00:19:37.214 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:36 smithi071 ceph-mon[32667]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:19:37.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:36 smithi071 ceph-mon[32667]: Cluster is now healthy 2024-04-25T00:19:37.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:36 smithi071 ceph-mon[32667]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:19:37.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.886+0000 7fd8957fa640 1 -- 172.21.15.45:0/372060475 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7fd8880c0580 con 0x7fd898102200 2024-04-25T00:19:37.887 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T00:19:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.890+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 msgr2=0x7fd8800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.890+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 0x7fd8800a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd898199050 tx=0x7fd88c00a250 comp rx=0 tx=0).stop 2024-04-25T00:19:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.890+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 msgr2=0x7fd898197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:37.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.890+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd888009230 tx=0x7fd888031b60 comp rx=0 tx=0).stop 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 shutdown_connections 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8800a2b90 0x7fd8800a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd898107460 0x7fd89819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd898104b30 0x7fd898198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 --2- 172.21.15.45:0/372060475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd898102200 0x7fd898197b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 >> 172.21.15.45:0/372060475 conn(0x7fd8980fbbf0 msgr2=0x7fd898102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 shutdown_connections 2024-04-25T00:19:37.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:37.891+0000 7fd89e4f5640 1 -- 172.21.15.45:0/372060475 wait complete. 2024-04-25T00:19:37.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/1478130985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 msgr2=0x7f0a68100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/1478130985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68100ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009900 tx=0x7f0a5802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/1478130985 shutdown_connections 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/1478130985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68102250 0x7f0a68108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/1478130985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68101890 0x7f0a68101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/1478130985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68100ac0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/1478130985 >> 172.21.15.45:0/1478130985 conn(0x7f0a680fbc30 msgr2=0x7f0a680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/1478130985 shutdown_connections 2024-04-25T00:19:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.026+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/1478130985 wait complete. 2024-04-25T00:19:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 Processor -- start 2024-04-25T00:19:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 -- start start 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 0x7f0a68196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68102250 0x7f0a6819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810c320 con 0x7f0a681006a0 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810c1a0 con 0x7f0a68101890 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6e6bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810c4a0 con 0x7f0a68102250 2024-04-25T00:19:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57154/0 (socket says 172.21.15.45:57154) 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 -- 172.21.15.45:0/3936996799 learned_addr learned my addr 172.21.15.45:0/3936996799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a6cc5a640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68102250 0x7f0a6819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a677fe640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 0x7f0a68196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 -- 172.21.15.45:0/3936996799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68102250 msgr2=0x7f0a6819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68102250 0x7f0a6819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 -- 172.21.15.45:0/3936996799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 msgr2=0x7f0a68196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 0x7f0a68196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a67fff640 1 -- 172.21.15.45:0/3936996799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6819e650 con 0x7f0a681006a0 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.027+0000 7f0a677fe640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 0x7f0a68196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a67fff640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a58031610 tx=0x7f0a58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5803f030 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a58003ad0 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6819e8e0 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58035070 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.028+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6819ee20 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.029+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a68107ab0 con 0x7f0a681006a0 2024-04-25T00:19:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.030+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a58042030 con 0x7f0a681006a0 2024-04-25T00:19:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.030+0000 7f0a657fa640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 0x7f0a500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.031+0000 7f0a677fe640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 0x7f0a500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.031+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f0a580ef970 con 0x7f0a681006a0 2024-04-25T00:19:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.031+0000 7f0a677fe640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 0x7f0a500a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a68197f40 tx=0x7f0a5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.035+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a580bc650 con 0x7f0a681006a0 2024-04-25T00:19:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:37 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:19:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:37 smithi045 ceph-mon[29194]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:19:38.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:37 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:19:38.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.262+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f0a6802d170 con 0x7f0a681006a0 2024-04-25T00:19:38.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:37 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:19:38.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:37 smithi122 ceph-mon[32959]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:19:38.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:37 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:19:38.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:37 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2162167533' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:19:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:37 smithi071 ceph-mon[32667]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:19:38.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:37 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:19:38.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:19:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:19:37.961+0000 7f7dfaee0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:19:38.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.894+0000 7f0a657fa640 1 -- 172.21.15.45:0/3936996799 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f0a580bbf50 con 0x7f0a681006a0 2024-04-25T00:19:38.895 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 4 to fsmap 2024-04-25T00:19:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 msgr2=0x7f0a500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 0x7f0a500a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a68197f40 tx=0x7f0a5c005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 msgr2=0x7f0a68196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:38.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a58031610 tx=0x7f0a58004270 comp rx=0 tx=0).stop 2024-04-25T00:19:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 shutdown_connections 2024-04-25T00:19:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0a500a2c10 0x7f0a500a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a68102250 0x7f0a6819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a68101890 0x7f0a68196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 --2- 172.21.15.45:0/3936996799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a681006a0 0x7f0a68196a30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 >> 172.21.15.45:0/3936996799 conn(0x7f0a680fbc30 msgr2=0x7f0a68104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 shutdown_connections 2024-04-25T00:19:38.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:38.897+0000 7f0a6e6bd640 1 -- 172.21.15.45:0/3936996799 wait complete. 2024-04-25T00:19:38.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:39.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- 172.21.15.45:0/448175495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f4103360 msgr2=0x7f96f4109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- 172.21.15.45:0/448175495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f4103360 0x7f96f4109bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009ca0 tx=0x7f96ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- 172.21.15.45:0/448175495 shutdown_connections 2024-04-25T00:19:39.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- 172.21.15.45:0/448175495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f4103360 0x7f96f4109bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- 172.21.15.45:0/448175495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f41029a0 0x7f96f4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- 172.21.15.45:0/448175495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41017b0 0x7f96f4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- 172.21.15.45:0/448175495 >> 172.21.15.45:0/448175495 conn(0x7f96f40fcd40 msgr2=0x7f96f40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- 172.21.15.45:0/448175495 shutdown_connections 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- 172.21.15.45:0/448175495 wait complete. 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 Processor -- start 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 -- start start 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.019+0000 7f96faf53640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41029a0 0x7f96f4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96faf53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f4103360 0x7f96f419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96faf53640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96f410d4d0 con 0x7f96f41017b0 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96faf53640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96f410d350 con 0x7f96f41029a0 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96faf53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96f410d650 con 0x7f96f4103360 2024-04-25T00:19:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57182/0 (socket says 172.21.15.45:57182) 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 -- 172.21.15.45:0/2470418028 learned_addr learned my addr 172.21.15.45:0/2470418028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f3fff640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41029a0 0x7f96f4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f94f0640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f4103360 0x7f96f419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 -- 172.21.15.45:0/2470418028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f4103360 msgr2=0x7f96f419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f4103360 0x7f96f419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 -- 172.21.15.45:0/2470418028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41029a0 msgr2=0x7f96f4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41029a0 0x7f96f4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 -- 172.21.15.45:0/2470418028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f419f700 con 0x7f96f41017b0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f8cef640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b450 tx=0x7f96e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8015020 con 0x7f96f41017b0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f419f9f0 con 0x7f96f41017b0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e8011d70 con 0x7f96f41017b0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.020+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e800f340 con 0x7f96f41017b0 2024-04-25T00:19:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.021+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f419ffc0 con 0x7f96f41017b0 2024-04-25T00:19:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.021+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f40689e0 con 0x7f96f41017b0 2024-04-25T00:19:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.023+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f96e8018020 con 0x7f96f41017b0 2024-04-25T00:19:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.023+0000 7f96f1ffb640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.023+0000 7f96f3fff640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.023+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f96e80ced40 con 0x7f96f41017b0 2024-04-25T00:19:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.024+0000 7f96f3fff640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 0x7f96e40a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96f4198ff0 tx=0x7f96e000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.028+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96f40689e0 con 0x7f96f41017b0 2024-04-25T00:19:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:38 smithi045 ceph-mon[29194]: pgmap v293: 129 pgs: 1 creating+activating, 13 creating+peering, 88 active+clean, 27 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-25T00:19:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:38 smithi045 ceph-mon[29194]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:38 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:19:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:38 smithi045 ceph-mon[29194]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:38 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:19:39.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.252+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96f4062340 con 0x7f96f41017b0 2024-04-25T00:19:39.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.253+0000 7f96f1ffb640 1 -- 172.21.15.45:0/2470418028 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f96e801f020 con 0x7f96f41017b0 2024-04-25T00:19:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:19:38.264424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-04-25T00:19:39.254 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 msgr2=0x7f96e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 0x7f96e40a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96f4198ff0 tx=0x7f96e000a250 comp rx=0 tx=0).stop 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 msgr2=0x7f96f4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b450 tx=0x7f96e800b910 comp rx=0 tx=0).stop 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 shutdown_connections 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f96e40a2b90 0x7f96e40a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f4103360 0x7f96f419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f41029a0 0x7f96f4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 --2- 172.21.15.45:0/2470418028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96f41017b0 0x7f96f4197ae0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 >> 172.21.15.45:0/2470418028 conn(0x7f96f40fcd40 msgr2=0x7f96f4105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 shutdown_connections 2024-04-25T00:19:39.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.255+0000 7f96faf53640 1 -- 172.21.15.45:0/2470418028 wait complete. 2024-04-25T00:19:39.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:38 smithi122 ceph-mon[32959]: pgmap v293: 129 pgs: 1 creating+activating, 13 creating+peering, 88 active+clean, 27 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-25T00:19:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:38 smithi122 ceph-mon[32959]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:38 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:19:39.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:38 smithi122 ceph-mon[32959]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:39.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:38 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- 172.21.15.45:0/3885515834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c20103360 msgr2=0x7f9c20109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 --2- 172.21.15.45:0/3885515834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c20103360 0x7f9c20109bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009ca0 tx=0x7f9c1802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- 172.21.15.45:0/3885515834 shutdown_connections 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 --2- 172.21.15.45:0/3885515834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c20103360 0x7f9c20109bf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 --2- 172.21.15.45:0/3885515834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c201029a0 0x7f9c20102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 --2- 172.21.15.45:0/3885515834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 0x7f9c20101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- 172.21.15.45:0/3885515834 >> 172.21.15.45:0/3885515834 conn(0x7f9c200fcd40 msgr2=0x7f9c200ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- 172.21.15.45:0/3885515834 shutdown_connections 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- 172.21.15.45:0/3885515834 wait complete. 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 Processor -- start 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.368+0000 7f9c282bc640 1 -- start start 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 0x7f9c200754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20103360 0x7f9c20074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2010d410 con 0x7f9c201029a0 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2010d290 con 0x7f9c20103360 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2010d590 con 0x7f9c201017b0 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57194/0 (socket says 172.21.15.45:57194) 2024-04-25T00:19:39.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 -- 172.21.15.45:0/2335216713 learned_addr learned my addr 172.21.15.45:0/2335216713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c26058640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 0x7f9c200754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 -- 172.21.15.45:0/2335216713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 msgr2=0x7f9c200754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 0x7f9c200754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 -- 172.21.15.45:0/2335216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20103360 msgr2=0x7f9c20074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c26859640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20103360 0x7f9c20074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20103360 0x7f9c20074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 -- 172.21.15.45:0/2335216713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c20074900 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c25857640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d950 tx=0x7f9c1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c017070 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c009c30 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c201a1b70 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00f6b0 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.369+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c201a20c0 con 0x7f9c201029a0 2024-04-25T00:19:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.370+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c20108ad0 con 0x7f9c201029a0 2024-04-25T00:19:39.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:38 smithi071 ceph-mon[32667]: pgmap v293: 129 pgs: 1 creating+activating, 13 creating+peering, 88 active+clean, 27 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-25T00:19:39.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:38 smithi071 ceph-mon[32667]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:19:39.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:38 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/372060475' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:19:39.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:38 smithi071 ceph-mon[32667]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:19:39.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:38 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:19:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.371+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c1c0109d0 con 0x7f9c201029a0 2024-04-25T00:19:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.372+0000 7f9c177fe640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 0x7f9c0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.372+0000 7f9c26058640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 0x7f9c0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.372+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9c1c0cefe0 con 0x7f9c201029a0 2024-04-25T00:19:39.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.373+0000 7f9c26058640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 0x7f9c0c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005fd0 tx=0x7f9c10005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.377+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c1c009de0 con 0x7f9c201029a0 2024-04-25T00:19:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.565+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c200623e0 con 0x7f9c201029a0 2024-04-25T00:19:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.567+0000 7f9c177fe640 1 -- 172.21.15.45:0/2335216713 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7f9c1c09bca0 con 0x7f9c201029a0 2024-04-25T00:19:39.568 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:39.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 msgr2=0x7f9c0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 0x7f9c0c0a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005fd0 tx=0x7f9c10005850 comp rx=0 tx=0).stop 2024-04-25T00:19:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 msgr2=0x7f9c20073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d950 tx=0x7f9c1c00de10 comp rx=0 tx=0).stop 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 shutdown_connections 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c0c0a2b90 0x7f9c0c0a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.569+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20103360 0x7f9c20074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.570+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c201029a0 0x7f9c20073b30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.570+0000 7f9c282bc640 1 --2- 172.21.15.45:0/2335216713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c201017b0 0x7f9c200754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.570+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 >> 172.21.15.45:0/2335216713 conn(0x7f9c200fcd40 msgr2=0x7f9c20105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.570+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 shutdown_connections 2024-04-25T00:19:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.570+0000 7f9c282bc640 1 -- 172.21.15.45:0/2335216713 wait complete. 2024-04-25T00:19:39.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.683+0000 7efe1fdfb640 1 -- 172.21.15.45:0/53812927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18102230 msgr2=0x7efe18108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.683+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/53812927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18102230 0x7efe18108ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe10009ca0 tx=0x7efe1002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- 172.21.15.45:0/53812927 shutdown_connections 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/53812927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18102230 0x7efe18108ac0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/53812927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/53812927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18100680 0x7efe18100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- 172.21.15.45:0/53812927 >> 172.21.15.45:0/53812927 conn(0x7efe180fbbf0 msgr2=0x7efe180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- 172.21.15.45:0/53812927 shutdown_connections 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- 172.21.15.45:0/53812927 wait complete. 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 Processor -- start 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- start start 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18102230 0x7efe1819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810c300 con 0x7efe18100680 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810c180 con 0x7efe18102230 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1fdfb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810c480 con 0x7efe18101870 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.684+0000 7efe1db97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57212/0 (socket says 172.21.15.45:57212) 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 -- 172.21.15.45:0/999907683 learned_addr learned my addr 172.21.15.45:0/999907683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1d396640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 -- 172.21.15.45:0/999907683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 msgr2=0x7efe18197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 -- 172.21.15.45:0/999907683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18102230 msgr2=0x7efe1819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1e398640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18102230 0x7efe1819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18102230 0x7efe1819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 -- 172.21.15.45:0/999907683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1819f660 con 0x7efe18100680 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1d396640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:39.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1db97640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c970 tx=0x7efe1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14004220 con 0x7efe18100680 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe140043c0 con 0x7efe18100680 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1819f950 con 0x7efe18100680 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1400f450 con 0x7efe18100680 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.685+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1819fef0 con 0x7efe18100680 2024-04-25T00:19:39.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.686+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe181a0180 con 0x7efe18100680 2024-04-25T00:19:39.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.687+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe140079b0 con 0x7efe18100680 2024-04-25T00:19:39.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.688+0000 7efe0effd640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 0x7efe040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:39.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.688+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efe140cf400 con 0x7efe18100680 2024-04-25T00:19:39.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.688+0000 7efe1d396640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 0x7efe040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:39.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.688+0000 7efe1d396640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 0x7efe040a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe080058c0 tx=0x7efe08005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:39.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.692+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe14014030 con 0x7efe18100680 2024-04-25T00:19:39.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.875+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe181a0350 con 0x7efe18100680 2024-04-25T00:19:39.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.877+0000 7efe0effd640 1 -- 172.21.15.45:0/999907683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7efe1409c1a0 con 0x7efe18100680 2024-04-25T00:19:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:39.878 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 msgr2=0x7efe040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 0x7efe040a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe080058c0 tx=0x7efe08005850 comp rx=0 tx=0).stop 2024-04-25T00:19:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 msgr2=0x7efe18197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:39.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c970 tx=0x7efe1400ce30 comp rx=0 tx=0).stop 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 shutdown_connections 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe040a2c10 0x7efe040a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe18102230 0x7efe1819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe18101870 0x7efe18197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 --2- 172.21.15.45:0/999907683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe18100680 0x7efe18197a40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 >> 172.21.15.45:0/999907683 conn(0x7efe180fbbf0 msgr2=0x7efe180fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 shutdown_connections 2024-04-25T00:19:39.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:39.879+0000 7efe1fdfb640 1 -- 172.21.15.45:0/999907683 wait complete. 2024-04-25T00:19:39.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:40.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1080113753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 msgr2=0x7f6e90101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1080113753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90101d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009900 tx=0x7f6e7802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1080113753 shutdown_connections 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1080113753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e90102250 0x7f6e90108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1080113753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90101d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1080113753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e901006a0 0x7f6e90100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1080113753 >> 172.21.15.45:0/1080113753 conn(0x7f6e900fbc30 msgr2=0x7f6e900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1080113753 shutdown_connections 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.008+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1080113753 wait complete. 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 Processor -- start 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 -- start start 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e901006a0 0x7f6e901969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 0x7f6e9019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010c300 con 0x7f6e90101890 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010c180 con 0x7f6e90102250 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e94ed4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9010c480 con 0x7f6e901006a0 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57236/0 (socket says 172.21.15.45:57236) 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 -- 172.21.15.45:0/1737785868 learned_addr learned my addr 172.21.15.45:0/1737785868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8ed9d640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 0x7f6e9019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8e59c640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e901006a0 0x7f6e901969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 -- 172.21.15.45:0/1737785868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e901006a0 msgr2=0x7f6e901969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e901006a0 0x7f6e901969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 -- 172.21.15.45:0/1737785868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 msgr2=0x7f6e9019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 0x7f6e9019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 -- 172.21.15.45:0/1737785868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9019e5c0 con 0x7f6e90101890 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8ed9d640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 0x7f6e9019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:40.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e8dd9b640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009230 tx=0x7f6e78004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.009+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7803f030 con 0x7f6e90101890 2024-04-25T00:19:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.010+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9019e850 con 0x7f6e90101890 2024-04-25T00:19:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.010+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e78003ce0 con 0x7f6e90101890 2024-04-25T00:19:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.010+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78035070 con 0x7f6e90101890 2024-04-25T00:19:40.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.010+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9019ed40 con 0x7f6e90101890 2024-04-25T00:19:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.011+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e90107a40 con 0x7f6e90101890 2024-04-25T00:19:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.012+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6e78042030 con 0x7f6e90101890 2024-04-25T00:19:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.012+0000 7f6e877fe640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 0x7f6e7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.012+0000 7f6e8e59c640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 0x7f6e7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.012+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6e780efd50 con 0x7f6e90101890 2024-04-25T00:19:40.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.013+0000 7f6e8e59c640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 0x7f6e7c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000a300 tx=0x7f6e8000a290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.017+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e780f3030 con 0x7f6e90101890 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2470418028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2335216713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:39 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/999907683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.239+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e9002d170 con 0x7f6e90101890 2024-04-25T00:19:40.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.240+0000 7f6e877fe640 1 -- 172.21.15.45:0/1737785868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f6e780bca70 con 0x7f6e90101890 2024-04-25T00:19:40.241 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:40.241 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:19:38.264424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-04-25T00:19:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 msgr2=0x7f6e7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 0x7f6e7c0a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000a300 tx=0x7f6e8000a290 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 msgr2=0x7f6e90196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009230 tx=0x7f6e78004360 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 shutdown_connections 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f6e7c0a2b90 0x7f6e7c0a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e90102250 0x7f6e9019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e90101890 0x7f6e90196ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 --2- 172.21.15.45:0/1737785868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e901006a0 0x7f6e901969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 >> 172.21.15.45:0/1737785868 conn(0x7f6e900fbc30 msgr2=0x7f6e90104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 shutdown_connections 2024-04-25T00:19:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.242+0000 7f6e94ed4640 1 -- 172.21.15.45:0/1737785868 wait complete. 2024-04-25T00:19:40.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2470418028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2335216713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:39 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/999907683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 -- 172.21.15.45:0/2437314603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34541006a0 msgr2=0x7f3454100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 --2- 172.21.15.45:0/2437314603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34541006a0 0x7f3454100ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3448009900 tx=0x7f344802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 -- 172.21.15.45:0/2437314603 shutdown_connections 2024-04-25T00:19:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 --2- 172.21.15.45:0/2437314603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3454102250 0x7f3454108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 --2- 172.21.15.45:0/2437314603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 0x7f3454101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 --2- 172.21.15.45:0/2437314603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34541006a0 0x7f3454100ac0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 -- 172.21.15.45:0/2437314603 >> 172.21.15.45:0/2437314603 conn(0x7f34540fbc30 msgr2=0x7f34540fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 -- 172.21.15.45:0/2437314603 shutdown_connections 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.355+0000 7f345c361640 1 -- 172.21.15.45:0/2437314603 wait complete. 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 Processor -- start 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 -- start start 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 0x7f3454197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 0x7f3454198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f345410d400 con 0x7f3454102250 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f345410d280 con 0x7f3454101890 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345c361640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f345410d580 con 0x7f34541006a0 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57264/0 (socket says 172.21.15.45:57264) 2024-04-25T00:19:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 -- 172.21.15.45:0/1194848448 learned_addr learned my addr 172.21.15.45:0/1194848448 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a0fd640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 0x7f3454197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f34598fc640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 0x7f3454198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 -- 172.21.15.45:0/1194848448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 msgr2=0x7f3454197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 0x7f3454197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 -- 172.21.15.45:0/1194848448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 msgr2=0x7f3454198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 0x7f3454198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a8fe640 1 -- 172.21.15.45:0/1194848448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345419f6f0 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.356+0000 7f345a0fd640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 0x7f3454197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f345a8fe640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f344c009fd0 tx=0x7f344c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c017070 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c003e50 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345419f9e0 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0106c0 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.357+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345419fee0 con 0x7f3454102250 2024-04-25T00:19:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.358+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3454107ae0 con 0x7f3454102250 2024-04-25T00:19:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.359+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f344c003970 con 0x7f3454102250 2024-04-25T00:19:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.359+0000 7f34477fe640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 0x7f34400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.360+0000 7f345a0fd640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 0x7f34400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.360+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f344c0cf120 con 0x7f3454102250 2024-04-25T00:19:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.360+0000 7f345a0fd640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 0x7f34400a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f3448032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.363+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f344c09bde0 con 0x7f3454102250 2024-04-25T00:19:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:19:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3936996799' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:19:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2470418028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2335216713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:39 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/999907683' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.550+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f345402d340 con 0x7f3454102250 2024-04-25T00:19:40.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.551+0000 7f34477fe640 1 -- 172.21.15.45:0/1194848448 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7f344c09b6e0 con 0x7f3454102250 2024-04-25T00:19:40.552 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:40.552 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 msgr2=0x7f34400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 0x7f34400a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3448006010 tx=0x7f3448032cf0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 msgr2=0x7f345419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f344c009fd0 tx=0x7f344c00b260 comp rx=0 tx=0).stop 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 shutdown_connections 2024-04-25T00:19:40.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34400a2b90 0x7f34400a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3454102250 0x7f345419f080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454101890 0x7f3454198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 --2- 172.21.15.45:0/1194848448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34541006a0 0x7f3454197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.553+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 >> 172.21.15.45:0/1194848448 conn(0x7f34540fbc30 msgr2=0x7f3454104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.554+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 shutdown_connections 2024-04-25T00:19:40.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.554+0000 7f345c361640 1 -- 172.21.15.45:0/1194848448 wait complete. 2024-04-25T00:19:40.566 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:19:33.221862+0000', 'modified': '2024-04-25T00:19:38.264424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/102345855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 102345855}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 102345855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24500, 'qdb_cluster': [24500]} 2024-04-25T00:19:40.566 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:19:40.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:19:40.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.668+0000 7fe074702640 1 -- 172.21.15.45:0/1973475367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 msgr2=0x7fe06c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.668+0000 7fe074702640 1 --2- 172.21.15.45:0/1973475367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c102e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009900 tx=0x7fe05c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:40.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- 172.21.15.45:0/1973475367 shutdown_connections 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- 172.21.15.45:0/1973475367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c103350 0x7fe06c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- 172.21.15.45:0/1973475367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c102e10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- 172.21.15.45:0/1973475367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c1017a0 0x7fe06c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- 172.21.15.45:0/1973475367 >> 172.21.15.45:0/1973475367 conn(0x7fe06c0fcd10 msgr2=0x7fe06c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- 172.21.15.45:0/1973475367 shutdown_connections 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- 172.21.15.45:0/1973475367 wait complete. 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 Processor -- start 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- start start 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 0x7fe06c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c103350 0x7fe06c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c10d4e0 con 0x7fe06c102990 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c10d360 con 0x7fe06c1017a0 2024-04-25T00:19:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe074702640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c10d660 con 0x7fe06c103350 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.669+0000 7fe071c9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57282/0 (socket says 172.21.15.45:57282) 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 -- 172.21.15.45:0/2179235912 learned_addr learned my addr 172.21.15.45:0/2179235912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe07249e640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 0x7fe06c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe072c9f640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c103350 0x7fe06c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 -- 172.21.15.45:0/2179235912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c103350 msgr2=0x7fe06c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c103350 0x7fe06c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 -- 172.21.15.45:0/2179235912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 msgr2=0x7fe06c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 0x7fe06c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 -- 172.21.15.45:0/2179235912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c19f770 con 0x7fe06c102990 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe07249e640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 0x7fe06c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe071c9d640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe05c031440 tx=0x7fe05c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe0637fe640 1 -- 172.21.15.45:0/2179235912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c03f030 con 0x7fe06c102990 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe0637fe640 1 -- 172.21.15.45:0/2179235912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c003ce0 con 0x7fe06c102990 2024-04-25T00:19:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c19fa00 con 0x7fe06c102990 2024-04-25T00:19:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe0637fe640 1 -- 172.21.15.45:0/2179235912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c035070 con 0x7fe06c102990 2024-04-25T00:19:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.670+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c19ff70 con 0x7fe06c102990 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.672+0000 7fe0637fe640 1 -- 172.21.15.45:0/2179235912 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe05c042030 con 0x7fe06c102990 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 Processor -- start 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 -- start start 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662f440 0x55995662f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559956634fa0 0x559956637430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559956592f00 con 0x55995662fe30 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55995653b750 con 0x559956634fa0 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe077405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55995658eec0 con 0x55995662f440 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe0637fe640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 0x7fe0580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57286/0 (socket says 172.21.15.45:57286) 2024-04-25T00:19:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe07249e640 1 -- 172.21.15.45:0/605830209 learned_addr learned my addr 172.21.15.45:0/605830209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe072c9f640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 0x7fe0580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe0637fe640 1 -- 172.21.15.45:0/2179235912 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe05c0ef650 con 0x7fe06c102990 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe071c9d640 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559956634fa0 0x559956637430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.673+0000 7fe072c9f640 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662f440 0x55995662f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 -- 172.21.15.45:0/605830209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662f440 msgr2=0x55995662f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662f440 0x55995662f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 -- 172.21.15.45:0/605830209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559956634fa0 msgr2=0x559956637430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559956634fa0 0x559956637430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 -- 172.21.15.45:0/605830209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599562a72d0 con 0x55995662fe30 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe07249e640 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe068007820 tx=0x7fe068007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8355c8f6dca48d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe072c9f640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 0x7fe0580a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x559956630690 tx=0x7fe0640093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe060ff9640 1 -- 172.21.15.45:0/605830209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06800d950 con 0x55995662fe30 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe060ff9640 1 -- 172.21.15.45:0/605830209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe068018070 con 0x55995662fe30 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.674+0000 7fe060ff9640 1 -- 172.21.15.45:0/605830209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680123e0 con 0x55995662fe30 2024-04-25T00:19:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- 172.21.15.45:0/605830209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 msgr2=0x559956634930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe068007820 tx=0x7fe068007ce0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- 172.21.15.45:0/605830209 shutdown_connections 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559956634fa0 0x559956637430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662fe30 0x559956634930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- 172.21.15.45:0/605830209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662f440 0x55995662f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- 172.21.15.45:0/605830209 >> 172.21.15.45:0/605830209 conn(0x55995662aab0 msgr2=0x55995662aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- 172.21.15.45:0/605830209 shutdown_connections 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- 172.21.15.45:0/605830209 wait complete. 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- start start 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 Processor -- start 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x559956639d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x559956640d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559956637f50 con 0x55995662f440 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55995654ecf0 con 0x55995662fe30 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5599566380d0 con 0x559956634fa0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.675+0000 7fe077405740 10 client.0 _ll_register_callbacks cb 0x55995654e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57300/0 (socket says 172.21.15.45:57300) 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 -- 172.21.15.45:0/3625929092 learned_addr learned my addr 172.21.15.45:0/3625929092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x559956639d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe072c9f640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x559956640d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 -- 172.21.15.45:0/3625929092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 msgr2=0x559956640d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x559956640d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 -- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 msgr2=0x559956639d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x559956639d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 -- 172.21.15.45:0/3625929092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599563716b0 con 0x55995662f440 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe07249e640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe068004700 tx=0x7fe0680137f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c7b3c11eb9217c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068024420 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe06801c3f0 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe068018070 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe06801c5f0 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe077405740 10 client.14766 fetch_fsmap learned FSMap version 17 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x559956643d70 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.676+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe068018070 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe068023860 con 0x55995662f440 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 10 client.14766 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 10 client.14766 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 20 client.14766 populate_metadata read hostname 'smithi045' 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559956646210 0x559956647680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 4 client.14766 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599565a0440 con 0x559956646210 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559956646210 0x559956647680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559956646210 0x559956647680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f5f0217afb49b6 server_cookie=f3908b92df385d77 in_seq=0 out_seq=0 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.677+0000 7fe055ffb640 10 client.14766 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe055ffb640 1 -- 172.21.15.45:0/3625929092 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599565a0440 con 0x559956646210 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe055ffb640 10 client.14766 handle_command_reply: tid=0 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 client.14766 shutdown 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 msgr2=0x559956639810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe068004700 tx=0x7fe0680137f0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 shutdown_connections 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x559956640d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x559956639d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559956646210 0x559956647680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 --2- 172.21.15.45:0/3625929092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956639810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe055ffb640 0 client.14766 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 >> 172.21.15.45:0/3625929092 conn(0x55995662aab0 msgr2=0x559956629ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 shutdown_connections 2024-04-25T00:19:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 1 -- 172.21.15.45:0/3625929092 wait complete. 2024-04-25T00:19:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.678+0000 7fe077405740 20 client.14766 trim_cache size 0 max 16384 2024-04-25T00:19:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 Processor -- start 2024-04-25T00:19:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 -- start start 2024-04-25T00:19:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 0x5599566306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 0x559956645170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5599566459c0 con 0x559956634fa0 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559956646090 con 0x55995662f440 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.696+0000 7fe077405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559956646760 con 0x55995662fe30 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57310/0 (socket says 172.21.15.45:57310) 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 -- 172.21.15.45:0/886795443 learned_addr learned my addr 172.21.15.45:0/886795443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe07249e640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 0x5599566306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 -- 172.21.15.45:0/886795443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 msgr2=0x559956645170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe071c9d640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 0x559956645170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 0x559956645170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 -- 172.21.15.45:0/886795443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 msgr2=0x5599566306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 0x5599566306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 -- 172.21.15.45:0/886795443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559956630210 con 0x559956634fa0 2024-04-25T00:19:40.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe07249e640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 0x5599566306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe071c9d640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 0x559956645170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe072c9f640 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe064014eb0 tx=0x7fe064013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518493dbe7fdabcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe057fff640 1 -- 172.21.15.45:0/886795443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064004b20 con 0x559956634fa0 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe057fff640 1 -- 172.21.15.45:0/886795443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064004cc0 con 0x559956634fa0 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.697+0000 7fe057fff640 1 -- 172.21.15.45:0/886795443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064029870 con 0x559956634fa0 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- 172.21.15.45:0/886795443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 msgr2=0x55995662e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe064014eb0 tx=0x7fe064013000 comp rx=0 tx=0).stop 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- 172.21.15.45:0/886795443 shutdown_connections 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559956634fa0 0x55995662e020 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55995662fe30 0x559956645170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- 172.21.15.45:0/886795443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662f440 0x5599566306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- 172.21.15.45:0/886795443 >> 172.21.15.45:0/886795443 conn(0x55995662aab0 msgr2=0x559956625690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- 172.21.15.45:0/886795443 shutdown_connections 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- 172.21.15.45:0/886795443 wait complete. 2024-04-25T00:19:40.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- start start 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 Processor -- start 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x5599566337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x5599566ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559956647500 con 0x55995662f440 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559956646e30 con 0x55995662fe30 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559956647bd0 con 0x559956634fa0 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe077405740 10 client.0 _ll_register_callbacks cb 0x7fe068013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe07249e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57322/0 (socket says 172.21.15.45:57322) 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.698+0000 7fe07249e640 1 -- 172.21.15.45:0/3593807235 learned_addr learned my addr 172.21.15.45:0/3593807235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe072c9f640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x5599566ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x5599566337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 -- 172.21.15.45:0/3593807235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 msgr2=0x5599566ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x5599566ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 -- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 msgr2=0x5599566337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x5599566337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 -- 172.21.15.45:0/3593807235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599566aede0 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe07249e640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe068004810 tx=0x7fe068023f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aabaaf76e1d653c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06800d530 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe06800dd60 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe064004cc0 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe068007120 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe077405740 10 client.14772 fetch_fsmap learned FSMap version 17 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5599566b0050 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064004cc0 con 0x55995662f440 2024-04-25T00:19:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe0680396e0 con 0x55995662f440 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.699+0000 7fe077405740 10 client.14772 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 10 client.14772 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 20 client.14772 populate_metadata read hostname 'smithi045' 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5599566b4d10 0x5599566b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 4 client.14772 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "status"}) -- 0x55995659a790 con 0x5599566b4d10 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5599566b4d10 0x5599566b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe071c9d640 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5599566b4d10 0x5599566b3a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0c98ffd9f299be server_cookie=855e3482c21c4991 in_seq=0 out_seq=0 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe055ffb640 10 client.14772 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe055ffb640 1 -- 172.21.15.45:0/3593807235 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55995659a790 con 0x5599566b4d10 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe055ffb640 10 client.14772 handle_command_reply: tid=0 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 client.14772 shutdown 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 msgr2=0x559956633260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe068004810 tx=0x7fe068023f10 comp rx=0 tx=0).stop 2024-04-25T00:19:40.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 shutdown_connections 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559956634fa0 0x5599566ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55995662fe30 0x5599566337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5599566b4d10 0x5599566b3a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 --2- 172.21.15.45:0/3593807235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55995662f440 0x559956633260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe055ffb640 0 client.14772 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.700+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 >> 172.21.15.45:0/3593807235 conn(0x55995662aab0 msgr2=0x5599566306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 shutdown_connections 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe077405740 1 -- 172.21.15.45:0/3593807235 wait complete. 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe077405740 20 client.14772 trim_cache size 0 max 16384 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:19:40.708 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24500, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 6.8328585549999996, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 89, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.153972444000001 2024-04-25T00:19:40.709 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 msgr2=0x7fe0580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 0x7fe0580a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x559956630690 tx=0x7fe0640093c0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 msgr2=0x7fe06c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe05c031440 tx=0x7fe05c032e20 comp rx=0 tx=0).stop 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 shutdown_connections 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe0580a2b90 0x7fe0580a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe06c103350 0x7fe06c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe06c102990 0x7fe06c198090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 --2- 172.21.15.45:0/2179235912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c1017a0 0x7fe06c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:40.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 >> 172.21.15.45:0/2179235912 conn(0x7fe06c0fcd10 msgr2=0x7fe06c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:40.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 shutdown_connections 2024-04-25T00:19:40.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:40.701+0000 7fe074702640 1 -- 172.21.15.45:0/2179235912 wait complete. 2024-04-25T00:19:40.712 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:19:40.713 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:40.713 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:19:40.961 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:19:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:40 smithi045 ceph-mon[29194]: pgmap v296: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-25T00:19:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:40 smithi045 ceph-mon[29194]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:19:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:40 smithi045 ceph-mon[29194]: Cluster is now healthy 2024-04-25T00:19:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:40 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1737785868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:40 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1194848448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:40 smithi122 ceph-mon[32959]: pgmap v296: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-25T00:19:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:40 smithi122 ceph-mon[32959]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:19:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:40 smithi122 ceph-mon[32959]: Cluster is now healthy 2024-04-25T00:19:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:40 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1737785868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:40 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1194848448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:40 smithi071 ceph-mon[32667]: pgmap v296: 129 pgs: 6 creating+peering, 103 active+clean, 20 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-04-25T00:19:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:40 smithi071 ceph-mon[32667]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:19:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:40 smithi071 ceph-mon[32667]: Cluster is now healthy 2024-04-25T00:19:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:40 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1737785868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:40 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1194848448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 -- 172.21.15.45:0/3135904003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 msgr2=0x7efe1410f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 --2- 172.21.15.45:0/3135904003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe1410f1b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00b0d0 tx=0x7efe0c00b3e0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 -- 172.21.15.45:0/3135904003 shutdown_connections 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 --2- 172.21.15.45:0/3135904003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe14100e30 0x7efe14101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 --2- 172.21.15.45:0/3135904003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe1410f1b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 --2- 172.21.15.45:0/3135904003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe141017e0 0x7efe14106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.564+0000 7efe1b97a700 1 -- 172.21.15.45:0/3135904003 >> 172.21.15.45:0/3135904003 conn(0x7efe14076450 msgr2=0x7efe14076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- 172.21.15.45:0/3135904003 shutdown_connections 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- 172.21.15.45:0/3135904003 wait complete. 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 Processor -- start 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- start start 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe14100e30 0x7efe1419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe141017e0 0x7efe1419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe14112550 con 0x7efe14107650 2024-04-25T00:19:42.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe141123d0 con 0x7efe14100e30 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.565+0000 7efe1b97a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe141126d0 con 0x7efe141017e0 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57328/0 (socket says 172.21.15.45:57328) 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 -- 172.21.15.45:0/628554465 learned_addr learned my addr 172.21.15.45:0/628554465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19716700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe14100e30 0x7efe1419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe18f15700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe141017e0 0x7efe1419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 -- 172.21.15.45:0/628554465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe141017e0 msgr2=0x7efe1419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe141017e0 0x7efe1419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 -- 172.21.15.45:0/628554465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe14100e30 msgr2=0x7efe1419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe14100e30 0x7efe1419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 -- 172.21.15.45:0/628554465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe14199030 con 0x7efe14107650 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe19f17700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006070 tx=0x7efe0c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c021070 con 0x7efe14107650 2024-04-25T00:19:42.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe141992c0 con 0x7efe14107650 2024-04-25T00:19:42.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c007c00 con 0x7efe14107650 2024-04-25T00:19:42.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0052a0 con 0x7efe14107650 2024-04-25T00:19:42.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.566+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141a5600 con 0x7efe14107650 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.567+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1406a480 con 0x7efe14107650 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.569+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe0c024070 con 0x7efe14107650 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.570+0000 7efe0a7fc700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 0x7efe000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.570+0000 7efe19716700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 0x7efe000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.570+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efe0c00f020 con 0x7efe14107650 2024-04-25T00:19:42.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.570+0000 7efe19716700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 0x7efe000a5160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe10005950 tx=0x7efe1000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.574+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe0c02b020 con 0x7efe14107650 2024-04-25T00:19:42.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.827+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe14199c80 con 0x7efe14107650 2024-04-25T00:19:42.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.827+0000 7efe0a7fc700 1 -- 172.21.15.45:0/628554465 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7efe0c007db0 con 0x7efe14107650 2024-04-25T00:19:42.828 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 msgr2=0x7efe000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 0x7efe000a5160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe10005950 tx=0x7efe1000a300 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 msgr2=0x7efe14198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006070 tx=0x7efe0c004e40 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 shutdown_connections 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe000a2ca0 0x7efe000a5160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe14107650 0x7efe14198750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe14100e30 0x7efe1419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.829+0000 7efe1b97a700 1 --2- 172.21.15.45:0/628554465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe141017e0 0x7efe1419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.830+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 >> 172.21.15.45:0/628554465 conn(0x7efe14076450 msgr2=0x7efe140feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.830+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 shutdown_connections 2024-04-25T00:19:42.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:42.830+0000 7efe1b97a700 1 -- 172.21.15.45:0/628554465 wait complete. 2024-04-25T00:19:43.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:42 smithi045 ceph-mon[29194]: pgmap v297: 129 pgs: 6 creating+peering, 121 active+clean, 2 unknown; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T00:19:43.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:42 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/628554465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:19:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:42 smithi122 ceph-mon[32959]: pgmap v297: 129 pgs: 6 creating+peering, 121 active+clean, 2 unknown; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T00:19:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:42 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/628554465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:19:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:42 smithi071 ceph-mon[32667]: pgmap v297: 129 pgs: 6 creating+peering, 121 active+clean, 2 unknown; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-25T00:19:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:42 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/628554465' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:19:43.596 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:19:43.596 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:19:43.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T00:19:43.596 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T00:19:43.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:43.597 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:19:43.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:19:43.597 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:19:43.597 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:19:43.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:43.597 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:19:43.615 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42209sec preferred_lft 42209sec 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T00:19:43.616 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T00:19:43.617 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:19:43.617 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:19:43.617 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-25T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-25T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:19:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:19:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:43.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:43.818 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:19:43.843 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T00:19:43.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:43.845 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T00:19:43.919 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T00:19:43.919 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:19:43.919 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T00:19:43.920 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T00:19:43.920 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:19:43.920 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:19:44.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:19:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:44.202 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T00:19:44.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:44.218 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-25T00:19:44.273 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:19:44.273 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:19:44.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:19:44.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T00:19:44.273 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:19:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:19:44.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:44.392 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T00:19:44.393 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:19:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:19:44.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:44.520 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:19:44.520 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:19:44.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T00:19:44.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:19:44.520 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:19:44.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:19:44.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:44.597 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T00:19:44.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:19:44.597 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:44.612 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:19:44.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:19:44.613 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:44.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-04-25T00:19:44.729 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-04-25T00:19:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:19:44.787 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/e3e1f1a0ae9d47a41eb7558367e574c697086102efe564893f82583ce468fe5d/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f98a1230dd57187e1e8c93668b926cf4a153127b9448a48b9d36c4b33a97a138/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/24aee8797610568b1423c3d83b413b84235fa81738f887dfc4e841fcab73aa32/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3fa21e6476b4a9966a4d5832bb0368345c9626d6ac64e5bf7fa2a973af02ea40/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/015e9803ad621f1681b22c9ab807aafa22a23dbd5b534401b082dc4a8b7dd1ac/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3092cdb0c48eb8e7e204179e295cf099e164142e74bcbde8ad82d87b1d4c77b/merged 2024-04-25T00:19:44.788 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/c5ecbe9ed7504af81f00348bd0f6e472aea3dd85d0ebe984fd7d23e4949bc121/merged 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/90cbb45734a7bedacc29482cc8c5302739a309229b1d50c773a23e47ac8d62f4/merged 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/62494f2c702d8003dc5b283a343b78cde9ac200a175f59231a39449c752872da/merged 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/13b8f1fcf66d0ceef241bb23c89918bbff88377a35ac00c11f76ff74ef56d604/merged 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:19:44.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:44.789 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-04-25T00:19:44.841 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:19:44.842 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T00:19:44.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-04-25T00:19:44.909 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-04-25T00:19:44.965 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:19:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/e3e1f1a0ae9d47a41eb7558367e574c697086102efe564893f82583ce468fe5d/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/f98a1230dd57187e1e8c93668b926cf4a153127b9448a48b9d36c4b33a97a138/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/24aee8797610568b1423c3d83b413b84235fa81738f887dfc4e841fcab73aa32/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3fa21e6476b4a9966a4d5832bb0368345c9626d6ac64e5bf7fa2a973af02ea40/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/015e9803ad621f1681b22c9ab807aafa22a23dbd5b534401b082dc4a8b7dd1ac/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d3092cdb0c48eb8e7e204179e295cf099e164142e74bcbde8ad82d87b1d4c77b/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/c5ecbe9ed7504af81f00348bd0f6e472aea3dd85d0ebe984fd7d23e4949bc121/merged 2024-04-25T00:19:44.967 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/90cbb45734a7bedacc29482cc8c5302739a309229b1d50c773a23e47ac8d62f4/merged 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/62494f2c702d8003dc5b283a343b78cde9ac200a175f59231a39449c752872da/merged 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/13b8f1fcf66d0ceef241bb23c89918bbff88377a35ac00c11f76ff74ef56d604/merged 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:19:44.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:44.968 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-04-25T00:19:44.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2024-04-25T00:19:44.996+0000 7f4266fff480 -1 init, newargv = 0x5645dcd080e0 newargc=15 2024-04-25T00:19:44.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[71093]: starting ceph client 2024-04-25T00:19:45.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[71093]: starting fuse 2024-04-25T00:19:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:115 2024-04-25T00:19:45.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-04-25T00:19:45.013 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2024-04-25T00:19:45.014 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T00:19:45.074 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:44 smithi045 ceph-mon[29194]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T00:19:45.133 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.71093.asok 2024-04-25T00:19:45.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:45.143 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71093.asok 2024-04-25T00:19:45.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:45.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71093.asok status 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2024-04-25T00:19:45.280 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "71093", 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 14784, 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 14784 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-04-25T00:19:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1370254493 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1370254493 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.14784 192.168.0.1:0/1370254493", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/1370254493", 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 17, 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 89, 2024-04-25T00:19:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:19:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-04-25T00:19:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-04-25T00:19:45.283 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:19:45.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:45.289 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-04-25T00:19:45.289 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:45.305 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-04-25T00:19:45.305 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:45.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:45.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:44 smithi122 ceph-mon[32959]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T00:19:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:44 smithi071 ceph-mon[32667]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T00:19:45.371 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T00:19:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:19:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2024-04-25T00:19:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T00:19:45.478 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:19:45.485 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:19:45.485 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:19:45.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:45.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 -- 172.21.15.45:0/647984892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990107470 msgr2=0x7fa9901098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 --2- 172.21.15.45:0/647984892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990107470 0x7fa9901098c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa98802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 -- 172.21.15.45:0/647984892 shutdown_connections 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 --2- 172.21.15.45:0/647984892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990107470 0x7fa9901098c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 --2- 172.21.15.45:0/647984892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990104b40 0x7fa990106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 --2- 172.21.15.45:0/647984892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102210 0x7fa990104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 -- 172.21.15.45:0/647984892 >> 172.21.15.45:0/647984892 conn(0x7fa9900fbc40 msgr2=0x7fa9900fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 -- 172.21.15.45:0/647984892 shutdown_connections 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.643+0000 7fa996466640 1 -- 172.21.15.45:0/647984892 wait complete. 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 Processor -- start 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 -- start start 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990102210 0x7fa990197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990107470 0x7fa99019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d410 con 0x7fa990104b40 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d290 con 0x7fa990102210 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa996466640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d590 con 0x7fa990107470 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57110/0 (socket says 172.21.15.45:57110) 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 -- 172.21.15.45:0/2784697694 learned_addr learned my addr 172.21.15.45:0/2784697694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98ffff640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990102210 0x7fa990197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa994a03640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990107470 0x7fa99019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 -- 172.21.15.45:0/2784697694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990107470 msgr2=0x7fa99019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990107470 0x7fa99019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 -- 172.21.15.45:0/2784697694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990102210 msgr2=0x7fa990197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990102210 0x7fa990197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.644+0000 7fa98f7fe640 1 -- 172.21.15.45:0/2784697694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99019f660 con 0x7fa990104b40 2024-04-25T00:19:45.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9c0 tx=0x7fa98000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000cd80 con 0x7fa990104b40 2024-04-25T00:19:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980005500 con 0x7fa990104b40 2024-04-25T00:19:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99019f950 con 0x7fa990104b40 2024-04-25T00:19:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980010690 con 0x7fa990104b40 2024-04-25T00:19:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.645+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99019fea0 con 0x7fa990104b40 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.646+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9900fd730 con 0x7fa990104b40 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.647+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa9800046c0 con 0x7fa990104b40 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.648+0000 7fa98d7fa640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 0x7fa97c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.648+0000 7fa98ffff640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 0x7fa97c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.648+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa9800cf620 con 0x7fa990104b40 2024-04-25T00:19:45.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.649+0000 7fa98ffff640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 0x7fa97c0a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bbe0 tx=0x7fa97800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:45.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.653+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa98009c340 con 0x7fa990104b40 2024-04-25T00:19:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.839+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa990062340 con 0x7fa990104b40 2024-04-25T00:19:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.841+0000 7fa98d7fa640 1 -- 172.21.15.45:0/2784697694 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7fa98009bc40 con 0x7fa990104b40 2024-04-25T00:19:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:45.842 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 msgr2=0x7fa97c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 0x7fa97c0a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97800bbe0 tx=0x7fa97800b390 comp rx=0 tx=0).stop 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 msgr2=0x7fa990197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9c0 tx=0x7fa98000ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 shutdown_connections 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa97c0a2b90 0x7fa97c0a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990107470 0x7fa99019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa990104b40 0x7fa990197f80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 --2- 172.21.15.45:0/2784697694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa990102210 0x7fa990197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 >> 172.21.15.45:0/2784697694 conn(0x7fa9900fbc40 msgr2=0x7fa990102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.843+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 shutdown_connections 2024-04-25T00:19:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.844+0000 7fa996466640 1 -- 172.21.15.45:0/2784697694 wait complete. 2024-04-25T00:19:45.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:19:45.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 -- 172.21.15.45:0/1666099600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 msgr2=0x7f357c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 --2- 172.21.15.45:0/1666099600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c104650 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3570009900 tx=0x7f357002a410 comp rx=0 tx=0).stop 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 -- 172.21.15.45:0/1666099600 shutdown_connections 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 --2- 172.21.15.45:0/1666099600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f357c1074c0 0x7f357c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 --2- 172.21.15.45:0/1666099600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 --2- 172.21.15.45:0/1666099600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c104650 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 -- 172.21.15.45:0/1666099600 >> 172.21.15.45:0/1666099600 conn(0x7f357c0fbc30 msgr2=0x7f357c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 -- 172.21.15.45:0/1666099600 shutdown_connections 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.963+0000 7f35830ee640 1 -- 172.21.15.45:0/1666099600 wait complete. 2024-04-25T00:19:45.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 Processor -- start 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 -- start start 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f357c1074c0 0x7f357c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d410 con 0x7f357c102260 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d290 con 0x7f357c1074c0 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f35830ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f357c10d590 con 0x7f357c104b90 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57122/0 (socket says 172.21.15.45:57122) 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 -- 172.21.15.45:0/1097284185 learned_addr learned my addr 172.21.15.45:0/1097284185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f357bfff640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 -- 172.21.15.45:0/1097284185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 msgr2=0x7f357c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 -- 172.21.15.45:0/1097284185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f357c1074c0 msgr2=0x7f357c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:45.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f357c1074c0 0x7f357c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 -- 172.21.15.45:0/1097284185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c19f700 con 0x7f357c102260 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f357bfff640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.964+0000 7f3580e8a640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3570009230 tx=0x7f3570004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.965+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357003f030 con 0x7f357c102260 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.965+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570003ce0 con 0x7f357c102260 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.965+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570035070 con 0x7f357c102260 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.965+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357c19f990 con 0x7f357c102260 2024-04-25T00:19:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.965+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357c19fed0 con 0x7f357c102260 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.966+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357c0fd720 con 0x7f357c102260 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.967+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3570042030 con 0x7f357c102260 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.968+0000 7f3579ffb640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.968+0000 7f357bfff640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.968+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f35700ef690 con 0x7f357c102260 2024-04-25T00:19:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.969+0000 7f357bfff640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 0x7f356c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f357c198ff0 tx=0x7f3568005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:45.973+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35700bc3b0 con 0x7f357c102260 2024-04-25T00:19:46.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2784697694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.158+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f357c062340 con 0x7f357c102260 2024-04-25T00:19:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.159+0000 7f3579ffb640 1 -- 172.21.15.45:0/1097284185 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f35700bbcb0 con 0x7f357c102260 2024-04-25T00:19:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 msgr2=0x7f356c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 0x7f356c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f357c198ff0 tx=0x7f3568005ec0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 msgr2=0x7f357c197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3570009230 tx=0x7f3570004430 comp rx=0 tx=0).stop 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 shutdown_connections 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f356c0a2c10 0x7f356c0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f357c1074c0 0x7f357c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f357c104b90 0x7f357c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 --2- 172.21.15.45:0/1097284185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f357c102260 0x7f357c197ae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 >> 172.21.15.45:0/1097284185 conn(0x7f357c0fbc30 msgr2=0x7f357c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 shutdown_connections 2024-04-25T00:19:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.161+0000 7f35830ee640 1 -- 172.21.15.45:0/1097284185 wait complete. 2024-04-25T00:19:46.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:19:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 -- 172.21.15.45:0/4239605569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4107460 msgr2=0x7f24d41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 --2- 172.21.15.45:0/4239605569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4107460 0x7f24d41098b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ca0 tx=0x7f24cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 -- 172.21.15.45:0/4239605569 shutdown_connections 2024-04-25T00:19:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 --2- 172.21.15.45:0/4239605569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4107460 0x7f24d41098b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 --2- 172.21.15.45:0/4239605569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4104b30 0x7f24d4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 --2- 172.21.15.45:0/4239605569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4102200 0x7f24d41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 -- 172.21.15.45:0/4239605569 >> 172.21.15.45:0/4239605569 conn(0x7f24d40fbbf0 msgr2=0x7f24d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 -- 172.21.15.45:0/4239605569 shutdown_connections 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.276+0000 7f24d888c640 1 -- 172.21.15.45:0/4239605569 wait complete. 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 Processor -- start 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 -- start start 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4104b30 0x7f24d4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4107460 0x7f24d419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d400 con 0x7f24d4102200 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d280 con 0x7f24d4104b30 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d888c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d580 con 0x7f24d4107460 2024-04-25T00:19:46.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57134/0 (socket says 172.21.15.45:57134) 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.277+0000 7f24d259c640 1 -- 172.21.15.45:0/4030479578 learned_addr learned my addr 172.21.15.45:0/4030479578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d2d9d640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4107460 0x7f24d419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d1d9b640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4104b30 0x7f24d4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 -- 172.21.15.45:0/4030479578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4107460 msgr2=0x7f24d419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4107460 0x7f24d419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 -- 172.21.15.45:0/4030479578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4104b30 msgr2=0x7f24d4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4104b30 0x7f24d4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 -- 172.21.15.45:0/4030479578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419f6c0 con 0x7f24d4102200 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d259c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d950 tx=0x7f24c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4017030 con 0x7f24d4102200 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c4004020 con 0x7f24d4102200 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419f9b0 con 0x7f24d4102200 2024-04-25T00:19:46.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c40047a0 con 0x7f24d4102200 2024-04-25T00:19:46.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.278+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d419ff50 con 0x7f24d4102200 2024-04-25T00:19:46.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.279+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d4068560 con 0x7f24d4102200 2024-04-25T00:19:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.280+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f24c40041d0 con 0x7f24d4102200 2024-04-25T00:19:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.281+0000 7f24c37fe640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 0x7f24bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.281+0000 7f24d1d9b640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 0x7f24bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.281+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f24c40ced40 con 0x7f24d4102200 2024-04-25T00:19:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.281+0000 7f24d1d9b640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 0x7f24bc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005d90 tx=0x7f24c8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.285+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f24c409ba60 con 0x7f24d4102200 2024-04-25T00:19:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2784697694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:46.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2784697694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.469+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24d4062370 con 0x7f24d4102200 2024-04-25T00:19:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.470+0000 7f24c37fe640 1 -- 172.21.15.45:0/4030479578 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f24c409b360 con 0x7f24d4102200 2024-04-25T00:19:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:19:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 msgr2=0x7f24bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 0x7f24bc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005d90 tx=0x7f24c8005d20 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 msgr2=0x7f24d4197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d950 tx=0x7f24c400de10 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 shutdown_connections 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f24bc0a2c10 0x7f24bc0a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24d4107460 0x7f24d419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d4104b30 0x7f24d4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 --2- 172.21.15.45:0/4030479578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24d4102200 0x7f24d4197aa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 >> 172.21.15.45:0/4030479578 conn(0x7f24d40fbbf0 msgr2=0x7f24d4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 shutdown_connections 2024-04-25T00:19:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.472+0000 7f24d888c640 1 -- 172.21.15.45:0/4030479578 wait complete. 2024-04-25T00:19:46.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:19:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- 172.21.15.45:0/1983757807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 msgr2=0x7feef0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- 172.21.15.45:0/1983757807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef0100aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feee8009ca0 tx=0x7feee802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- 172.21.15.45:0/1983757807 shutdown_connections 2024-04-25T00:19:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- 172.21.15.45:0/1983757807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0102230 0x7feef0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- 172.21.15.45:0/1983757807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101870 0x7feef0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- 172.21.15.45:0/1983757807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef0100aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- 172.21.15.45:0/1983757807 >> 172.21.15.45:0/1983757807 conn(0x7feef00fbbf0 msgr2=0x7feef00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- 172.21.15.45:0/1983757807 shutdown_connections 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- 172.21.15.45:0/1983757807 wait complete. 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 Processor -- start 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- start start 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0101870 0x7feef0109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0102230 0x7feef0109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feef010d420 con 0x7feef0100680 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feef010d2a0 con 0x7feef0101870 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.588+0000 7feef6e76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feef010d5a0 con 0x7feef0102230 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57156/0 (socket says 172.21.15.45:57156) 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 -- 172.21.15.45:0/2393799265 learned_addr learned my addr 172.21.15.45:0/2393799265 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feeeffff640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0101870 0x7feef0109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef5413640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0102230 0x7feef0109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 -- 172.21.15.45:0/2393799265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0102230 msgr2=0x7feef0109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0102230 0x7feef0109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 -- 172.21.15.45:0/2393799265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0101870 msgr2=0x7feef0109640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0101870 0x7feef0109640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 -- 172.21.15.45:0/2393799265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef010a410 con 0x7feef0100680 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef4c12640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feee8004af0 tx=0x7feee8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee802fb60 con 0x7feef0100680 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01a1c00 con 0x7feef0100680 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee802fd00 con 0x7feef0100680 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8046920 con 0x7feef0100680 2024-04-25T00:19:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.589+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a21a0 con 0x7feef0100680 2024-04-25T00:19:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.590+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0107a70 con 0x7feef0100680 2024-04-25T00:19:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.591+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feee8030c30 con 0x7feef0100680 2024-04-25T00:19:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.592+0000 7feeedffb640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 0x7feed80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.592+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7feee80f07b0 con 0x7feef0100680 2024-04-25T00:19:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.592+0000 7feeeffff640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 0x7feed80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.593+0000 7feeeffff640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 0x7feed80a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feef010ad70 tx=0x7feee400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.596+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feee80bd4d0 con 0x7feef0100680 2024-04-25T00:19:46.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.780+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feef0062340 con 0x7feef0100680 2024-04-25T00:19:46.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.781+0000 7feeedffb640 1 -- 172.21.15.45:0/2393799265 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7feee80bcdd0 con 0x7feef0100680 2024-04-25T00:19:46.782 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 msgr2=0x7feed80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 0x7feed80a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feef010ad70 tx=0x7feee400a2d0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 msgr2=0x7feef010b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feee8004af0 tx=0x7feee8031ee0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 shutdown_connections 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7feed80a2b90 0x7feed80a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0102230 0x7feef0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feef0101870 0x7feef0109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 --2- 172.21.15.45:0/2393799265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feef0100680 0x7feef010b000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 >> 172.21.15.45:0/2393799265 conn(0x7feef00fbbf0 msgr2=0x7feef0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 shutdown_connections 2024-04-25T00:19:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.783+0000 7feef6e76640 1 -- 172.21.15.45:0/2393799265 wait complete. 2024-04-25T00:19:46.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 -- 172.21.15.45:0/3750512222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 msgr2=0x7f6680102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3750512222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680102e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6670009900 tx=0x7f667002a410 comp rx=0 tx=0).stop 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 -- 172.21.15.45:0/3750512222 shutdown_connections 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3750512222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 0x7f6680109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3750512222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680102e10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3750512222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 0x7f6680101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 -- 172.21.15.45:0/3750512222 >> 172.21.15.45:0/3750512222 conn(0x7f66800fcd10 msgr2=0x7f66800ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 -- 172.21.15.45:0/3750512222 shutdown_connections 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.898+0000 7f6686ed2640 1 -- 172.21.15.45:0/3750512222 wait complete. 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 Processor -- start 2024-04-25T00:19:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 -- start start 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 0x7f66800754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 0x7f6680074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d420 con 0x7f6680102990 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d2a0 con 0x7f66801017a0 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6686ed2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f668010d5a0 con 0x7f6680103350 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57172/0 (socket says 172.21.15.45:57172) 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 -- 172.21.15.45:0/3364334117 learned_addr learned my addr 172.21.15.45:0/3364334117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f6684c6e640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 0x7f66800754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f668546f640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 0x7f6680074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 -- 172.21.15.45:0/3364334117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 msgr2=0x7f6680074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 0x7f6680074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 -- 172.21.15.45:0/3364334117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 msgr2=0x7f66800754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 0x7f66800754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 -- 172.21.15.45:0/3364334117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6680074900 con 0x7f6680102990 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.899+0000 7f667ffff640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6670009230 tx=0x7f6670031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.900+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667003f030 con 0x7f6680102990 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.900+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801a1b60 con 0x7f6680102990 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.900+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6670003ad0 con 0x7f6680102990 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.900+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670035070 con 0x7f6680102990 2024-04-25T00:19:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.900+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a2100 con 0x7f6680102990 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.901+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680108ae0 con 0x7f6680102990 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.902+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6670042030 con 0x7f6680102990 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.903+0000 7f667dffb640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.903+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f66700efc70 con 0x7f6680102990 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.903+0000 7f6684c6e640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.903+0000 7f6684c6e640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 0x7f666c0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66801027f0 tx=0x7f666800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:46.906+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66700bc930 con 0x7f6680102990 2024-04-25T00:19:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.095+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f668002d170 con 0x7f6680102990 2024-04-25T00:19:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.095+0000 7f667dffb640 1 -- 172.21.15.45:0/3364334117 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f66700bc230 con 0x7f6680102990 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 msgr2=0x7f666c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 0x7f666c0a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66801027f0 tx=0x7f666800b390 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 msgr2=0x7f6680073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6670009230 tx=0x7f6670031b60 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 shutdown_connections 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6680103350 0x7f6680074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6680102990 0x7f6680073b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 --2- 172.21.15.45:0/3364334117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66801017a0 0x7f66800754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 >> 172.21.15.45:0/3364334117 conn(0x7f66800fcd10 msgr2=0x7f6680105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 shutdown_connections 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.097+0000 7f6686ed2640 1 -- 172.21.15.45:0/3364334117 wait complete. 2024-04-25T00:19:47.107 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:46 smithi045 ceph-mon[29194]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-25T00:19:47.107 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1097284185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4030479578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2393799265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:19:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:46 smithi071 ceph-mon[32667]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-25T00:19:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1097284185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4030479578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2393799265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- 172.21.15.45:0/2411580638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102990 msgr2=0x7ff0d8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 --2- 172.21.15.45:0/2411580638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102990 0x7ff0d8102e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009900 tx=0x7ff0c802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- 172.21.15.45:0/2411580638 shutdown_connections 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 --2- 172.21.15.45:0/2411580638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8103350 0x7ff0d8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 --2- 172.21.15.45:0/2411580638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102990 0x7ff0d8102e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 --2- 172.21.15.45:0/2411580638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d81017a0 0x7ff0d8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- 172.21.15.45:0/2411580638 >> 172.21.15.45:0/2411580638 conn(0x7ff0d80fcd10 msgr2=0x7ff0d80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- 172.21.15.45:0/2411580638 shutdown_connections 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- 172.21.15.45:0/2411580638 wait complete. 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 Processor -- start 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.211+0000 7ff0dead4640 1 -- start start 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8102990 0x7ff0d8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d8103350 0x7ff0d819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810d400 con 0x7ff0d81017a0 2024-04-25T00:19:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810d280 con 0x7ff0d8102990 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d810d580 con 0x7ff0d8103350 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57202/0 (socket says 172.21.15.45:57202) 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8102990 0x7ff0d8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 -- 172.21.15.45:0/1376996490 learned_addr learned my addr 172.21.15.45:0/1376996490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dd071640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d8103350 0x7ff0d819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 -- 172.21.15.45:0/1376996490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d8103350 msgr2=0x7ff0d819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d8103350 0x7ff0d819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 -- 172.21.15.45:0/1376996490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8102990 msgr2=0x7ff0d8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8102990 0x7ff0d8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 -- 172.21.15.45:0/1376996490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d819f6d0 con 0x7ff0d81017a0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dc870640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c970 tx=0x7ff0d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4004220 con 0x7ff0d81017a0 2024-04-25T00:19:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d40043c0 con 0x7ff0d81017a0 2024-04-25T00:19:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400f450 con 0x7ff0d81017a0 2024-04-25T00:19:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.212+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d819f9c0 con 0x7ff0d81017a0 2024-04-25T00:19:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.213+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d819fee0 con 0x7ff0d81017a0 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.213+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d8108ac0 con 0x7ff0d81017a0 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.214+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff0d40079b0 con 0x7ff0d81017a0 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.215+0000 7ff0cdffb640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.215+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff0d40cef60 con 0x7ff0d81017a0 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.215+0000 7ff0cffff640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.216+0000 7ff0cffff640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 0x7ff0c00a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80314e0 tx=0x7ff0c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.220+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0d4014030 con 0x7ff0d81017a0 2024-04-25T00:19:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:46 smithi122 ceph-mon[32959]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-25T00:19:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1097284185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4030479578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2393799265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.420+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0d8062340 con 0x7ff0d81017a0 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.420+0000 7ff0cdffb640 1 -- 172.21.15.45:0/1376996490 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7ff0d409b630 con 0x7ff0d81017a0 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3254971976 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1950515838 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2906156154 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3376575480 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:47.421 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:47.422 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:47.422 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2002798989 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:47.422 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2389946368 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-04-25T00:19:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.422+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 msgr2=0x7ff0c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.422+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 0x7ff0c00a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80314e0 tx=0x7ff0c8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.422+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 msgr2=0x7ff0d8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.422+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c970 tx=0x7ff0d400ce30 comp rx=0 tx=0).stop 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 shutdown_connections 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d8103350 0x7ff0d819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0d8102990 0x7ff0d8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 --2- 172.21.15.45:0/1376996490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d81017a0 0x7ff0d8197ab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 >> 172.21.15.45:0/1376996490 conn(0x7ff0d80fcd10 msgr2=0x7ff0d8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 shutdown_connections 2024-04-25T00:19:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.423+0000 7ff0dead4640 1 -- 172.21.15.45:0/1376996490 wait complete. 2024-04-25T00:19:47.434 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:19:47.434 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:19:47.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 -- 172.21.15.45:0/1221338527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8107460 msgr2=0x7f73c81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 --2- 172.21.15.45:0/1221338527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8107460 0x7f73c81098b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009ca0 tx=0x7f73b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 -- 172.21.15.45:0/1221338527 shutdown_connections 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 --2- 172.21.15.45:0/1221338527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8107460 0x7f73c81098b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 --2- 172.21.15.45:0/1221338527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8104b30 0x7f73c8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 --2- 172.21.15.45:0/1221338527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8102200 0x7f73c81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.591+0000 7f73d006b640 1 -- 172.21.15.45:0/1221338527 >> 172.21.15.45:0/1221338527 conn(0x7f73c80fbbf0 msgr2=0x7f73c80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- 172.21.15.45:0/1221338527 shutdown_connections 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- 172.21.15.45:0/1221338527 wait complete. 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 Processor -- start 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- start start 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8104b30 0x7f73c81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8107460 0x7f73c81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d420 con 0x7f73c8102200 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d2a0 con 0x7f73c8107460 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73d006b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810d5a0 con 0x7f73c8104b30 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73cde07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73cde07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57222/0 (socket says 172.21.15.45:57222) 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73cde07640 1 -- 172.21.15.45:0/3585031445 learned_addr learned my addr 172.21.15.45:0/3585031445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.592+0000 7f73ce608640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8107460 0x7f73c81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cd606640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8104b30 0x7f73c81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 -- 172.21.15.45:0/3585031445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8104b30 msgr2=0x7f73c81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8104b30 0x7f73c81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 -- 172.21.15.45:0/3585031445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8107460 msgr2=0x7f73c81006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8107460 0x7f73c81006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 -- 172.21.15.45:0/3585031445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c8100f80 con 0x7f73c8102200 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73cde07640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73c000e9f0 tx=0x7f73c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c000cd80 con 0x7f73c8102200 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0005500 con 0x7f73c8102200 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c81a1b90 con 0x7f73c8102200 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0010690 con 0x7f73c8102200 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.593+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c81a20e0 con 0x7f73c8102200 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.594+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c80fd6e0 con 0x7f73c8102200 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.595+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73c00046c0 con 0x7f73c8102200 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.596+0000 7f73c6ffd640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 0x7f73b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.596+0000 7f73cd606640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 0x7f73b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.596+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f73c00cf710 con 0x7f73c8102200 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.596+0000 7f73cd606640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 0x7f73b00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73c81018e0 tx=0x7f73bc00b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.601+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73c009c430 con 0x7f73c8102200 2024-04-25T00:19:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.787+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73c8062340 con 0x7f73c8102200 2024-04-25T00:19:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.788+0000 7f73c6ffd640 1 -- 172.21.15.45:0/3585031445 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7f73c009bd30 con 0x7f73c8102200 2024-04-25T00:19:47.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:47.789 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 msgr2=0x7f73b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 0x7f73b00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73c81018e0 tx=0x7f73bc00b590 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 msgr2=0x7f73c8101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73c000e9f0 tx=0x7f73c000edf0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 shutdown_connections 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f73b00a2c10 0x7f73b00a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c8107460 0x7f73c81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73c8104b30 0x7f73c81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 --2- 172.21.15.45:0/3585031445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c8102200 0x7f73c8101b70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 >> 172.21.15.45:0/3585031445 conn(0x7f73c80fbbf0 msgr2=0x7f73c8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 shutdown_connections 2024-04-25T00:19:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.790+0000 7f73d006b640 1 -- 172.21.15.45:0/3585031445 wait complete. 2024-04-25T00:19:47.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:19:47.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:19:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/3169308725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0101670 msgr2=0x7f4dc0101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/3169308725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0101670 0x7f4dc0101a90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/3169308725 shutdown_connections 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/3169308725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0103220 0x7f4dc0109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/3169308725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0102860 0x7f4dc0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/3169308725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0101670 0x7f4dc0101a90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/3169308725 >> 172.21.15.45:0/3169308725 conn(0x7f4dc00fcbe0 msgr2=0x7f4dc00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/3169308725 shutdown_connections 2024-04-25T00:19:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.907+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/3169308725 wait complete. 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 Processor -- start 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 -- start start 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0101670 0x7f4dc01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0103220 0x7f4dc019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc010d2f0 con 0x7f4dc0102860 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc010d170 con 0x7f4dc0101670 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc5f9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc010d470 con 0x7f4dc0103220 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57238/0 (socket says 172.21.15.45:57238) 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 -- 172.21.15.45:0/4151141243 learned_addr learned my addr 172.21.15.45:0/4151141243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc4f9c640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0101670 0x7f4dc01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dc579d640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0103220 0x7f4dc019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 -- 172.21.15.45:0/4151141243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0103220 msgr2=0x7f4dc019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0103220 0x7f4dc019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 -- 172.21.15.45:0/4151141243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0101670 msgr2=0x7f4dc01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0101670 0x7f4dc01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 -- 172.21.15.45:0/4151141243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc019f5e0 con 0x7f4dc0102860 2024-04-25T00:19:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.908+0000 7f4dbffff640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4db000ea80 tx=0x7f4db000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.910+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000ce40 con 0x7f4dc0102860 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.910+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db0005020 con 0x7f4dc0102860 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.910+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc019f8d0 con 0x7f4dc0102860 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.910+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0011760 con 0x7f4dc0102860 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.910+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc019fe20 con 0x7f4dc0102860 2024-04-25T00:19:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.911+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc01089b0 con 0x7f4dc0102860 2024-04-25T00:19:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.912+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4db00051d0 con 0x7f4dc0102860 2024-04-25T00:19:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.913+0000 7f4dbdffb640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.913+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4db00cf830 con 0x7f4dc0102860 2024-04-25T00:19:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.913+0000 7f4dc4f9c640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.913+0000 7f4dc4f9c640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:47.919+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4db0016030 con 0x7f4dc0102860 2024-04-25T00:19:48.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.101+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4dc0062340 con 0x7f4dc0102860 2024-04-25T00:19:48.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.101+0000 7f4dbdffb640 1 -- 172.21.15.45:0/4151141243 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f4db009bea0 con 0x7f4dc0102860 2024-04-25T00:19:48.102 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:19:48.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.103+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 msgr2=0x7f4dac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.103+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.103+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 msgr2=0x7f4dc0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.103+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4db000ea80 tx=0x7f4db000ef40 comp rx=0 tx=0).stop 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 shutdown_connections 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc0103220 0x7f4dc019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dc0102860 0x7f4dc0197f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 --2- 172.21.15.45:0/4151141243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dc0101670 0x7f4dc01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 >> 172.21.15.45:0/4151141243 conn(0x7f4dc00fcbe0 msgr2=0x7f4dc0105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 shutdown_connections 2024-04-25T00:19:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.104+0000 7f4dc5f9e640 1 -- 172.21.15.45:0/4151141243 wait complete. 2024-04-25T00:19:48.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3364334117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:48.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1376996490' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:48.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3585031445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 -- 172.21.15.45:0/2562334643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c8102990 msgr2=0x7f89c8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 --2- 172.21.15.45:0/2562334643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c8102990 0x7f89c8102e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009900 tx=0x7f89bc02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 -- 172.21.15.45:0/2562334643 shutdown_connections 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 --2- 172.21.15.45:0/2562334643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 0x7f89c8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 --2- 172.21.15.45:0/2562334643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c8102990 0x7f89c8102e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 --2- 172.21.15.45:0/2562334643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c81017a0 0x7f89c8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 -- 172.21.15.45:0/2562334643 >> 172.21.15.45:0/2562334643 conn(0x7f89c80fcd10 msgr2=0x7f89c80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 -- 172.21.15.45:0/2562334643 shutdown_connections 2024-04-25T00:19:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.219+0000 7f89cd215640 1 -- 172.21.15.45:0/2562334643 wait complete. 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 Processor -- start 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 -- start start 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c8102990 0x7f89c8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 0x7f89c819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810d420 con 0x7f89c81017a0 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810d2a0 con 0x7f89c8103350 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.220+0000 7f89cd215640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89c810d5a0 con 0x7f89c8102990 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57256/0 (socket says 172.21.15.45:57256) 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 -- 172.21.15.45:0/464215640 learned_addr learned my addr 172.21.15.45:0/464215640 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c759e640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 0x7f89c819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c659c640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c8102990 0x7f89c8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 -- 172.21.15.45:0/464215640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c8102990 msgr2=0x7f89c8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c8102990 0x7f89c8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 -- 172.21.15.45:0/464215640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 msgr2=0x7f89c819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 0x7f89c819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 -- 172.21.15.45:0/464215640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c819f680 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89c6d9d640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89b800e9c0 tx=0x7f89b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b800cd80 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b80035f0 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c819f970 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8010690 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.221+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c819fe40 con 0x7f89c81017a0 2024-04-25T00:19:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.222+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c8108a60 con 0x7f89c81017a0 2024-04-25T00:19:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.223+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f89b80037a0 con 0x7f89c81017a0 2024-04-25T00:19:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.224+0000 7f89affff640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 0x7f89b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.224+0000 7f89c659c640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 0x7f89b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.224+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f89b80cf6e0 con 0x7f89c81017a0 2024-04-25T00:19:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.225+0000 7f89c659c640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 0x7f89b00a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.228+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f89b809c3a0 con 0x7f89c81017a0 2024-04-25T00:19:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3364334117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1376996490' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3585031445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3364334117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:48.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1376996490' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:48.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3585031445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.415+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f89c8062340 con 0x7f89c81017a0 2024-04-25T00:19:48.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.416+0000 7f89affff640 1 -- 172.21.15.45:0/464215640 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f89b809bca0 con 0x7f89c81017a0 2024-04-25T00:19:48.416 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:19:48.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 msgr2=0x7f89b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 0x7f89b00a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc0315d0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 msgr2=0x7f89c8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f89b800e9c0 tx=0x7f89b800ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 shutdown_connections 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f89b00a2b90 0x7f89b00a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89c8103350 0x7f89c819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c8102990 0x7f89c8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 --2- 172.21.15.45:0/464215640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89c81017a0 0x7f89c8197a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 >> 172.21.15.45:0/464215640 conn(0x7f89c80fcd10 msgr2=0x7f89c8105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 shutdown_connections 2024-04-25T00:19:48.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.418+0000 7f89cd215640 1 -- 172.21.15.45:0/464215640 wait complete. 2024-04-25T00:19:48.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- 172.21.15.45:0/275920143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c100680 msgr2=0x7efe3c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- 172.21.15.45:0/275920143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c100680 0x7efe3c100aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efe38009900 tx=0x7efe3802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- 172.21.15.45:0/275920143 shutdown_connections 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- 172.21.15.45:0/275920143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c102230 0x7efe3c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- 172.21.15.45:0/275920143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 0x7efe3c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- 172.21.15.45:0/275920143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c100680 0x7efe3c100aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- 172.21.15.45:0/275920143 >> 172.21.15.45:0/275920143 conn(0x7efe3c0fbbf0 msgr2=0x7efe3c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- 172.21.15.45:0/275920143 shutdown_connections 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- 172.21.15.45:0/275920143 wait complete. 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 Processor -- start 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 -- start start 2024-04-25T00:19:48.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 0x7efe3c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.534+0000 7efe4339b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 0x7efe3c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10c2d0 con 0x7efe3c102230 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10c150 con 0x7efe3c100680 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c10c450 con 0x7efe3c101870 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57280/0 (socket says 172.21.15.45:57280) 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 -- 172.21.15.45:0/4237286172 learned_addr learned my addr 172.21.15.45:0/4237286172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41137640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 0x7efe3c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe40936640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 0x7efe3c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 -- 172.21.15.45:0/4237286172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 msgr2=0x7efe3c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 0x7efe3c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 -- 172.21.15.45:0/4237286172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 msgr2=0x7efe3c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 0x7efe3c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 -- 172.21.15.45:0/4237286172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c19f6c0 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41137640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 0x7efe3c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe41938640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe3400cce0 tx=0x7efe3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3400f450 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3400fad0 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c19f9b0 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34018c10 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.535+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c19feb0 con 0x7efe3c102230 2024-04-25T00:19:48.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.536+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c107a70 con 0x7efe3c102230 2024-04-25T00:19:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.538+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe3400f5f0 con 0x7efe3c102230 2024-04-25T00:19:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.538+0000 7efe327fc640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 0x7efe280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.538+0000 7efe41137640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 0x7efe280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.538+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efe340cf970 con 0x7efe3c102230 2024-04-25T00:19:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.539+0000 7efe41137640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 0x7efe280a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe38006010 tx=0x7efe38032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.543+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe34014030 con 0x7efe3c102230 2024-04-25T00:19:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.727+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe3c062340 con 0x7efe3c102230 2024-04-25T00:19:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.728+0000 7efe327fc640 1 -- 172.21.15.45:0/4237286172 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7efe3409c690 con 0x7efe3c102230 2024-04-25T00:19:48.728 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:19:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 msgr2=0x7efe280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 0x7efe280a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe38006010 tx=0x7efe38032dd0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 msgr2=0x7efe3c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe3400cce0 tx=0x7efe3400a620 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 shutdown_connections 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7efe280a2b90 0x7efe280a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe3c102230 0x7efe3c19ee00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe3c101870 0x7efe3c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 --2- 172.21.15.45:0/4237286172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe3c100680 0x7efe3c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.729+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 >> 172.21.15.45:0/4237286172 conn(0x7efe3c0fbbf0 msgr2=0x7efe3c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.730+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 shutdown_connections 2024-04-25T00:19:48.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.730+0000 7efe4339b640 1 -- 172.21.15.45:0/4237286172 wait complete. 2024-04-25T00:19:48.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 -- 172.21.15.45:0/1243999336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9354103350 msgr2=0x7f9354109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1243999336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9354103350 0x7f9354109be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f934c009ca0 tx=0x7f934c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 -- 172.21.15.45:0/1243999336 shutdown_connections 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1243999336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9354103350 0x7f9354109be0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1243999336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 0x7f9354102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1243999336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93541017a0 0x7f9354101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 -- 172.21.15.45:0/1243999336 >> 172.21.15.45:0/1243999336 conn(0x7f93540fcd10 msgr2=0x7f93540ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 -- 172.21.15.45:0/1243999336 shutdown_connections 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.845+0000 7f935b2ba640 1 -- 172.21.15.45:0/1243999336 wait complete. 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 Processor -- start 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 -- start start 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 0x7f9354073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9354103350 0x7f9354074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f935410d420 con 0x7f93541017a0 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f935410d2a0 con 0x7f9354102990 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f935b2ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f935410d5a0 con 0x7f9354103350 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f9359056640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f9359056640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57302/0 (socket says 172.21.15.45:57302) 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f9359056640 1 -- 172.21.15.45:0/1668656770 learned_addr learned my addr 172.21.15.45:0/1668656770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.846+0000 7f9358855640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 0x7f9354073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359857640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9354103350 0x7f9354074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 -- 172.21.15.45:0/1668656770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9354103350 msgr2=0x7f9354074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9354103350 0x7f9354074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 -- 172.21.15.45:0/1668656770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 msgr2=0x7f9354073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 0x7f9354073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 -- 172.21.15.45:0/1668656770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9354074900 con 0x7f93541017a0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f9359056640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9c0 tx=0x7f935000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935000cd80 con 0x7f93541017a0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93500035f0 con 0x7f93541017a0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93541a1b80 con 0x7f93541017a0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350010690 con 0x7f93541017a0 2024-04-25T00:19:48.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.847+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541a2080 con 0x7f93541017a0 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.848+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9354108ac0 con 0x7f93541017a0 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.849+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f93500037a0 con 0x7f93541017a0 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.850+0000 7f934a7fc640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.850+0000 7f9358855640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.850+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f93500cf7c0 con 0x7f93541017a0 2024-04-25T00:19:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.850+0000 7f9358855640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 0x7f93400a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9354075260 tx=0x7f9344005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:48.854+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f935009c4e0 con 0x7f93541017a0 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.039+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f935402d3f0 con 0x7f93541017a0 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.039+0000 7f934a7fc640 1 -- 172.21.15.45:0/1668656770 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f935009bde0 con 0x7f93541017a0 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 msgr2=0x7f93400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 0x7f93400a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9354075260 tx=0x7f9344005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 msgr2=0x7f93540754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9c0 tx=0x7f935000ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 shutdown_connections 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f93400a2b90 0x7f93400a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9354103350 0x7f9354074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9354102990 0x7f9354073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 --2- 172.21.15.45:0/1668656770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93541017a0 0x7f93540754f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 >> 172.21.15.45:0/1668656770 conn(0x7f93540fcd10 msgr2=0x7f9354105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 shutdown_connections 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.041+0000 7f935b2ba640 1 -- 172.21.15.45:0/1668656770 wait complete. 2024-04-25T00:19:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:48 smithi045 ceph-mon[29194]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2024-04-25T00:19:49.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:48 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4151141243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:48 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/464215640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.052 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:48 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4237286172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/2727200482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4101870 msgr2=0x7f8fd4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/2727200482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4101870 0x7f8fd4101cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009900 tx=0x7f8fbc02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/2727200482 shutdown_connections 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/2727200482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/2727200482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4101870 0x7f8fd4101cf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/2727200482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4100680 0x7f8fd4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/2727200482 >> 172.21.15.45:0/2727200482 conn(0x7f8fd40fbbf0 msgr2=0x7f8fd40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/2727200482 shutdown_connections 2024-04-25T00:19:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.155+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/2727200482 wait complete. 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 Processor -- start 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 -- start start 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4101870 0x7f8fd4102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410c350 con 0x7f8fd4100680 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410c1d0 con 0x7f8fd4101870 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fdb1e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd410c4d0 con 0x7f8fd4102230 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fd8f7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fd8f7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57332/0 (socket says 172.21.15.45:57332) 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fd8f7f640 1 -- 172.21.15.45:0/1170819724 learned_addr learned my addr 172.21.15.45:0/1170819724 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fd9780640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.156+0000 7f8fd8f7f640 1 -- 172.21.15.45:0/1170819724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 msgr2=0x7f8fd4103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd8f7f640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd8f7f640 1 -- 172.21.15.45:0/1170819724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4101870 msgr2=0x7f8fd4102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd3fff640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4101870 0x7f8fd4102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd8f7f640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4101870 0x7f8fd4102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd8f7f640 1 -- 172.21.15.45:0/1170819724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd4103a20 con 0x7f8fd4100680 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd9780640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd8f7f640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00ebc0 tx=0x7f8fcc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc003b70 con 0x7f8fd4100680 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc003d10 con 0x7f8fd4100680 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd41a1bd0 con 0x7f8fd4100680 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc019980 con 0x7f8fd4100680 2024-04-25T00:19:49.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.157+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd41a2120 con 0x7f8fd4100680 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.158+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd41a2350 con 0x7f8fd4100680 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.159+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8fcc019b20 con 0x7f8fd4100680 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.160+0000 7f8fd1ffb640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 0x7f8fc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.160+0000 7f8fd3fff640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 0x7f8fc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.160+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8fcc0cfca0 con 0x7f8fd4100680 2024-04-25T00:19:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.160+0000 7f8fd3fff640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 0x7f8fc00a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc031610 tx=0x7f8fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.165+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fcc09c960 con 0x7f8fd4100680 2024-04-25T00:19:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:48 smithi122 ceph-mon[32959]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2024-04-25T00:19:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:48 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4151141243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:48 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/464215640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:48 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4237286172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.364+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8fd41a2520 con 0x7f8fd4100680 2024-04-25T00:19:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.364+0000 7f8fd1ffb640 1 -- 172.21.15.45:0/1170819724 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7f8fcc09c260 con 0x7f8fd4100680 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3254971976 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1950515838 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2906156154 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3376575480 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2002798989 2024-04-26T00:11:58.120353+0000 2024-04-25T00:19:49.365 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2389946368 2024-04-26T00:12:25.175459+0000 2024-04-25T00:19:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 msgr2=0x7f8fc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 0x7f8fc00a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc031610 tx=0x7f8fbc005fd0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 msgr2=0x7f8fd4104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00ebc0 tx=0x7f8fcc00c6a0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 shutdown_connections 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8fc00a2b90 0x7f8fc00a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fd4102230 0x7f8fd4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd4101870 0x7f8fd4102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 --2- 172.21.15.45:0/1170819724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fd4100680 0x7f8fd4104610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 >> 172.21.15.45:0/1170819724 conn(0x7f8fd40fbbf0 msgr2=0x7f8fd40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 shutdown_connections 2024-04-25T00:19:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:49.366+0000 7f8fdb1e3640 1 -- 172.21.15.45:0/1170819724 wait complete. 2024-04-25T00:19:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:48 smithi071 ceph-mon[32667]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 109 B/s wr, 0 op/s 2024-04-25T00:19:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:48 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4151141243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:48 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/464215640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:48 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4237286172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:19:49.378 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-04-25T00:19:49.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:49.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:19:49.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[71093]: fuse finished with error 0 and tester_r 0 2024-04-25T00:19:49.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:19:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:49 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1668656770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:49 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1170819724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:50.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:49 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1668656770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:50.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:49 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1170819724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:50.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:49 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1668656770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:19:50.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:49 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1170819724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:19:51.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:50 smithi122 ceph-mon[32959]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:50 smithi071 ceph-mon[32667]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:50 smithi045 ceph-mon[29194]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:19:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:52 smithi122 ceph-mon[32959]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:52 smithi071 ceph-mon[32667]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:52 smithi045 ceph-mon[29194]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:55.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:54 smithi122 ceph-mon[32959]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:54 smithi071 ceph-mon[32667]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:54 smithi045 ceph-mon[29194]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T00:19:55.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:19:55.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:55.505 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T00:19:55.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 -- 172.21.15.45:0/65437771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 msgr2=0x7f0f28108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/65437771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f28108ac0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009ca0 tx=0x7f0f2002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 -- 172.21.15.45:0/65437771 shutdown_connections 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/65437771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f28108ac0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/65437771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28101870 0x7f0f28101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/65437771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28100680 0x7f0f28100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 -- 172.21.15.45:0/65437771 >> 172.21.15.45:0/65437771 conn(0x7f0f280fbbf0 msgr2=0x7f0f280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.687+0000 7f0f2fd09640 1 -- 172.21.15.45:0/65437771 shutdown_connections 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 -- 172.21.15.45:0/65437771 wait complete. 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 Processor -- start 2024-04-25T00:19:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 -- start start 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 0x7f0f28197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28101870 0x7f0f28197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810d440 con 0x7f0f28102230 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810d2c0 con 0x7f0f28100680 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2fd09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810d5c0 con 0x7f0f28101870 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2e2a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2e2a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47604/0 (socket says 172.21.15.45:47604) 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.688+0000 7f0f2e2a6640 1 -- 172.21.15.45:0/2758602704 learned_addr learned my addr 172.21.15.45:0/2758602704 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2daa5640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 0x7f0f28197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2d2a4640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28101870 0x7f0f28197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 -- 172.21.15.45:0/2758602704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28101870 msgr2=0x7f0f28197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28101870 0x7f0f28197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 -- 172.21.15.45:0/2758602704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 msgr2=0x7f0f28197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 0x7f0f28197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 -- 172.21.15.45:0/2758602704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2819f620 con 0x7f0f28102230 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2daa5640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 0x7f0f28197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2e2a6640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0f200319f0 tx=0x7f0f2002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20003c70 con 0x7f0f28102230 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f20003e10 con 0x7f0f28102230 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2819f8b0 con 0x7f0f28102230 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002fd30 con 0x7f0f28102230 2024-04-25T00:19:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.689+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2819fe50 con 0x7f0f28102230 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.690+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f28107940 con 0x7f0f28102230 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.691+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f20004040 con 0x7f0f28102230 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.692+0000 7f0f1effd640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 0x7f0f140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.692+0000 7f0f2daa5640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 0x7f0f140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.692+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0f200f0890 con 0x7f0f28102230 2024-04-25T00:19:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.692+0000 7f0f2daa5640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 0x7f0f140a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f240058c0 tx=0x7f0f24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.697+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f200bd5b0 con 0x7f0f28102230 2024-04-25T00:19:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.884+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f28062340 con 0x7f0f28102230 2024-04-25T00:19:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.886+0000 7f0f1effd640 1 -- 172.21.15.45:0/2758602704 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7f0f200bceb0 con 0x7f0f28102230 2024-04-25T00:19:55.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:55.887 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 msgr2=0x7f0f140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 0x7f0f140a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0f240058c0 tx=0x7f0f24005850 comp rx=0 tx=0).stop 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 msgr2=0x7f0f2819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0f200319f0 tx=0x7f0f2002f450 comp rx=0 tx=0).stop 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 shutdown_connections 2024-04-25T00:19:55.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f0f140a2c10 0x7f0f140a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f28102230 0x7f0f2819efb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f28101870 0x7f0f28197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 --2- 172.21.15.45:0/2758602704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f28100680 0x7f0f28197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.888+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 >> 172.21.15.45:0/2758602704 conn(0x7f0f280fbbf0 msgr2=0x7f0f28104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.889+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 shutdown_connections 2024-04-25T00:19:55.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:55.889+0000 7f0f2fd09640 1 -- 172.21.15.45:0/2758602704 wait complete. 2024-04-25T00:19:55.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:19:56.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.002+0000 7ffa782da640 1 -- 172.21.15.45:0/2184828450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 msgr2=0x7ffa70102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.002+0000 7ffa782da640 1 --2- 172.21.15.45:0/2184828450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70102e20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:56.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- 172.21.15.45:0/2184828450 shutdown_connections 2024-04-25T00:19:56.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- 172.21.15.45:0/2184828450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa70103360 0x7ffa70109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- 172.21.15.45:0/2184828450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70102e20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- 172.21.15.45:0/2184828450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa701017b0 0x7ffa70101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- 172.21.15.45:0/2184828450 >> 172.21.15.45:0/2184828450 conn(0x7ffa700fcd40 msgr2=0x7ffa700ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- 172.21.15.45:0/2184828450 shutdown_connections 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- 172.21.15.45:0/2184828450 wait complete. 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 Processor -- start 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- start start 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa701017b0 0x7ffa70197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa70103360 0x7ffa7019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010d410 con 0x7ffa701029a0 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010d290 con 0x7ffa701017b0 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.003+0000 7ffa782da640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010d590 con 0x7ffa70103360 2024-04-25T00:19:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47624/0 (socket says 172.21.15.45:47624) 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 -- 172.21.15.45:0/425930438 learned_addr learned my addr 172.21.15.45:0/425930438 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa76877640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa70103360 0x7ffa7019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa76076640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa701017b0 0x7ffa70197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 -- 172.21.15.45:0/425930438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa70103360 msgr2=0x7ffa7019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa70103360 0x7ffa7019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 -- 172.21.15.45:0/425930438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa701017b0 msgr2=0x7ffa70197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa701017b0 0x7ffa70197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 -- 172.21.15.45:0/425930438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7019f720 con 0x7ffa701029a0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa75875640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009230 tx=0x7ffa64004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6403f030 con 0x7ffa701029a0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64003ce0 con 0x7ffa701029a0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7019f9b0 con 0x7ffa701029a0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64035070 con 0x7ffa701029a0 2024-04-25T00:19:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.004+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7019fea0 con 0x7ffa701029a0 2024-04-25T00:19:56.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.005+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa70108ad0 con 0x7ffa701029a0 2024-04-25T00:19:56.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.006+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa64042030 con 0x7ffa701029a0 2024-04-25T00:19:56.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.007+0000 7ffa6f7fe640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 0x7ffa580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.007+0000 7ffa76076640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 0x7ffa580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.007+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ffa640efc70 con 0x7ffa701029a0 2024-04-25T00:19:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.007+0000 7ffa76076640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 0x7ffa580a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa68005d00 tx=0x7ffa68005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.011+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa640bc990 con 0x7ffa701029a0 2024-04-25T00:19:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.196+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa70062340 con 0x7ffa701029a0 2024-04-25T00:19:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.198+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/425930438 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7ffa640bc290 con 0x7ffa701029a0 2024-04-25T00:19:56.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:19:56.199 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:19:56.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 msgr2=0x7ffa580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 0x7ffa580a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa68005d00 tx=0x7ffa68005c50 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 msgr2=0x7ffa70198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009230 tx=0x7ffa64004360 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 shutdown_connections 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffa580a2b90 0x7ffa580a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa70103360 0x7ffa7019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa701029a0 0x7ffa70198040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 --2- 172.21.15.45:0/425930438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa701017b0 0x7ffa70197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 >> 172.21.15.45:0/425930438 conn(0x7ffa700fcd40 msgr2=0x7ffa70105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 shutdown_connections 2024-04-25T00:19:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.200+0000 7ffa782da640 1 -- 172.21.15.45:0/425930438 wait complete. 2024-04-25T00:19:56.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:55 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2758602704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:56.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:19:56.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:55 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2758602704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 -- 172.21.15.45:0/3426230617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff244101890 msgr2=0x7ff244101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 --2- 172.21.15.45:0/3426230617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff244101890 0x7ff244101d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff234009900 tx=0x7ff23402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 -- 172.21.15.45:0/3426230617 shutdown_connections 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 --2- 172.21.15.45:0/3426230617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff244108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 --2- 172.21.15.45:0/3426230617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff244101890 0x7ff244101d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 --2- 172.21.15.45:0/3426230617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2441006a0 0x7ff244100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 -- 172.21.15.45:0/3426230617 >> 172.21.15.45:0/3426230617 conn(0x7ff2440fbc30 msgr2=0x7ff2440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 -- 172.21.15.45:0/3426230617 shutdown_connections 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.316+0000 7ff24bfce640 1 -- 172.21.15.45:0/3426230617 wait complete. 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 Processor -- start 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 -- start start 2024-04-25T00:19:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff244101890 0x7ff24419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff2441a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410c320 con 0x7ff2441006a0 2024-04-25T00:19:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410c1a0 con 0x7ff244102250 2024-04-25T00:19:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24bfce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410c4a0 con 0x7ff244101890 2024-04-25T00:19:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47644/0 (socket says 172.21.15.45:47644) 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 -- 172.21.15.45:0/4211460137 learned_addr learned my addr 172.21.15.45:0/4211460137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249569640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff244101890 0x7ff24419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24a56b640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff2441a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 -- 172.21.15.45:0/4211460137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff244101890 msgr2=0x7ff24419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff244101890 0x7ff24419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 -- 172.21.15.45:0/4211460137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 msgr2=0x7ff2441a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff2441a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 -- 172.21.15.45:0/4211460137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2441a1910 con 0x7ff2441006a0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff24a56b640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff2441a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.317+0000 7ff249d6a640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d950 tx=0x7ff24000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.318+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240017070 con 0x7ff2441006a0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.318+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240009c30 con 0x7ff2441006a0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.318+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2441a1c00 con 0x7ff2441006a0 2024-04-25T00:19:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.318+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24000f600 con 0x7ff2441006a0 2024-04-25T00:19:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.318+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a2150 con 0x7ff2441006a0 2024-04-25T00:19:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.319+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2441a2320 con 0x7ff2441006a0 2024-04-25T00:19:56.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.320+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff24001a070 con 0x7ff2441006a0 2024-04-25T00:19:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.320+0000 7ff23affd640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 0x7ff2300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.321+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff2400ced50 con 0x7ff2441006a0 2024-04-25T00:19:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.321+0000 7ff249569640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 0x7ff2300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.321+0000 7ff249569640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 0x7ff2300a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff234000c00 tx=0x7ff2340315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.324+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff240009de0 con 0x7ff2441006a0 2024-04-25T00:19:56.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:55 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2758602704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.546+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff2441a2710 con 0x7ff2441006a0 2024-04-25T00:19:56.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.935+0000 7ff23affd640 1 -- 172.21.15.45:0/4211460137 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff24009baf0 con 0x7ff2441006a0 2024-04-25T00:19:56.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 msgr2=0x7ff2300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 0x7ff2300a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff234000c00 tx=0x7ff2340315d0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 msgr2=0x7ff244199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d950 tx=0x7ff24000de10 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 shutdown_connections 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff2300a2c10 0x7ff2300a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff244102250 0x7ff2441a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff244101890 0x7ff24419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 --2- 172.21.15.45:0/4211460137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2441006a0 0x7ff244199d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 >> 172.21.15.45:0/4211460137 conn(0x7ff2440fbc30 msgr2=0x7ff2440fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 shutdown_connections 2024-04-25T00:19:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:56.937+0000 7ff24bfce640 1 -- 172.21.15.45:0/4211460137 wait complete. 2024-04-25T00:19:56.950 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T00:19:56.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 -- 172.21.15.45:0/1778565719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 msgr2=0x7f1818100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 --2- 172.21.15.45:0/1778565719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f1818100ac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1808009900 tx=0x7f180802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 -- 172.21.15.45:0/1778565719 shutdown_connections 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 --2- 172.21.15.45:0/1778565719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818102250 0x7f1818108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 --2- 172.21.15.45:0/1778565719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818101890 0x7f1818101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 --2- 172.21.15.45:0/1778565719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f1818100ac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 -- 172.21.15.45:0/1778565719 >> 172.21.15.45:0/1778565719 conn(0x7f18180fbc30 msgr2=0x7f18180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 -- 172.21.15.45:0/1778565719 shutdown_connections 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.058+0000 7f181f08c640 1 -- 172.21.15.45:0/1778565719 wait complete. 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 Processor -- start 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 -- start start 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 0x7f1818196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 0x7f181819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c2f0 con 0x7f18181006a0 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c170 con 0x7f1818102250 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181f08c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c470 con 0x7f1818101890 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181ce28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181ce28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47660/0 (socket says 172.21.15.45:47660) 2024-04-25T00:19:57.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.059+0000 7f181ce28640 1 -- 172.21.15.45:0/1323717514 learned_addr learned my addr 172.21.15.45:0/1323717514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181d629640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 0x7f181819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f1817fff640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 0x7f1818196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 -- 172.21.15.45:0/1323717514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 msgr2=0x7f1818196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 0x7f1818196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 -- 172.21.15.45:0/1323717514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 msgr2=0x7f181819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 0x7f181819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 -- 172.21.15.45:0/1323717514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181819e5e0 con 0x7f18181006a0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181d629640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 0x7f181819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f1817fff640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 0x7f1818196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181ce28640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1808031610 tx=0x7f18080043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180803f030 con 0x7f18181006a0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808003ce0 con 0x7f18181006a0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181819e870 con 0x7f18181006a0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808035070 con 0x7f18181006a0 2024-04-25T00:19:57.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.060+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181819edb0 con 0x7f18181006a0 2024-04-25T00:19:57.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.061+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1818107ae0 con 0x7f18181006a0 2024-04-25T00:19:57.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.062+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1808042030 con 0x7f18181006a0 2024-04-25T00:19:57.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.063+0000 7f1815ffb640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 0x7f18000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.063+0000 7f1817fff640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 0x7f18000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.063+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f18080ef8e0 con 0x7f18181006a0 2024-04-25T00:19:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.064+0000 7f1817fff640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 0x7f18000a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1818197ed0 tx=0x7f180c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.068+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f18080bc5a0 con 0x7f18181006a0 2024-04-25T00:19:57.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.291+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f181802d170 con 0x7f18181006a0 2024-04-25T00:19:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:56 smithi122 ceph-mon[32959]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/425930438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:19:57.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:56 smithi071 ceph-mon[32667]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/425930438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:19:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:56 smithi045 ceph-mon[29194]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/425930438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:19:57.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.944+0000 7f1815ffb640 1 -- 172.21.15.45:0/1323717514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f181802d170 con 0x7f18181006a0 2024-04-25T00:19:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 msgr2=0x7f18000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 0x7f18000a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1818197ed0 tx=0x7f180c005d30 comp rx=0 tx=0).stop 2024-04-25T00:19:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 msgr2=0x7f18181969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1808031610 tx=0x7f18080043b0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 shutdown_connections 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f18000a2c10 0x7f18000a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1818102250 0x7f181819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1818101890 0x7f1818196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 --2- 172.21.15.45:0/1323717514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18181006a0 0x7f18181969c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 >> 172.21.15.45:0/1323717514 conn(0x7f18180fbc30 msgr2=0x7f1818104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 shutdown_connections 2024-04-25T00:19:57.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:57.948+0000 7f181f08c640 1 -- 172.21.15.45:0/1323717514 wait complete. 2024-04-25T00:19:57.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T00:19:58.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- 172.21.15.45:0/3051491795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 msgr2=0x7fdddc101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3051491795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc101bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009900 tx=0x7fddd802a410 comp rx=0 tx=0).stop 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- 172.21.15.45:0/3051491795 shutdown_connections 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3051491795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc103350 0x7fdddc109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3051491795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc102990 0x7fdddc102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3051491795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc101bc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- 172.21.15.45:0/3051491795 >> 172.21.15.45:0/3051491795 conn(0x7fdddc0fcd10 msgr2=0x7fdddc0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- 172.21.15.45:0/3051491795 shutdown_connections 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- 172.21.15.45:0/3051491795 wait complete. 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 Processor -- start 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- start start 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc102990 0x7fdddc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103350 0x7fdddc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d410 con 0x7fdddc1017a0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d290 con 0x7fdddc102990 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.076+0000 7fdde37ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d590 con 0x7fdddc103350 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47668/0 (socket says 172.21.15.45:47668) 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 -- 172.21.15.45:0/3994185585 learned_addr learned my addr 172.21.15.45:0/3994185585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde1d8b640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103350 0x7fdddc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde0d89640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc102990 0x7fdddc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 -- 172.21.15.45:0/3994185585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103350 msgr2=0x7fdddc19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103350 0x7fdddc19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 -- 172.21.15.45:0/3994185585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc102990 msgr2=0x7fdddc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc102990 0x7fdddc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 -- 172.21.15.45:0/3994185585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc19f700 con 0x7fdddc1017a0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde158a640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddd8031610 tx=0x7fddd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd803f030 con 0x7fdddc1017a0 2024-04-25T00:19:58.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddd8003ad0 con 0x7fdddc1017a0 2024-04-25T00:19:58.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc19f990 con 0x7fdddc1017a0 2024-04-25T00:19:58.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8035070 con 0x7fdddc1017a0 2024-04-25T00:19:58.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.077+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc19fe80 con 0x7fdddc1017a0 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.079+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc108b60 con 0x7fdddc1017a0 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.080+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fddd8042030 con 0x7fdddc1017a0 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.080+0000 7fddd27fc640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 0x7fddc80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.080+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fddd80ef9b0 con 0x7fdddc1017a0 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.080+0000 7fdde0d89640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 0x7fddc80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:58.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.081+0000 7fdde0d89640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 0x7fddc80a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdddc198ff0 tx=0x7fddcc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:58.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.085+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fddd80bc6d0 con 0x7fdddc1017a0 2024-04-25T00:19:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:19:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:57 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:58.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:19:58.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.308+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdddc02d170 con 0x7fdddc1017a0 2024-04-25T00:19:58.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:19:58.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:57 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:58.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:19:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4211460137' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:19:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:57 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:58.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T00:19:58.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.957+0000 7fddd27fc640 1 -- 172.21.15.45:0/3994185585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fddd80bbfd0 con 0x7fdddc1017a0 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 msgr2=0x7fddc80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 0x7fddc80a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdddc198ff0 tx=0x7fddcc005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 msgr2=0x7fdddc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddd8031610 tx=0x7fddd8004270 comp rx=0 tx=0).stop 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 shutdown_connections 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fddc80a2b90 0x7fddc80a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103350 0x7fdddc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdddc102990 0x7fdddc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 --2- 172.21.15.45:0/3994185585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdddc1017a0 0x7fdddc197ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.961+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 >> 172.21.15.45:0/3994185585 conn(0x7fdddc0fcd10 msgr2=0x7fdddc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.962+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 shutdown_connections 2024-04-25T00:19:58.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:58.962+0000 7fdde37ee640 1 -- 172.21.15.45:0/3994185585 wait complete. 2024-04-25T00:19:58.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 -- 172.21.15.45:0/3711541083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78841006a0 msgr2=0x7f7884100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 --2- 172.21.15.45:0/3711541083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78841006a0 0x7f7884100ac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 -- 172.21.15.45:0/3711541083 shutdown_connections 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 --2- 172.21.15.45:0/3711541083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7884102250 0x7f7884108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 --2- 172.21.15.45:0/3711541083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884101890 0x7f7884101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 --2- 172.21.15.45:0/3711541083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78841006a0 0x7f7884100ac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 -- 172.21.15.45:0/3711541083 >> 172.21.15.45:0/3711541083 conn(0x7f78840fbc30 msgr2=0x7f78840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 -- 172.21.15.45:0/3711541083 shutdown_connections 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.091+0000 7f7888f90640 1 -- 172.21.15.45:0/3711541083 wait complete. 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 Processor -- start 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 -- start start 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 0x7f7884104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884102250 0x7f78841031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f788410c2f0 con 0x7f7884101890 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f788410c170 con 0x7f78841006a0 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7888f90640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f788410c470 con 0x7f7884102250 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f787bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f787bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47686/0 (socket says 172.21.15.45:47686) 2024-04-25T00:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f788259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 0x7f7884104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f787bfff640 1 -- 172.21.15.45:0/3277570570 learned_addr learned my addr 172.21.15.45:0/3277570570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.092+0000 7f7882d9d640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884102250 0x7f78841031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 -- 172.21.15.45:0/3277570570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884102250 msgr2=0x7f78841031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884102250 0x7f78841031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 -- 172.21.15.45:0/3277570570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 msgr2=0x7f7884104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 0x7f7884104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 -- 172.21.15.45:0/3277570570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7884103a40 con 0x7f7884101890 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f788259c640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 0x7f7884104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787bfff640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c920 tx=0x7f786c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c007c70 con 0x7f7884101890 2024-04-25T00:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841a1bd0 con 0x7f7884101890 2024-04-25T00:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786c007e10 con 0x7f7884101890 2024-04-25T00:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c018890 con 0x7f7884101890 2024-04-25T00:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.093+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78841a2120 con 0x7f7884101890 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.094+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78841a2350 con 0x7f7884101890 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.095+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f786c018a50 con 0x7f7884101890 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.096+0000 7f787b7fe640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 0x7f78700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.096+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f786c0cf670 con 0x7f7884101890 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.096+0000 7f788259c640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 0x7f78700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.097+0000 7f788259c640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 0x7f78700a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7874005ab0 tx=0x7f7874000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:59.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.099+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f786c014030 con 0x7f7884101890 2024-04-25T00:19:59.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:59.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:19:59.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:58 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:19:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:58 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:19:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.324+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f78841a2640 con 0x7f7884101890 2024-04-25T00:19:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:19:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:19:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1323717514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T00:19:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:19:59.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:19:59.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:19:59.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:19:59.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:58 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:19:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.966+0000 7f787b7fe640 1 -- 172.21.15.45:0/3277570570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f786c09c3c0 con 0x7f7884101890 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 msgr2=0x7f78700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 0x7f78700a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7874005ab0 tx=0x7f7874000bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 msgr2=0x7f7884102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f786c00c920 tx=0x7f786c00cde0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 shutdown_connections 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f78700a2b90 0x7f78700a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7884102250 0x7f78841031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7884101890 0x7f7884102c70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 --2- 172.21.15.45:0/3277570570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78841006a0 0x7f7884104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.970+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 >> 172.21.15.45:0/3277570570 conn(0x7f78840fbc30 msgr2=0x7f78840fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.971+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 shutdown_connections 2024-04-25T00:19:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:19:59.971+0000 7f7888f90640 1 -- 172.21.15.45:0/3277570570 wait complete. 2024-04-25T00:19:59.989 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T00:19:59.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T00:20:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 -- 172.21.15.45:0/104004957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c101870 msgr2=0x7ff60c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 --2- 172.21.15.45:0/104004957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c101870 0x7ff60c101cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009900 tx=0x7ff5fc02a410 comp rx=0 tx=0).stop 2024-04-25T00:20:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 -- 172.21.15.45:0/104004957 shutdown_connections 2024-04-25T00:20:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 --2- 172.21.15.45:0/104004957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c102230 0x7ff60c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 --2- 172.21.15.45:0/104004957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c101870 0x7ff60c101cf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 --2- 172.21.15.45:0/104004957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c100680 0x7ff60c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 -- 172.21.15.45:0/104004957 >> 172.21.15.45:0/104004957 conn(0x7ff60c0fbbf0 msgr2=0x7ff60c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 -- 172.21.15.45:0/104004957 shutdown_connections 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.102+0000 7ff614509640 1 -- 172.21.15.45:0/104004957 wait complete. 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 Processor -- start 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 -- start start 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c100680 0x7ff60c104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c101870 0x7ff60c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c10c300 con 0x7ff60c102230 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c10c180 con 0x7ff60c101870 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff614509640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c10c480 con 0x7ff60c100680 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff612aa6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:00.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff612aa6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47704/0 (socket says 172.21.15.45:47704) 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff612aa6640 1 -- 172.21.15.45:0/474586135 learned_addr learned my addr 172.21.15.45:0/474586135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.103+0000 7ff611aa4640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c101870 0x7ff60c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff6122a5640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c100680 0x7ff60c104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 -- 172.21.15.45:0/474586135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c100680 msgr2=0x7ff60c104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c100680 0x7ff60c104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 -- 172.21.15.45:0/474586135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c101870 msgr2=0x7ff60c102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c101870 0x7ff60c102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 -- 172.21.15.45:0/474586135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c103a20 con 0x7ff60c102230 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff612aa6640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff60400eaa0 tx=0x7ff60400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60400cdb0 con 0x7ff60c102230 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff604005020 con 0x7ff60c102230 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c1a1c20 con 0x7ff60c102230 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040116b0 con 0x7ff60c102230 2024-04-25T00:20:00.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.104+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1a2140 con 0x7ff60c102230 2024-04-25T00:20:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.105+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c068560 con 0x7ff60c102230 2024-04-25T00:20:00.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.107+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff604005560 con 0x7ff60c102230 2024-04-25T00:20:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.108+0000 7ff6037fe640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 0x7ff5f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.108+0000 7ff6122a5640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 0x7ff5f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.108+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff6040cf8a0 con 0x7ff60c102230 2024-04-25T00:20:00.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.109+0000 7ff6122a5640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 0x7ff5f80a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095f0 tx=0x7ff608005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:00.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.111+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff604016030 con 0x7ff60c102230 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:active 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:20:00.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:20:00.273 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:19:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:active 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:20:00.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:19:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:20:00.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.322+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff60c0fde10 con 0x7ff5f80a2b90 2024-04-25T00:20:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3994185585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:20:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] up:active 2024-04-25T00:20:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:20:00.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:20:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T00:20:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:20:00.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:19:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:20:00.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:20:00.327+0000 7efe7f4db700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:20:00.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:20:00.327+0000 7efe7f4db700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:20:00.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:20:00.327+0000 7efe7f4db700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:20:00.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:20:00.327+0000 7efe7f4db700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:20:00.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:20:00.327+0000 7efe7f4db700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:20:00.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.994+0000 7ff6037fe640 1 -- 172.21.15.45:0/474586135 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff60c0fde10 con 0x7ff5f80a2b90 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 msgr2=0x7ff5f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 0x7ff5f80a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6080095f0 tx=0x7ff608005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 msgr2=0x7ff60c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff60400eaa0 tx=0x7ff60400ef60 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 shutdown_connections 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff5f80a2b90 0x7ff5f80a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c102230 0x7ff60c103190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff60c101870 0x7ff60c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 --2- 172.21.15.45:0/474586135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff60c100680 0x7ff60c104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 >> 172.21.15.45:0/474586135 conn(0x7ff60c0fbbf0 msgr2=0x7ff60c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 shutdown_connections 2024-04-25T00:20:00.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:00.997+0000 7ff614509640 1 -- 172.21.15.45:0/474586135 wait complete. 2024-04-25T00:20:01.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T00:20:01.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- 172.21.15.45:0/452278960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 msgr2=0x7f264c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 --2- 172.21.15.45:0/452278960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c101cf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f264802a410 comp rx=0 tx=0).stop 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- 172.21.15.45:0/452278960 shutdown_connections 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 --2- 172.21.15.45:0/452278960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f264c102230 0x7f264c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 --2- 172.21.15.45:0/452278960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c101cf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 --2- 172.21.15.45:0/452278960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 0x7f264c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- 172.21.15.45:0/452278960 >> 172.21.15.45:0/452278960 conn(0x7f264c0fbbf0 msgr2=0x7f264c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- 172.21.15.45:0/452278960 shutdown_connections 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- 172.21.15.45:0/452278960 wait complete. 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 Processor -- start 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.118+0000 7f26543e9640 1 -- start start 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 0x7f264c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f264c102230 0x7f264c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c360 con 0x7f264c101870 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c1e0 con 0x7f264c100680 2024-04-25T00:20:01.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c4e0 con 0x7f264c102230 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47738/0 (socket says 172.21.15.45:47738) 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 -- 172.21.15.45:0/2995767185 learned_addr learned my addr 172.21.15.45:0/2995767185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2652185640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 0x7f264c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 -- 172.21.15.45:0/2995767185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f264c102230 msgr2=0x7f264c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f264c102230 0x7f264c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 -- 172.21.15.45:0/2995767185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 msgr2=0x7f264c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 0x7f264c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 -- 172.21.15.45:0/2995767185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c1a1990 con 0x7f264c101870 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f2651984640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2648009230 tx=0x7f2648031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264803f030 con 0x7f264c101870 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c1a1c20 con 0x7f264c101870 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648003ad0 con 0x7f264c101870 2024-04-25T00:20:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035070 con 0x7f264c101870 2024-04-25T00:20:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.119+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1a21e0 con 0x7f264c101870 2024-04-25T00:20:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.121+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c1a23b0 con 0x7f264c101870 2024-04-25T00:20:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.122+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2648042030 con 0x7f264c101870 2024-04-25T00:20:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.123+0000 7f26437fe640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.123+0000 7f2652185640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.123+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f26480ef830 con 0x7f264c101870 2024-04-25T00:20:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.123+0000 7f2652185640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 0x7f26380a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f264c1016d0 tx=0x7f263c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.127+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26480bc550 con 0x7f264c101870 2024-04-25T00:20:01.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:20:01.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:20:01.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:active 2024-04-25T00:20:01.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:20:01.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T00:20:01.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T00:20:01.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T00:20:01.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:00 smithi045 ceph-mon[29194]: fs cephfs has 4 MDS online, but wants 5 2024-04-25T00:20:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:20:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:20:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:active 2024-04-25T00:20:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:00 smithi122 ceph-mon[32959]: fs cephfs has 4 MDS online, but wants 5 2024-04-25T00:20:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.341+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f264c0fde30 con 0x7f26380a2b90 2024-04-25T00:20:01.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.367+0000 7f26437fe640 1 -- 172.21.15.45:0/2995767185 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f264c0fde30 con 0x7f26380a2b90 2024-04-25T00:20:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:20:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3277570570' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:20:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] up:active 2024-04-25T00:20:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T00:20:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T00:20:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T00:20:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T00:20:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: [WRN] MDS_UP_LESS_THAN_MAX: 1 filesystem is online with fewer MDS than max_mds 2024-04-25T00:20:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:00 smithi071 ceph-mon[32667]: fs cephfs has 4 MDS online, but wants 5 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 msgr2=0x7f26380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 0x7f26380a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f264c1016d0 tx=0x7f263c005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 msgr2=0x7f264c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2648009230 tx=0x7f2648031b60 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 shutdown_connections 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f264c102230 0x7f264c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f264c101870 0x7f264c19a2b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 --2- 172.21.15.45:0/2995767185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c100680 0x7f264c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 >> 172.21.15.45:0/2995767185 conn(0x7f264c0fbbf0 msgr2=0x7f264c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 shutdown_connections 2024-04-25T00:20:01.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.369+0000 7f26543e9640 1 -- 172.21.15.45:0/2995767185 wait complete. 2024-04-25T00:20:01.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T00:20:01.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- 172.21.15.45:0/1823389486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 msgr2=0x7f03dc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 --2- 172.21.15.45:0/1823389486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc101cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009900 tx=0x7f03cc02a410 comp rx=0 tx=0).stop 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- 172.21.15.45:0/1823389486 shutdown_connections 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 --2- 172.21.15.45:0/1823389486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 0x7f03dc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 --2- 172.21.15.45:0/1823389486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc101cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 --2- 172.21.15.45:0/1823389486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- 172.21.15.45:0/1823389486 >> 172.21.15.45:0/1823389486 conn(0x7f03dc0fbbf0 msgr2=0x7f03dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- 172.21.15.45:0/1823389486 shutdown_connections 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- 172.21.15.45:0/1823389486 wait complete. 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 Processor -- start 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.487+0000 7f03e2fee640 1 -- start start 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc1045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 0x7f03dc103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10c3b0 con 0x7f03dc101870 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10c230 con 0x7f03dc100680 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10c530 con 0x7f03dc102230 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e0d8a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc1045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47758/0 (socket says 172.21.15.45:47758) 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 -- 172.21.15.45:0/3842385254 learned_addr learned my addr 172.21.15.45:0/3842385254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e158b640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 0x7f03dc103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 -- 172.21.15.45:0/3842385254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 msgr2=0x7f03dc103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 0x7f03dc103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 -- 172.21.15.45:0/3842385254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 msgr2=0x7f03dc1045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc1045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 -- 172.21.15.45:0/3842385254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1039b0 con 0x7f03dc101870 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e0d8a640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d3fff640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03cc031440 tx=0x7f03cc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc03f030 con 0x7f03dc101870 2024-04-25T00:20:01.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03cc003ce0 con 0x7f03dc101870 2024-04-25T00:20:01.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc035070 con 0x7f03dc101870 2024-04-25T00:20:01.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.488+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc1a1c60 con 0x7f03dc101870 2024-04-25T00:20:01.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.489+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1a21a0 con 0x7f03dc101870 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.490+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc1a2490 con 0x7f03dc101870 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.491+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f03cc042030 con 0x7f03dc101870 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.491+0000 7f03d1ffb640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 0x7f03c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.492+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f03cc0ef7e0 con 0x7f03dc101870 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.492+0000 7f03e0d8a640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 0x7f03c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.492+0000 7f03e0d8a640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 0x7f03c40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f03dc1016d0 tx=0x7f03d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.496+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f03cc0bc500 con 0x7f03dc101870 2024-04-25T00:20:01.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.711+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f03dc0fddd0 con 0x7f03c40a2c10 2024-04-25T00:20:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.716+0000 7f03d1ffb640 1 -- 172.21.15.45:0/3842385254 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f03dc0fddd0 con 0x7f03c40a2c10 2024-04-25T00:20:01.717 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/3dc44924-319d-4f79-87cf-8b3049586a3f 2024-04-25T00:20:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 msgr2=0x7f03c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 0x7f03c40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f03dc1016d0 tx=0x7f03d8005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 msgr2=0x7f03dc102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03cc031440 tx=0x7f03cc032ce0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 shutdown_connections 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f03c40a2c10 0x7f03c40a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03dc102230 0x7f03dc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03dc101870 0x7f03dc102c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 --2- 172.21.15.45:0/3842385254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03dc100680 0x7f03dc1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 >> 172.21.15.45:0/3842385254 conn(0x7f03dc0fbbf0 msgr2=0x7f03dc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 shutdown_connections 2024-04-25T00:20:01.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.718+0000 7f03e2fee640 1 -- 172.21.15.45:0/3842385254 wait complete. 2024-04-25T00:20:01.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 -- 172.21.15.45:0/1125142260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68301074c0 msgr2=0x7f68301098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 --2- 172.21.15.45:0/1125142260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68301074c0 0x7f68301098d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6828009be0 tx=0x7f682802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 -- 172.21.15.45:0/1125142260 shutdown_connections 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 --2- 172.21.15.45:0/1125142260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68301074c0 0x7f68301098d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 --2- 172.21.15.45:0/1125142260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830104b90 0x7f6830106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 --2- 172.21.15.45:0/1125142260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6830102260 0x7f6830104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 -- 172.21.15.45:0/1125142260 >> 172.21.15.45:0/1125142260 conn(0x7f68300fbc30 msgr2=0x7f68300fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 -- 172.21.15.45:0/1125142260 shutdown_connections 2024-04-25T00:20:01.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.840+0000 7f6836a07640 1 -- 172.21.15.45:0/1125142260 wait complete. 2024-04-25T00:20:01.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 Processor -- start 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 -- start start 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 0x7f68301a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d440 con 0x7f6830104b90 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d2c0 con 0x7f68301074c0 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6836a07640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d5c0 con 0x7f6830102260 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f682f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f682ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f6834fa4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 0x7f68301a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f682f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47774/0 (socket says 172.21.15.45:47774) 2024-04-25T00:20:01.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.841+0000 7f682ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:57726/0 (socket says 172.21.15.45:57726) 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f682f7fe640 1 -- 172.21.15.45:0/1053018937 learned_addr learned my addr 172.21.15.45:0/1053018937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 -- 172.21.15.45:0/1053018937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 msgr2=0x7f6830199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 -- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 msgr2=0x7f683019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 -- 172.21.15.45:0/1053018937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68301a1920 con 0x7f68301074c0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f682ffff640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f682f7fe640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6834fa4640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 0x7f68301a1280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6828034040 tx=0x7f68280305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828041020 con 0x7f68301074c0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68301a1bb0 con 0x7f68301074c0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.842+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a20c0 con 0x7f68301074c0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.843+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828030720 con 0x7f68301074c0 2024-04-25T00:20:01.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.843+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828036040 con 0x7f68301074c0 2024-04-25T00:20:01.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.844+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6830108fb0 con 0x7f68301074c0 2024-04-25T00:20:01.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.845+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6828044020 con 0x7f68301074c0 2024-04-25T00:20:01.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.845+0000 7f682d7fa640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 0x7f68180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:01.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.845+0000 7f682ffff640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 0x7f68180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:01.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.846+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f68280efae0 con 0x7f68301074c0 2024-04-25T00:20:01.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.846+0000 7f682ffff640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 0x7f68180a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68200096c0 tx=0x7f6820009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:01.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.850+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f682804b030 con 0x7f68301074c0 2024-04-25T00:20:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:01.991+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6828003c70 con 0x7f68301074c0 2024-04-25T00:20:02.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.070+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6830061c10 con 0x7f68180a2c10 2024-04-25T00:20:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.095+0000 7f682d7fa640 1 -- 172.21.15.45:0/1053018937 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6830061c10 con 0x7f68180a2c10 2024-04-25T00:20:02.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 msgr2=0x7f68180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 0x7f68180a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68200096c0 tx=0x7f6820009260 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 msgr2=0x7f68301a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 0x7f68301a1280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6828034040 tx=0x7f68280305b0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 shutdown_connections 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f68180a2c10 0x7f68180a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68301074c0 0x7f68301a1280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6830104b90 0x7f683019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 --2- 172.21.15.45:0/1053018937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6830102260 0x7f6830199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 >> 172.21.15.45:0/1053018937 conn(0x7f68300fbc30 msgr2=0x7f6830101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 shutdown_connections 2024-04-25T00:20:02.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.097+0000 7f6836a07640 1 -- 172.21.15.45:0/1053018937 wait complete. 2024-04-25T00:20:02.107 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:active 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: Cluster is now healthy 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:active 2024-04-25T00:20:02.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:01 smithi045 ceph-mon[29194]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:20:02.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T00:20:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 -- 172.21.15.45:0/881392376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c102200 msgr2=0x7fef4c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 --2- 172.21.15.45:0/881392376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c102200 0x7fef4c1045f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009ca0 tx=0x7fef3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 -- 172.21.15.45:0/881392376 shutdown_connections 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 --2- 172.21.15.45:0/881392376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c107460 0x7fef4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 --2- 172.21.15.45:0/881392376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c104b30 0x7fef4c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 --2- 172.21.15.45:0/881392376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c102200 0x7fef4c1045f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 -- 172.21.15.45:0/881392376 >> 172.21.15.45:0/881392376 conn(0x7fef4c0fbbf0 msgr2=0x7fef4c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 -- 172.21.15.45:0/881392376 shutdown_connections 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.215+0000 7fef5139e640 1 -- 172.21.15.45:0/881392376 wait complete. 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 Processor -- start 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 -- start start 2024-04-25T00:20:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c102200 0x7fef4c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 0x7fef4c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10d430 con 0x7fef4c104b30 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10d2b0 con 0x7fef4c102200 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef5139e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c10d5b0 con 0x7fef4c107460 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef4a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef4a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47794/0 (socket says 172.21.15.45:47794) 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.216+0000 7fef4a7fc640 1 -- 172.21.15.45:0/1581654416 learned_addr learned my addr 172.21.15.45:0/1581654416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4affd640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c102200 0x7fef4c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4b7fe640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 0x7fef4c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 -- 172.21.15.45:0/1581654416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 msgr2=0x7fef4c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 0x7fef4c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 -- 172.21.15.45:0/1581654416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c102200 msgr2=0x7fef4c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c102200 0x7fef4c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 -- 172.21.15.45:0/1581654416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c19f6e0 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4b7fe640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 0x7fef4c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef4a7fc640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef3400c970 tx=0x7fef3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34007bb0 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef34007d50 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c19f9d0 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340187a0 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.217+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c19fed0 con 0x7fef4c104b30 2024-04-25T00:20:02.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.218+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c0fd6e0 con 0x7fef4c104b30 2024-04-25T00:20:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.219+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fef3400f450 con 0x7fef4c104b30 2024-04-25T00:20:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.220+0000 7fef43fff640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 0x7fef380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.220+0000 7fef4affd640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 0x7fef380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.220+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fef340cf450 con 0x7fef4c104b30 2024-04-25T00:20:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.220+0000 7fef4affd640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 0x7fef380a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006010 tx=0x7fef3c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.224+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fef34014030 con 0x7fef4c104b30 2024-04-25T00:20:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:active 2024-04-25T00:20:02.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: Cluster is now healthy 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:active 2024-04-25T00:20:02.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:01 smithi122 ceph-mon[32959]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:20:02.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] up:active 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: Cluster is now healthy 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] up:active 2024-04-25T00:20:02.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:01 smithi071 ceph-mon[32667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:20:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.436+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fef4c061b80 con 0x7fef380a2c20 2024-04-25T00:20:02.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.441+0000 7fef43fff640 1 -- 172.21.15.45:0/1581654416 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fef4c061b80 con 0x7fef380a2c20 2024-04-25T00:20:02.441 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/c7bcad05-6684-43f3-a676-47abd672942f 2024-04-25T00:20:02.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 msgr2=0x7fef380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 0x7fef380a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006010 tx=0x7fef3c000f40 comp rx=0 tx=0).stop 2024-04-25T00:20:02.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 msgr2=0x7fef4c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef3400c970 tx=0x7fef3400ce30 comp rx=0 tx=0).stop 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 shutdown_connections 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fef380a2c20 0x7fef380a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef4c107460 0x7fef4c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef4c104b30 0x7fef4c198000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 --2- 172.21.15.45:0/1581654416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef4c102200 0x7fef4c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 >> 172.21.15.45:0/1581654416 conn(0x7fef4c0fbbf0 msgr2=0x7fef4c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 shutdown_connections 2024-04-25T00:20:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.443+0000 7fef5139e640 1 -- 172.21.15.45:0/1581654416 wait complete. 2024-04-25T00:20:02.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:20:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- 172.21.15.45:0/3506251574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9641029a0 msgr2=0x7fc964102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/3506251574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9641029a0 0x7fc964102e20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc958009900 tx=0x7fc95802a410 comp rx=0 tx=0).stop 2024-04-25T00:20:02.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- 172.21.15.45:0/3506251574 shutdown_connections 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/3506251574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc964103360 0x7fc964109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/3506251574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9641029a0 0x7fc964102e20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/3506251574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 0x7fc964101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- 172.21.15.45:0/3506251574 >> 172.21.15.45:0/3506251574 conn(0x7fc9640fcd40 msgr2=0x7fc9640ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- 172.21.15.45:0/3506251574 shutdown_connections 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- 172.21.15.45:0/3506251574 wait complete. 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 Processor -- start 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.556+0000 7fc96a7e3640 1 -- start start 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 0x7fc964197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9641029a0 0x7fc964198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d490 con 0x7fc964103360 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d310 con 0x7fc9641017b0 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc96a7e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d610 con 0x7fc9641029a0 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47810/0 (socket says 172.21.15.45:47810) 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 -- 172.21.15.45:0/2532800709 learned_addr learned my addr 172.21.15.45:0/2532800709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc963fff640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 0x7fc964197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc9637fe640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9641029a0 0x7fc964198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 -- 172.21.15.45:0/2532800709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9641029a0 msgr2=0x7fc964198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9641029a0 0x7fc964198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 -- 172.21.15.45:0/2532800709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 msgr2=0x7fc964197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 0x7fc964197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 -- 172.21.15.45:0/2532800709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96419f750 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.557+0000 7fc968d80640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00b970 tx=0x7fc95c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.558+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c019070 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.558+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c00cb40 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.558+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96419fa40 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.558+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c007650 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.558+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96419ffe0 con 0x7fc964103360 2024-04-25T00:20:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.559+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc964108a70 con 0x7fc964103360 2024-04-25T00:20:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.560+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc95c01c070 con 0x7fc964103360 2024-04-25T00:20:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.560+0000 7fc9617fa640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 0x7fc94c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.560+0000 7fc963fff640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 0x7fc94c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.561+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc95c0cedb0 con 0x7fc964103360 2024-04-25T00:20:02.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.561+0000 7fc963fff640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 0x7fc94c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc964102800 tx=0x7fc95400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.564+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc95c09b3d0 con 0x7fc964103360 2024-04-25T00:20:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.790+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc96402d170 con 0x7fc964103360 2024-04-25T00:20:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.790+0000 7fc9617fa640 1 -- 172.21.15.45:0/2532800709 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7fc95c09f6d0 con 0x7fc964103360 2024-04-25T00:20:02.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:02.791 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:20:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 msgr2=0x7fc94c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 0x7fc94c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc964102800 tx=0x7fc95400a2d0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 msgr2=0x7fc96419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00b970 tx=0x7fc95c00be30 comp rx=0 tx=0).stop 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 shutdown_connections 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc94c0a2c20 0x7fc94c0a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc964103360 0x7fc96419f0e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9641029a0 0x7fc964198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 --2- 172.21.15.45:0/2532800709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9641017b0 0x7fc964197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 >> 172.21.15.45:0/2532800709 conn(0x7fc9640fcd40 msgr2=0x7fc964105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 shutdown_connections 2024-04-25T00:20:02.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.792+0000 7fc96a7e3640 1 -- 172.21.15.45:0/2532800709 wait complete. 2024-04-25T00:20:02.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 -- 172.21.15.45:0/1818075273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc1006a0 msgr2=0x7f2fcc100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/1818075273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc100ac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc802a410 comp rx=0 tx=0).stop 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 -- 172.21.15.45:0/1818075273 shutdown_connections 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/1818075273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc102250 0x7f2fcc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/1818075273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc101890 0x7f2fcc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/1818075273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc100ac0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 -- 172.21.15.45:0/1818075273 >> 172.21.15.45:0/1818075273 conn(0x7f2fcc0fbc30 msgr2=0x7f2fcc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 -- 172.21.15.45:0/1818075273 shutdown_connections 2024-04-25T00:20:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.907+0000 7f2fd43c1640 1 -- 172.21.15.45:0/1818075273 wait complete. 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 Processor -- start 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 -- start start 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 0x7f2fcc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10d430 con 0x7f2fcc101890 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10d2b0 con 0x7f2fcc1006a0 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd43c1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc10d5b0 con 0x7f2fcc102250 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47824/0 (socket says 172.21.15.45:47824) 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 -- 172.21.15.45:0/4120542987 learned_addr learned my addr 172.21.15.45:0/4120542987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd215d640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd295e640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 0x7f2fcc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 -- 172.21.15.45:0/4120542987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 msgr2=0x7f2fcc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 0x7f2fcc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 -- 172.21.15.45:0/4120542987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc1006a0 msgr2=0x7f2fcc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd195c640 1 -- 172.21.15.45:0/4120542987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc19f6e0 con 0x7f2fcc101890 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.908+0000 7f2fd295e640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 0x7f2fcc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fd195c640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c970 tx=0x7f2fbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc007bb0 con 0x7f2fcc101890 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc007d50 con 0x7f2fcc101890 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc19f9d0 con 0x7f2fcc101890 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00f450 con 0x7f2fcc101890 2024-04-25T00:20:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.909+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc19ff20 con 0x7f2fcc101890 2024-04-25T00:20:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.910+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc107a40 con 0x7f2fcc101890 2024-04-25T00:20:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.911+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2fbc005020 con 0x7f2fcc101890 2024-04-25T00:20:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.912+0000 7f2fc37fe640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 0x7f2fb80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.912+0000 7f2fd215d640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 0x7f2fb80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.912+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2fbc0cf1e0 con 0x7f2fcc101890 2024-04-25T00:20:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.912+0000 7f2fd215d640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 0x7f2fb80a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:02.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:02.916+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2fbc014030 con 0x7f2fcc101890 2024-04-25T00:20:03.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.103+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fcc02d3c0 con 0x7f2fcc101890 2024-04-25T00:20:03.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.105+0000 7f2fc37fe640 1 -- 172.21.15.45:0/4120542987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7f2fbc097020 con 0x7f2fcc101890 2024-04-25T00:20:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:03.106 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:20:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 msgr2=0x7f2fb80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 0x7f2fb80a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 msgr2=0x7f2fcc198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c970 tx=0x7f2fbc00ce30 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 shutdown_connections 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2fb80a2ca0 0x7f2fb80a5160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc102250 0x7f2fcc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fcc101890 0x7f2fcc198000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 --2- 172.21.15.45:0/4120542987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fcc1006a0 0x7f2fcc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 >> 172.21.15.45:0/4120542987 conn(0x7f2fcc0fbc30 msgr2=0x7f2fcc104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 shutdown_connections 2024-04-25T00:20:03.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.107+0000 7f2fd43c1640 1 -- 172.21.15.45:0/4120542987 wait complete. 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2532800709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:03.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:20:03.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 -- 172.21.15.45:0/4114363354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 msgr2=0x7ffbb4102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/4114363354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4102e20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009900 tx=0x7ffb9c02a410 comp rx=0 tx=0).stop 2024-04-25T00:20:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 -- 172.21.15.45:0/4114363354 shutdown_connections 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/4114363354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb4103360 0x7ffbb4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/4114363354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4102e20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/4114363354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb41017b0 0x7ffbb4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 -- 172.21.15.45:0/4114363354 >> 172.21.15.45:0/4114363354 conn(0x7ffbb40fcd40 msgr2=0x7ffbb40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 -- 172.21.15.45:0/4114363354 shutdown_connections 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.220+0000 7ffbb91c4640 1 -- 172.21.15.45:0/4114363354 wait complete. 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 Processor -- start 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 -- start start 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41017b0 0x7ffbb4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 0x7ffbb419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb410d4a0 con 0x7ffbb41029a0 2024-04-25T00:20:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb410d320 con 0x7ffbb41017b0 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb91c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb410d620 con 0x7ffbb4103360 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47836/0 (socket says 172.21.15.45:47836) 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 -- 172.21.15.45:0/842946857 learned_addr learned my addr 172.21.15.45:0/842946857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb359e640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 0x7ffbb419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb2d9d640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41017b0 0x7ffbb4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 -- 172.21.15.45:0/842946857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 msgr2=0x7ffbb419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 0x7ffbb419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 -- 172.21.15.45:0/842946857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41017b0 msgr2=0x7ffbb4197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41017b0 0x7ffbb4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbabfff640 1 -- 172.21.15.45:0/842946857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb419f760 con 0x7ffbb41029a0 2024-04-25T00:20:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.221+0000 7ffbb359e640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 0x7ffbb419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbabfff640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009230 tx=0x7ffb9c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c03f030 con 0x7ffbb41029a0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9c003ce0 con 0x7ffbb41029a0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb419f9f0 con 0x7ffbb41029a0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c035070 con 0x7ffbb41029a0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.222+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb419ff60 con 0x7ffbb41029a0 2024-04-25T00:20:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.223+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4108ad0 con 0x7ffbb41029a0 2024-04-25T00:20:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.224+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffb9c042030 con 0x7ffbb41029a0 2024-04-25T00:20:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.225+0000 7ffbb0d99640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 0x7ffba00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.225+0000 7ffbb2d9d640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 0x7ffba00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.225+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ffb9c0efed0 con 0x7ffbb41029a0 2024-04-25T00:20:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.225+0000 7ffbb2d9d640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 0x7ffba00a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4102800 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.228+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb9c0bca90 con 0x7ffbb41029a0 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2532800709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:03.368 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:20:02.964+0000 7f7dfaee0700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:20:02.964+0000 7f7dfaee0700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:20:02.964+0000 7f7dfaee0700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: from='client.24698 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2532800709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.410+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbb4061b80 con 0x7ffba00a2c20 2024-04-25T00:20:03.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.421+0000 7ffbb0d99640 1 -- 172.21.15.45:0/842946857 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340703 (secure 0 0 0) 0x7ffbb4061b80 con 0x7ffba00a2c20 2024-04-25T00:20:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:03.428 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.424+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 msgr2=0x7ffba00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.424+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 0x7ffba00a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4102800 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.424+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 msgr2=0x7ffbb4198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.424+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009230 tx=0x7ffb9c004360 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 shutdown_connections 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ffba00a2c20 0x7ffba00a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb4103360 0x7ffbb419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb41029a0 0x7ffbb4198080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 --2- 172.21.15.45:0/842946857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41017b0 0x7ffbb4197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 >> 172.21.15.45:0/842946857 conn(0x7ffbb40fcd40 msgr2=0x7ffbb4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 shutdown_connections 2024-04-25T00:20:03.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.425+0000 7ffbb91c4640 1 -- 172.21.15.45:0/842946857 wait complete. 2024-04-25T00:20:03.436 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-04-25T00:20:03.267469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600590,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":149,"num_read_kb":131,"num_write":344,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385868,"kb_used_data":11756,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555956,"statfs":{"total":1151940427776,"available":1151545298944,"internally_reserved":0,"allocated":12038144,"data_stored":7567909,"data_compressed":284160,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5723,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":116,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542178+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:19:39.542178+0000","last_peered":"2024-04-25T00:19:39.542178+0000","last_clean":"2024-04-25T00:19:39.542178+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:19:39.542178+0000","last_undegraded":"2024-04-25T00:19:39.542178+0000","last_fullsized":"2024-04-25T00:19:39.542178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541616+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:19:39.541616+0000","last_peered":"2024-04-25T00:19:39.541616+0000","last_clean":"2024-04-25T00:19:39.541616+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:19:39.541616+0000","last_undegraded":"2024-04-25T00:19:39.541616+0000","last_fullsized":"2024-04-25T00:19:39.541616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541497+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:19:39.541497+0000","last_peered":"2024-04-25T00:19:39.541497+0000","last_clean":"2024-04-25T00:19:39.541497+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:19:39.541497+0000","last_undegraded":"2024-04-25T00:19:39.541497+0000","last_fullsized":"2024-04-25T00:19:39.541497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542700+0000","last_change":"2024-04-25T00:19:33.880302+0000","last_active":"2024-04-25T00:19:39.542700+0000","last_peered":"2024-04-25T00:19:39.542700+0000","last_clean":"2024-04-25T00:19:39.542700+0000","last_became_active":"2024-04-25T00:19:33.879849+0000","last_became_peered":"2024-04-25T00:19:33.879849+0000","last_unstale":"2024-04-25T00:19:39.542700+0000","last_undegraded":"2024-04-25T00:19:39.542700+0000","last_fullsized":"2024-04-25T00:19:39.542700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:32.449680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045744+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:19:39.045744+0000","last_peered":"2024-04-25T00:19:39.045744+0000","last_clean":"2024-04-25T00:19:39.045744+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:19:39.045744+0000","last_undegraded":"2024-04-25T00:19:39.045744+0000","last_fullsized":"2024-04-25T00:19:39.045744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891976+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:19:38.891976+0000","last_peered":"2024-04-25T00:19:38.891976+0000","last_clean":"2024-04-25T00:19:38.891976+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:19:38.891976+0000","last_undegraded":"2024-04-25T00:19:38.891976+0000","last_fullsized":"2024-04-25T00:19:38.891976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541604+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:19:39.541604+0000","last_peered":"2024-04-25T00:19:39.541604+0000","last_clean":"2024-04-25T00:19:39.541604+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:19:39.541604+0000","last_undegraded":"2024-04-25T00:19:39.541604+0000","last_fullsized":"2024-04-25T00:19:39.541604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542714+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:19:39.542714+0000","last_peered":"2024-04-25T00:19:39.542714+0000","last_clean":"2024-04-25T00:19:39.542714+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:19:39.542714+0000","last_undegraded":"2024-04-25T00:19:39.542714+0000","last_fullsized":"2024-04-25T00:19:39.542714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542461+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:19:39.542461+0000","last_peered":"2024-04-25T00:19:39.542461+0000","last_clean":"2024-04-25T00:19:39.542461+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:19:39.542461+0000","last_undegraded":"2024-04-25T00:19:39.542461+0000","last_fullsized":"2024-04-25T00:19:39.542461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.892228+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:19:38.892228+0000","last_peered":"2024-04-25T00:19:38.892228+0000","last_clean":"2024-04-25T00:19:38.892228+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:19:38.892228+0000","last_undegraded":"2024-04-25T00:19:38.892228+0000","last_fullsized":"2024-04-25T00:19:38.892228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542824+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:19:39.542824+0000","last_peered":"2024-04-25T00:19:39.542824+0000","last_clean":"2024-04-25T00:19:39.542824+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:19:39.542824+0000","last_undegraded":"2024-04-25T00:19:39.542824+0000","last_fullsized":"2024-04-25T00:19:39.542824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542568+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:19:39.542568+0000","last_peered":"2024-04-25T00:19:39.542568+0000","last_clean":"2024-04-25T00:19:39.542568+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:19:39.542568+0000","last_undegraded":"2024-04-25T00:19:39.542568+0000","last_fullsized":"2024-04-25T00:19:39.542568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542324+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:19:39.542324+0000","last_peered":"2024-04-25T00:19:39.542324+0000","last_clean":"2024-04-25T00:19:39.542324+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:19:39.542324+0000","last_undegraded":"2024-04-25T00:19:39.542324+0000","last_fullsized":"2024-04-25T00:19:39.542324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541185+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:19:39.541185+0000","last_peered":"2024-04-25T00:19:39.541185+0000","last_clean":"2024-04-25T00:19:39.541185+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:19:39.541185+0000","last_undegraded":"2024-04-25T00:19:39.541185+0000","last_fullsized":"2024-04-25T00:19:39.541185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541199+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:19:39.541199+0000","last_peered":"2024-04-25T00:19:39.541199+0000","last_clean":"2024-04-25T00:19:39.541199+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:19:39.541199+0000","last_undegraded":"2024-04-25T00:19:39.541199+0000","last_fullsized":"2024-04-25T00:19:39.541199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541173+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:19:39.541173+0000","last_peered":"2024-04-25T00:19:39.541173+0000","last_clean":"2024-04-25T00:19:39.541173+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:19:39.541173+0000","last_undegraded":"2024-04-25T00:19:39.541173+0000","last_fullsized":"2024-04-25T00:19:39.541173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542005+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:19:39.542005+0000","last_peered":"2024-04-25T00:19:39.542005+0000","last_clean":"2024-04-25T00:19:39.542005+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:19:39.542005+0000","last_undegraded":"2024-04-25T00:19:39.542005+0000","last_fullsized":"2024-04-25T00:19:39.542005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891843+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:19:38.891843+0000","last_peered":"2024-04-25T00:19:38.891843+0000","last_clean":"2024-04-25T00:19:38.891843+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:19:38.891843+0000","last_undegraded":"2024-04-25T00:19:38.891843+0000","last_fullsized":"2024-04-25T00:19:38.891843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891543+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:19:38.891543+0000","last_peered":"2024-04-25T00:19:38.891543+0000","last_clean":"2024-04-25T00:19:38.891543+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:19:38.891543+0000","last_undegraded":"2024-04-25T00:19:38.891543+0000","last_fullsized":"2024-04-25T00:19:38.891543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891296+0000","last_change":"2024-04-25T00:19:37.914577+0000","last_active":"2024-04-25T00:19:38.891296+0000","last_peered":"2024-04-25T00:19:38.891296+0000","last_clean":"2024-04-25T00:19:38.891296+0000","last_became_active":"2024-04-25T00:19:37.914014+0000","last_became_peered":"2024-04-25T00:19:37.914014+0000","last_unstale":"2024-04-25T00:19:38.891296+0000","last_undegraded":"2024-04-25T00:19:38.891296+0000","last_fullsized":"2024-04-25T00:19:38.891296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:55.580561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891881+0000","last_change":"2024-04-25T00:19:32.869791+0000","last_active":"2024-04-25T00:19:38.891881+0000","last_peered":"2024-04-25T00:19:38.891881+0000","last_clean":"2024-04-25T00:19:38.891881+0000","last_became_active":"2024-04-25T00:19:32.869432+0000","last_became_peered":"2024-04-25T00:19:32.869432+0000","last_unstale":"2024-04-25T00:19:38.891881+0000","last_undegraded":"2024-04-25T00:19:38.891881+0000","last_fullsized":"2024-04-25T00:19:38.891881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:52.026951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543614+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:19:39.543614+0000","last_peered":"2024-04-25T00:19:39.543614+0000","last_clean":"2024-04-25T00:19:39.543614+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:19:39.543614+0000","last_undegraded":"2024-04-25T00:19:39.543614+0000","last_fullsized":"2024-04-25T00:19:39.543614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541546+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:19:39.541546+0000","last_peered":"2024-04-25T00:19:39.541546+0000","last_clean":"2024-04-25T00:19:39.541546+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:19:39.541546+0000","last_undegraded":"2024-04-25T00:19:39.541546+0000","last_fullsized":"2024-04-25T00:19:39.541546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543785+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:19:39.543785+0000","last_peered":"2024-04-25T00:19:39.543785+0000","last_clean":"2024-04-25T00:19:39.543785+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:19:39.543785+0000","last_undegraded":"2024-04-25T00:19:39.543785+0000","last_fullsized":"2024-04-25T00:19:39.543785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541382+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:19:39.541382+0000","last_peered":"2024-04-25T00:19:39.541382+0000","last_clean":"2024-04-25T00:19:39.541382+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:19:39.541382+0000","last_undegraded":"2024-04-25T00:19:39.541382+0000","last_fullsized":"2024-04-25T00:19:39.541382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543053+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:19:39.543053+0000","last_peered":"2024-04-25T00:19:39.543053+0000","last_clean":"2024-04-25T00:19:39.543053+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:19:39.543053+0000","last_undegraded":"2024-04-25T00:19:39.543053+0000","last_fullsized":"2024-04-25T00:19:39.543053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991690+0000","last_change":"2024-04-25T00:19:32.874816+0000","last_active":"2024-04-25T00:20:00.991690+0000","last_peered":"2024-04-25T00:20:00.991690+0000","last_clean":"2024-04-25T00:20:00.991690+0000","last_became_active":"2024-04-25T00:19:32.874180+0000","last_became_peered":"2024-04-25T00:19:32.874180+0000","last_unstale":"2024-04-25T00:20:00.991690+0000","last_undegraded":"2024-04-25T00:20:00.991690+0000","last_fullsized":"2024-04-25T00:20:00.991690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:38.437038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542061+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:19:39.542061+0000","last_peered":"2024-04-25T00:19:39.542061+0000","last_clean":"2024-04-25T00:19:39.542061+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:19:39.542061+0000","last_undegraded":"2024-04-25T00:19:39.542061+0000","last_fullsized":"2024-04-25T00:19:39.542061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542126+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:19:39.542126+0000","last_peered":"2024-04-25T00:19:39.542126+0000","last_clean":"2024-04-25T00:19:39.542126+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:19:39.542126+0000","last_undegraded":"2024-04-25T00:19:39.542126+0000","last_fullsized":"2024-04-25T00:19:39.542126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991117+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:20:00.991117+0000","last_peered":"2024-04-25T00:20:00.991117+0000","last_clean":"2024-04-25T00:20:00.991117+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:20:00.991117+0000","last_undegraded":"2024-04-25T00:20:00.991117+0000","last_fullsized":"2024-04-25T00:20:00.991117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541985+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:19:39.541985+0000","last_peered":"2024-04-25T00:19:39.541985+0000","last_clean":"2024-04-25T00:19:39.541985+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:19:39.541985+0000","last_undegraded":"2024-04-25T00:19:39.541985+0000","last_fullsized":"2024-04-25T00:19:39.541985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891537+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:19:38.891537+0000","last_peered":"2024-04-25T00:19:38.891537+0000","last_clean":"2024-04-25T00:19:38.891537+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:19:38.891537+0000","last_undegraded":"2024-04-25T00:19:38.891537+0000","last_fullsized":"2024-04-25T00:19:38.891537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.963884+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:19:57.963884+0000","last_peered":"2024-04-25T00:19:57.963884+0000","last_clean":"2024-04-25T00:19:57.963884+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:19:57.963884+0000","last_undegraded":"2024-04-25T00:19:57.963884+0000","last_fullsized":"2024-04-25T00:19:57.963884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045745+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:19:39.045745+0000","last_peered":"2024-04-25T00:19:39.045745+0000","last_clean":"2024-04-25T00:19:39.045745+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:19:39.045745+0000","last_undegraded":"2024-04-25T00:19:39.045745+0000","last_fullsized":"2024-04-25T00:19:39.045745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891366+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:19:38.891366+0000","last_peered":"2024-04-25T00:19:38.891366+0000","last_clean":"2024-04-25T00:19:38.891366+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:19:38.891366+0000","last_undegraded":"2024-04-25T00:19:38.891366+0000","last_fullsized":"2024-04-25T00:19:38.891366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.971937+0000","last_change":"2024-04-25T00:19:32.869204+0000","last_active":"2024-04-25T00:19:58.971937+0000","last_peered":"2024-04-25T00:19:58.971937+0000","last_clean":"2024-04-25T00:19:58.971937+0000","last_became_active":"2024-04-25T00:19:32.868826+0000","last_became_peered":"2024-04-25T00:19:32.868826+0000","last_unstale":"2024-04-25T00:19:58.971937+0000","last_undegraded":"2024-04-25T00:19:58.971937+0000","last_fullsized":"2024-04-25T00:19:58.971937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:01.439105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542054+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:19:39.542054+0000","last_peered":"2024-04-25T00:19:39.542054+0000","last_clean":"2024-04-25T00:19:39.542054+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:19:39.542054+0000","last_undegraded":"2024-04-25T00:19:39.542054+0000","last_fullsized":"2024-04-25T00:19:39.542054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541896+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:19:39.541896+0000","last_peered":"2024-04-25T00:19:39.541896+0000","last_clean":"2024-04-25T00:19:39.541896+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:19:39.541896+0000","last_undegraded":"2024-04-25T00:19:39.541896+0000","last_fullsized":"2024-04-25T00:19:39.541896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.972710+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:19:58.972710+0000","last_peered":"2024-04-25T00:19:58.972710+0000","last_clean":"2024-04-25T00:19:58.972710+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:19:58.972710+0000","last_undegraded":"2024-04-25T00:19:58.972710+0000","last_fullsized":"2024-04-25T00:19:58.972710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542261+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:19:39.542261+0000","last_peered":"2024-04-25T00:19:39.542261+0000","last_clean":"2024-04-25T00:19:39.542261+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:19:39.542261+0000","last_undegraded":"2024-04-25T00:19:39.542261+0000","last_fullsized":"2024-04-25T00:19:39.542261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541608+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:19:39.541608+0000","last_peered":"2024-04-25T00:19:39.541608+0000","last_clean":"2024-04-25T00:19:39.541608+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:19:39.541608+0000","last_undegraded":"2024-04-25T00:19:39.541608+0000","last_fullsized":"2024-04-25T00:19:39.541608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.981752+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:19:59.981752+0000","last_peered":"2024-04-25T00:19:59.981752+0000","last_clean":"2024-04-25T00:19:59.981752+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:19:59.981752+0000","last_undegraded":"2024-04-25T00:19:59.981752+0000","last_fullsized":"2024-04-25T00:19:59.981752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891176+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:19:38.891176+0000","last_peered":"2024-04-25T00:19:38.891176+0000","last_clean":"2024-04-25T00:19:38.891176+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:19:38.891176+0000","last_undegraded":"2024-04-25T00:19:38.891176+0000","last_fullsized":"2024-04-25T00:19:38.891176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543105+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:19:39.543105+0000","last_peered":"2024-04-25T00:19:39.543105+0000","last_clean":"2024-04-25T00:19:39.543105+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:19:39.543105+0000","last_undegraded":"2024-04-25T00:19:39.543105+0000","last_fullsized":"2024-04-25T00:19:39.543105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961812+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:19:57.961812+0000","last_peered":"2024-04-25T00:19:57.961812+0000","last_clean":"2024-04-25T00:19:57.961812+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:19:57.961812+0000","last_undegraded":"2024-04-25T00:19:57.961812+0000","last_fullsized":"2024-04-25T00:19:57.961812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542662+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:19:39.542662+0000","last_peered":"2024-04-25T00:19:39.542662+0000","last_clean":"2024-04-25T00:19:39.542662+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:19:39.542662+0000","last_undegraded":"2024-04-25T00:19:39.542662+0000","last_fullsized":"2024-04-25T00:19:39.542662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891025+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:19:38.891025+0000","last_peered":"2024-04-25T00:19:38.891025+0000","last_clean":"2024-04-25T00:19:38.891025+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:19:38.891025+0000","last_undegraded":"2024-04-25T00:19:38.891025+0000","last_fullsized":"2024-04-25T00:19:38.891025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961669+0000","last_change":"2024-04-25T00:19:32.874259+0000","last_active":"2024-04-25T00:19:57.961669+0000","last_peered":"2024-04-25T00:19:57.961669+0000","last_clean":"2024-04-25T00:19:57.961669+0000","last_became_active":"2024-04-25T00:19:32.873885+0000","last_became_peered":"2024-04-25T00:19:32.873885+0000","last_unstale":"2024-04-25T00:19:57.961669+0000","last_undegraded":"2024-04-25T00:19:57.961669+0000","last_fullsized":"2024-04-25T00:19:57.961669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:45.132551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541629+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:19:39.541629+0000","last_peered":"2024-04-25T00:19:39.541629+0000","last_clean":"2024-04-25T00:19:39.541629+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:19:39.541629+0000","last_undegraded":"2024-04-25T00:19:39.541629+0000","last_fullsized":"2024-04-25T00:19:39.541629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542880+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:19:39.542880+0000","last_peered":"2024-04-25T00:19:39.542880+0000","last_clean":"2024-04-25T00:19:39.542880+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:19:39.542880+0000","last_undegraded":"2024-04-25T00:19:39.542880+0000","last_fullsized":"2024-04-25T00:19:39.542880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.959859+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:19:57.959859+0000","last_peered":"2024-04-25T00:19:57.959859+0000","last_clean":"2024-04-25T00:19:57.959859+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:19:57.959859+0000","last_undegraded":"2024-04-25T00:19:57.959859+0000","last_fullsized":"2024-04-25T00:19:57.959859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542194+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:19:39.542194+0000","last_peered":"2024-04-25T00:19:39.542194+0000","last_clean":"2024-04-25T00:19:39.542194+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:19:39.542194+0000","last_undegraded":"2024-04-25T00:19:39.542194+0000","last_fullsized":"2024-04-25T00:19:39.542194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891265+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:19:38.891265+0000","last_peered":"2024-04-25T00:19:38.891265+0000","last_clean":"2024-04-25T00:19:38.891265+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:19:38.891265+0000","last_undegraded":"2024-04-25T00:19:38.891265+0000","last_fullsized":"2024-04-25T00:19:38.891265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961673+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:19:57.961673+0000","last_peered":"2024-04-25T00:19:57.961673+0000","last_clean":"2024-04-25T00:19:57.961673+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:19:57.961673+0000","last_undegraded":"2024-04-25T00:19:57.961673+0000","last_fullsized":"2024-04-25T00:19:57.961673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045944+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:19:39.045944+0000","last_peered":"2024-04-25T00:19:39.045944+0000","last_clean":"2024-04-25T00:19:39.045944+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:19:39.045944+0000","last_undegraded":"2024-04-25T00:19:39.045944+0000","last_fullsized":"2024-04-25T00:19:39.045944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542209+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:19:39.542209+0000","last_peered":"2024-04-25T00:19:39.542209+0000","last_clean":"2024-04-25T00:19:39.542209+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:19:39.542209+0000","last_undegraded":"2024-04-25T00:19:39.542209+0000","last_fullsized":"2024-04-25T00:19:39.542209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:01.841927+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:20:01.841927+0000","last_peered":"2024-04-25T00:20:01.841927+0000","last_clean":"2024-04-25T00:20:01.841927+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:20:01.841927+0000","last_undegraded":"2024-04-25T00:20:01.841927+0000","last_fullsized":"2024-04-25T00:20:01.841927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542087+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:19:39.542087+0000","last_peered":"2024-04-25T00:19:39.542087+0000","last_clean":"2024-04-25T00:19:39.542087+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:19:39.542087+0000","last_undegraded":"2024-04-25T00:19:39.542087+0000","last_fullsized":"2024-04-25T00:19:39.542087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541932+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:19:39.541932+0000","last_peered":"2024-04-25T00:19:39.541932+0000","last_clean":"2024-04-25T00:19:39.541932+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:19:39.541932+0000","last_undegraded":"2024-04-25T00:19:39.541932+0000","last_fullsized":"2024-04-25T00:19:39.541932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.981411+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:19:59.981411+0000","last_peered":"2024-04-25T00:19:59.981411+0000","last_clean":"2024-04-25T00:19:59.981411+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:19:59.981411+0000","last_undegraded":"2024-04-25T00:19:59.981411+0000","last_fullsized":"2024-04-25T00:19:59.981411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542158+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:19:39.542158+0000","last_peered":"2024-04-25T00:19:39.542158+0000","last_clean":"2024-04-25T00:19:39.542158+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:19:39.542158+0000","last_undegraded":"2024-04-25T00:19:39.542158+0000","last_fullsized":"2024-04-25T00:19:39.542158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891681+0000","last_change":"2024-04-25T00:19:37.915144+0000","last_active":"2024-04-25T00:19:38.891681+0000","last_peered":"2024-04-25T00:19:38.891681+0000","last_clean":"2024-04-25T00:19:38.891681+0000","last_became_active":"2024-04-25T00:19:37.914736+0000","last_became_peered":"2024-04-25T00:19:37.914736+0000","last_unstale":"2024-04-25T00:19:38.891681+0000","last_undegraded":"2024-04-25T00:19:38.891681+0000","last_fullsized":"2024-04-25T00:19:38.891681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.900136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.980949+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:19:59.980949+0000","last_peered":"2024-04-25T00:19:59.980949+0000","last_clean":"2024-04-25T00:19:59.980949+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:19:59.980949+0000","last_undegraded":"2024-04-25T00:19:59.980949+0000","last_fullsized":"2024-04-25T00:19:59.980949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541948+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:19:39.541948+0000","last_peered":"2024-04-25T00:19:39.541948+0000","last_clean":"2024-04-25T00:19:39.541948+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:19:39.541948+0000","last_undegraded":"2024-04-25T00:19:39.541948+0000","last_fullsized":"2024-04-25T00:19:39.541948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542716+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:19:39.542716+0000","last_peered":"2024-04-25T00:19:39.542716+0000","last_clean":"2024-04-25T00:19:39.542716+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:19:39.542716+0000","last_undegraded":"2024-04-25T00:19:39.542716+0000","last_fullsized":"2024-04-25T00:19:39.542716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991123+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:20:00.991123+0000","last_peered":"2024-04-25T00:20:00.991123+0000","last_clean":"2024-04-25T00:20:00.991123+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:20:00.991123+0000","last_undegraded":"2024-04-25T00:20:00.991123+0000","last_fullsized":"2024-04-25T00:20:00.991123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541483+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:19:39.541483+0000","last_peered":"2024-04-25T00:19:39.541483+0000","last_clean":"2024-04-25T00:19:39.541483+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:19:39.541483+0000","last_undegraded":"2024-04-25T00:19:39.541483+0000","last_fullsized":"2024-04-25T00:19:39.541483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541945+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:19:39.541945+0000","last_peered":"2024-04-25T00:19:39.541945+0000","last_clean":"2024-04-25T00:19:39.541945+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:19:39.541945+0000","last_undegraded":"2024-04-25T00:19:39.541945+0000","last_fullsized":"2024-04-25T00:19:39.541945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.968016+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:20:00.968016+0000","last_peered":"2024-04-25T00:20:00.968016+0000","last_clean":"2024-04-25T00:20:00.968016+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:20:00.968016+0000","last_undegraded":"2024-04-25T00:20:00.968016+0000","last_fullsized":"2024-04-25T00:20:00.968016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541810+0000","last_change":"2024-04-25T00:19:33.880006+0000","last_active":"2024-04-25T00:19:39.541810+0000","last_peered":"2024-04-25T00:19:39.541810+0000","last_clean":"2024-04-25T00:19:39.541810+0000","last_became_active":"2024-04-25T00:19:33.879054+0000","last_became_peered":"2024-04-25T00:19:33.879054+0000","last_unstale":"2024-04-25T00:19:39.541810+0000","last_undegraded":"2024-04-25T00:19:39.541810+0000","last_fullsized":"2024-04-25T00:19:39.541810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:43.232276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045832+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:19:39.045832+0000","last_peered":"2024-04-25T00:19:39.045832+0000","last_clean":"2024-04-25T00:19:39.045832+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:19:39.045832+0000","last_undegraded":"2024-04-25T00:19:39.045832+0000","last_fullsized":"2024-04-25T00:19:39.045832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891695+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:19:38.891695+0000","last_peered":"2024-04-25T00:19:38.891695+0000","last_clean":"2024-04-25T00:19:38.891695+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:19:38.891695+0000","last_undegraded":"2024-04-25T00:19:38.891695+0000","last_fullsized":"2024-04-25T00:19:38.891695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.892108+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:19:38.892108+0000","last_peered":"2024-04-25T00:19:38.892108+0000","last_clean":"2024-04-25T00:19:38.892108+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:19:38.892108+0000","last_undegraded":"2024-04-25T00:19:38.892108+0000","last_fullsized":"2024-04-25T00:19:38.892108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542466+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:19:39.542466+0000","last_peered":"2024-04-25T00:19:39.542466+0000","last_clean":"2024-04-25T00:19:39.542466+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:19:39.542466+0000","last_undegraded":"2024-04-25T00:19:39.542466+0000","last_fullsized":"2024-04-25T00:19:39.542466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542583+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:19:39.542583+0000","last_peered":"2024-04-25T00:19:39.542583+0000","last_clean":"2024-04-25T00:19:39.542583+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:19:39.542583+0000","last_undegraded":"2024-04-25T00:19:39.542583+0000","last_fullsized":"2024-04-25T00:19:39.542583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891818+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:19:38.891818+0000","last_peered":"2024-04-25T00:19:38.891818+0000","last_clean":"2024-04-25T00:19:38.891818+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:19:38.891818+0000","last_undegraded":"2024-04-25T00:19:38.891818+0000","last_fullsized":"2024-04-25T00:19:38.891818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542458+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:19:39.542458+0000","last_peered":"2024-04-25T00:19:39.542458+0000","last_clean":"2024-04-25T00:19:39.542458+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:19:39.542458+0000","last_undegraded":"2024-04-25T00:19:39.542458+0000","last_fullsized":"2024-04-25T00:19:39.542458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542189+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:19:39.542189+0000","last_peered":"2024-04-25T00:19:39.542189+0000","last_clean":"2024-04-25T00:19:39.542189+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:19:39.542189+0000","last_undegraded":"2024-04-25T00:19:39.542189+0000","last_fullsized":"2024-04-25T00:19:39.542189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.974682+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:19:58.974682+0000","last_peered":"2024-04-25T00:19:58.974682+0000","last_clean":"2024-04-25T00:19:58.974682+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:19:58.974682+0000","last_undegraded":"2024-04-25T00:19:58.974682+0000","last_fullsized":"2024-04-25T00:19:58.974682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542723+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:19:39.542723+0000","last_peered":"2024-04-25T00:19:39.542723+0000","last_clean":"2024-04-25T00:19:39.542723+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:19:39.542723+0000","last_undegraded":"2024-04-25T00:19:39.542723+0000","last_fullsized":"2024-04-25T00:19:39.542723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542915+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:19:39.542915+0000","last_peered":"2024-04-25T00:19:39.542915+0000","last_clean":"2024-04-25T00:19:39.542915+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:19:39.542915+0000","last_undegraded":"2024-04-25T00:19:39.542915+0000","last_fullsized":"2024-04-25T00:19:39.542915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.982155+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:19:59.982155+0000","last_peered":"2024-04-25T00:19:59.982155+0000","last_clean":"2024-04-25T00:19:59.982155+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:19:59.982155+0000","last_undegraded":"2024-04-25T00:19:59.982155+0000","last_fullsized":"2024-04-25T00:19:59.982155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541817+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:19:39.541817+0000","last_peered":"2024-04-25T00:19:39.541817+0000","last_clean":"2024-04-25T00:19:39.541817+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:19:39.541817+0000","last_undegraded":"2024-04-25T00:19:39.541817+0000","last_fullsized":"2024-04-25T00:19:39.541817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541827+0000","last_change":"2024-04-25T00:19:37.967364+0000","last_active":"2024-04-25T00:19:39.541827+0000","last_peered":"2024-04-25T00:19:39.541827+0000","last_clean":"2024-04-25T00:19:39.541827+0000","last_became_active":"2024-04-25T00:19:37.966868+0000","last_became_peered":"2024-04-25T00:19:37.966868+0000","last_unstale":"2024-04-25T00:19:39.541827+0000","last_undegraded":"2024-04-25T00:19:39.541827+0000","last_fullsized":"2024-04-25T00:19:39.541827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:39.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541450+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:19:39.541450+0000","last_peered":"2024-04-25T00:19:39.541450+0000","last_clean":"2024-04-25T00:19:39.541450+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:19:39.541450+0000","last_undegraded":"2024-04-25T00:19:39.541450+0000","last_fullsized":"2024-04-25T00:19:39.541450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.974295+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:19:58.974295+0000","last_peered":"2024-04-25T00:19:58.974295+0000","last_clean":"2024-04-25T00:19:58.974295+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:19:58.974295+0000","last_undegraded":"2024-04-25T00:19:58.974295+0000","last_fullsized":"2024-04-25T00:19:58.974295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541762+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:19:39.541762+0000","last_peered":"2024-04-25T00:19:39.541762+0000","last_clean":"2024-04-25T00:19:39.541762+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:19:39.541762+0000","last_undegraded":"2024-04-25T00:19:39.541762+0000","last_fullsized":"2024-04-25T00:19:39.541762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.990678+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:20:00.990678+0000","last_peered":"2024-04-25T00:20:00.990678+0000","last_clean":"2024-04-25T00:20:00.990678+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:20:00.990678+0000","last_undegraded":"2024-04-25T00:20:00.990678+0000","last_fullsized":"2024-04-25T00:20:00.990678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542396+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:19:39.542396+0000","last_peered":"2024-04-25T00:19:39.542396+0000","last_clean":"2024-04-25T00:19:39.542396+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:19:39.542396+0000","last_undegraded":"2024-04-25T00:19:39.542396+0000","last_fullsized":"2024-04-25T00:19:39.542396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542205+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:19:39.542205+0000","last_peered":"2024-04-25T00:19:39.542205+0000","last_clean":"2024-04-25T00:19:39.542205+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:19:39.542205+0000","last_undegraded":"2024-04-25T00:19:39.542205+0000","last_fullsized":"2024-04-25T00:19:39.542205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891705+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:19:38.891705+0000","last_peered":"2024-04-25T00:19:38.891705+0000","last_clean":"2024-04-25T00:19:38.891705+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:19:38.891705+0000","last_undegraded":"2024-04-25T00:19:38.891705+0000","last_fullsized":"2024-04-25T00:19:38.891705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543492+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:19:39.543492+0000","last_peered":"2024-04-25T00:19:39.543492+0000","last_clean":"2024-04-25T00:19:39.543492+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:19:39.543492+0000","last_undegraded":"2024-04-25T00:19:39.543492+0000","last_fullsized":"2024-04-25T00:19:39.543492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891162+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:19:38.891162+0000","last_peered":"2024-04-25T00:19:38.891162+0000","last_clean":"2024-04-25T00:19:38.891162+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:19:38.891162+0000","last_undegraded":"2024-04-25T00:19:38.891162+0000","last_fullsized":"2024-04-25T00:19:38.891162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.960962+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:19:57.960962+0000","last_peered":"2024-04-25T00:19:57.960962+0000","last_clean":"2024-04-25T00:19:57.960962+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:19:57.960962+0000","last_undegraded":"2024-04-25T00:19:57.960962+0000","last_fullsized":"2024-04-25T00:19:57.960962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541725+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:19:39.541725+0000","last_peered":"2024-04-25T00:19:39.541725+0000","last_clean":"2024-04-25T00:19:39.541725+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:19:39.541725+0000","last_undegraded":"2024-04-25T00:19:39.541725+0000","last_fullsized":"2024-04-25T00:19:39.541725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542977+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:19:39.542977+0000","last_peered":"2024-04-25T00:19:39.542977+0000","last_clean":"2024-04-25T00:19:39.542977+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:19:39.542977+0000","last_undegraded":"2024-04-25T00:19:39.542977+0000","last_fullsized":"2024-04-25T00:19:39.542977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961745+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:19:57.961745+0000","last_peered":"2024-04-25T00:19:57.961745+0000","last_clean":"2024-04-25T00:19:57.961745+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:19:57.961745+0000","last_undegraded":"2024-04-25T00:19:57.961745+0000","last_fullsized":"2024-04-25T00:19:57.961745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045667+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:19:39.045667+0000","last_peered":"2024-04-25T00:19:39.045667+0000","last_clean":"2024-04-25T00:19:39.045667+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:19:39.045667+0000","last_undegraded":"2024-04-25T00:19:39.045667+0000","last_fullsized":"2024-04-25T00:19:39.045667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891910+0000","last_change":"2024-04-25T00:19:37.915452+0000","last_active":"2024-04-25T00:19:38.891910+0000","last_peered":"2024-04-25T00:19:38.891910+0000","last_clean":"2024-04-25T00:19:38.891910+0000","last_became_active":"2024-04-25T00:19:37.915009+0000","last_became_peered":"2024-04-25T00:19:37.915009+0000","last_unstale":"2024-04-25T00:19:38.891910+0000","last_undegraded":"2024-04-25T00:19:38.891910+0000","last_fullsized":"2024-04-25T00:19:38.891910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:04.882939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.990152+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:20:00.990152+0000","last_peered":"2024-04-25T00:20:00.990152+0000","last_clean":"2024-04-25T00:20:00.990152+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:20:00.990152+0000","last_undegraded":"2024-04-25T00:20:00.990152+0000","last_fullsized":"2024-04-25T00:20:00.990152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541326+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:19:39.541326+0000","last_peered":"2024-04-25T00:19:39.541326+0000","last_clean":"2024-04-25T00:19:39.541326+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:19:39.541326+0000","last_undegraded":"2024-04-25T00:19:39.541326+0000","last_fullsized":"2024-04-25T00:19:39.541326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542680+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:19:39.542680+0000","last_peered":"2024-04-25T00:19:39.542680+0000","last_clean":"2024-04-25T00:19:39.542680+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:19:39.542680+0000","last_undegraded":"2024-04-25T00:19:39.542680+0000","last_fullsized":"2024-04-25T00:19:39.542680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.957340+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:19:59.957340+0000","last_peered":"2024-04-25T00:19:59.957340+0000","last_clean":"2024-04-25T00:19:59.957340+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:19:59.957340+0000","last_undegraded":"2024-04-25T00:19:59.957340+0000","last_fullsized":"2024-04-25T00:19:59.957340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543184+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:19:39.543184+0000","last_peered":"2024-04-25T00:19:39.543184+0000","last_clean":"2024-04-25T00:19:39.543184+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:19:39.543184+0000","last_undegraded":"2024-04-25T00:19:39.543184+0000","last_fullsized":"2024-04-25T00:19:39.543184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542625+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:19:39.542625+0000","last_peered":"2024-04-25T00:19:39.542625+0000","last_clean":"2024-04-25T00:19:39.542625+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:19:39.542625+0000","last_undegraded":"2024-04-25T00:19:39.542625+0000","last_fullsized":"2024-04-25T00:19:39.542625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.962110+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:19:57.962110+0000","last_peered":"2024-04-25T00:19:57.962110+0000","last_clean":"2024-04-25T00:19:57.962110+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:19:57.962110+0000","last_undegraded":"2024-04-25T00:19:57.962110+0000","last_fullsized":"2024-04-25T00:19:57.962110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542699+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:19:39.542699+0000","last_peered":"2024-04-25T00:19:39.542699+0000","last_clean":"2024-04-25T00:19:39.542699+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:19:39.542699+0000","last_undegraded":"2024-04-25T00:19:39.542699+0000","last_fullsized":"2024-04-25T00:19:39.542699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541453+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:19:39.541453+0000","last_peered":"2024-04-25T00:19:39.541453+0000","last_clean":"2024-04-25T00:19:39.541453+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:19:39.541453+0000","last_undegraded":"2024-04-25T00:19:39.541453+0000","last_fullsized":"2024-04-25T00:19:39.541453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:01.845521+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:20:01.845521+0000","last_peered":"2024-04-25T00:20:01.845521+0000","last_clean":"2024-04-25T00:20:01.845521+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:20:01.845521+0000","last_undegraded":"2024-04-25T00:20:01.845521+0000","last_fullsized":"2024-04-25T00:20:01.845521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542854+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:19:39.542854+0000","last_peered":"2024-04-25T00:19:39.542854+0000","last_clean":"2024-04-25T00:19:39.542854+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:19:39.542854+0000","last_undegraded":"2024-04-25T00:19:39.542854+0000","last_fullsized":"2024-04-25T00:19:39.542854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541295+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:19:39.541295+0000","last_peered":"2024-04-25T00:19:39.541295+0000","last_clean":"2024-04-25T00:19:39.541295+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:19:39.541295+0000","last_undegraded":"2024-04-25T00:19:39.541295+0000","last_fullsized":"2024-04-25T00:19:39.541295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.982994+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:19:59.982994+0000","last_peered":"2024-04-25T00:19:59.982994+0000","last_clean":"2024-04-25T00:19:59.982994+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:19:59.982994+0000","last_undegraded":"2024-04-25T00:19:59.982994+0000","last_fullsized":"2024-04-25T00:19:59.982994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543008+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:19:39.543008+0000","last_peered":"2024-04-25T00:19:39.543008+0000","last_clean":"2024-04-25T00:19:39.543008+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:19:39.543008+0000","last_undegraded":"2024-04-25T00:19:39.543008+0000","last_fullsized":"2024-04-25T00:19:39.543008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541306+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:19:39.541306+0000","last_peered":"2024-04-25T00:19:39.541306+0000","last_clean":"2024-04-25T00:19:39.541306+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:19:39.541306+0000","last_undegraded":"2024-04-25T00:19:39.541306+0000","last_fullsized":"2024-04-25T00:19:39.541306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542586+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:19:39.542586+0000","last_peered":"2024-04-25T00:19:39.542586+0000","last_clean":"2024-04-25T00:19:39.542586+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:19:39.542586+0000","last_undegraded":"2024-04-25T00:19:39.542586+0000","last_fullsized":"2024-04-25T00:19:39.542586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543217+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:19:39.543217+0000","last_peered":"2024-04-25T00:19:39.543217+0000","last_clean":"2024-04-25T00:19:39.543217+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:19:39.543217+0000","last_undegraded":"2024-04-25T00:19:39.543217+0000","last_fullsized":"2024-04-25T00:19:39.543217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541452+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:19:39.541452+0000","last_peered":"2024-04-25T00:19:39.541452+0000","last_clean":"2024-04-25T00:19:39.541452+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:19:39.541452+0000","last_undegraded":"2024-04-25T00:19:39.541452+0000","last_fullsized":"2024-04-25T00:19:39.541452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891500+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:19:38.891500+0000","last_peered":"2024-04-25T00:19:38.891500+0000","last_clean":"2024-04-25T00:19:38.891500+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:19:38.891500+0000","last_undegraded":"2024-04-25T00:19:38.891500+0000","last_fullsized":"2024-04-25T00:19:38.891500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891130+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:19:38.891130+0000","last_peered":"2024-04-25T00:19:38.891130+0000","last_clean":"2024-04-25T00:19:38.891130+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:19:38.891130+0000","last_undegraded":"2024-04-25T00:19:38.891130+0000","last_fullsized":"2024-04-25T00:19:38.891130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541374+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:19:39.541374+0000","last_peered":"2024-04-25T00:19:39.541374+0000","last_clean":"2024-04-25T00:19:39.541374+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:19:39.541374+0000","last_undegraded":"2024-04-25T00:19:39.541374+0000","last_fullsized":"2024-04-25T00:19:39.541374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542476+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:19:39.542476+0000","last_peered":"2024-04-25T00:19:39.542476+0000","last_clean":"2024-04-25T00:19:39.542476+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:19:39.542476+0000","last_undegraded":"2024-04-25T00:19:39.542476+0000","last_fullsized":"2024-04-25T00:19:39.542476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542106+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:19:39.542106+0000","last_peered":"2024-04-25T00:19:39.542106+0000","last_clean":"2024-04-25T00:19:39.542106+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:19:39.542106+0000","last_undegraded":"2024-04-25T00:19:39.542106+0000","last_fullsized":"2024-04-25T00:19:39.542106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891636+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:19:38.891636+0000","last_peered":"2024-04-25T00:19:38.891636+0000","last_clean":"2024-04-25T00:19:38.891636+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:19:38.891636+0000","last_undegraded":"2024-04-25T00:19:38.891636+0000","last_fullsized":"2024-04-25T00:19:38.891636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543595+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:19:39.543595+0000","last_peered":"2024-04-25T00:19:39.543595+0000","last_clean":"2024-04-25T00:19:39.543595+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:19:39.543595+0000","last_undegraded":"2024-04-25T00:19:39.543595+0000","last_fullsized":"2024-04-25T00:19:39.543595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541537+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:19:39.541537+0000","last_peered":"2024-04-25T00:19:39.541537+0000","last_clean":"2024-04-25T00:19:39.541537+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:19:39.541537+0000","last_undegraded":"2024-04-25T00:19:39.541537+0000","last_fullsized":"2024-04-25T00:19:39.541537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542861+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:19:39.542861+0000","last_peered":"2024-04-25T00:19:39.542861+0000","last_clean":"2024-04-25T00:19:39.542861+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:19:39.542861+0000","last_undegraded":"2024-04-25T00:19:39.542861+0000","last_fullsized":"2024-04-25T00:19:39.542861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541848+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:19:39.541848+0000","last_peered":"2024-04-25T00:19:39.541848+0000","last_clean":"2024-04-25T00:19:39.541848+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:19:39.541848+0000","last_undegraded":"2024-04-25T00:19:39.541848+0000","last_fullsized":"2024-04-25T00:19:39.541848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542227+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:19:39.542227+0000","last_peered":"2024-04-25T00:19:39.542227+0000","last_clean":"2024-04-25T00:19:39.542227+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:19:39.542227+0000","last_undegraded":"2024-04-25T00:19:39.542227+0000","last_fullsized":"2024-04-25T00:19:39.542227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542109+0000","last_change":"2024-04-25T00:19:33.879484+0000","last_active":"2024-04-25T00:19:39.542109+0000","last_peered":"2024-04-25T00:19:39.542109+0000","last_clean":"2024-04-25T00:19:39.542109+0000","last_became_active":"2024-04-25T00:19:33.879046+0000","last_became_peered":"2024-04-25T00:19:33.879046+0000","last_unstale":"2024-04-25T00:19:39.542109+0000","last_undegraded":"2024-04-25T00:19:39.542109+0000","last_fullsized":"2024-04-25T00:19:39.542109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:43.540889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284395,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10222,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":90575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":478407,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":477999,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":478715,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1069375,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":519438,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":477909,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":490020,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":477645,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":478272,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1069821,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":47423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:20:03.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:20:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 -- 172.21.15.45:0/732267584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d01009b0 msgr2=0x7fc0d0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/732267584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d01009b0 0x7fc0d0100e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009900 tx=0x7fc0c402a410 comp rx=0 tx=0).stop 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 -- 172.21.15.45:0/732267584 shutdown_connections 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/732267584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 0x7fc0d01017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/732267584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d01009b0 0x7fc0d0100e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/732267584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d0100050 0x7fc0d0100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.577+0000 7fc0d79a5640 1 -- 172.21.15.45:0/732267584 >> 172.21.15.45:0/732267584 conn(0x7fc0d00fbbf0 msgr2=0x7fc0d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- 172.21.15.45:0/732267584 shutdown_connections 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- 172.21.15.45:0/732267584 wait complete. 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 Processor -- start 2024-04-25T00:20:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- start start 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 0x7fc0d019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 0x7fc0d01a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d010f610 con 0x7fc0d0100050 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d010f490 con 0x7fc0d0101350 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d79a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d010f790 con 0x7fc0d01009b0 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d5741640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d5741640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47842/0 (socket says 172.21.15.45:47842) 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.578+0000 7fc0d5741640 1 -- 172.21.15.45:0/3070128563 learned_addr learned my addr 172.21.15.45:0/3070128563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5f42640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 0x7fc0d01a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d4f40640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 0x7fc0d019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 -- 172.21.15.45:0/3070128563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 msgr2=0x7fc0d019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 0x7fc0d019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 -- 172.21.15.45:0/3070128563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 msgr2=0x7fc0d01a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 0x7fc0d01a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 -- 172.21.15.45:0/3070128563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d01a18c0 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d4f40640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 0x7fc0d019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d5741640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e9f0 tx=0x7fc0c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800cd80 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8010910 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d01a1bb0 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8018990 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.579+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d01a2120 con 0x7fc0d0100050 2024-04-25T00:20:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.580+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d010aee0 con 0x7fc0d0100050 2024-04-25T00:20:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.581+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc0c8010ac0 con 0x7fc0d0100050 2024-04-25T00:20:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.582+0000 7fc0ce7fc640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 0x7fc0b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.582+0000 7fc0d4f40640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 0x7fc0b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.582+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc0c80cf730 con 0x7fc0d0100050 2024-04-25T00:20:03.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.582+0000 7fc0d4f40640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 0x7fc0b80a5160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006010 tx=0x7fc0c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.586+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0c809c2f0 con 0x7fc0d0100050 2024-04-25T00:20:03.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.765+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0d0061b80 con 0x7fc0b80a2ca0 2024-04-25T00:20:03.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.774+0000 7fc0ce7fc640 1 -- 172.21.15.45:0/3070128563 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340703 (secure 0 0 0) 0x7fc0d0061b80 con 0x7fc0b80a2ca0 2024-04-25T00:20:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:03.781 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 msgr2=0x7fc0b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 0x7fc0b80a5160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006010 tx=0x7fc0c4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 msgr2=0x7fc0d0199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e9f0 tx=0x7fc0c800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 shutdown_connections 2024-04-25T00:20:03.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc0b80a2ca0 0x7fc0b80a5160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d0101350 0x7fc0d01a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0d01009b0 0x7fc0d019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 --2- 172.21.15.45:0/3070128563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0d0100050 0x7fc0d0199c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 >> 172.21.15.45:0/3070128563 conn(0x7fc0d00fbbf0 msgr2=0x7fc0d00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 shutdown_connections 2024-04-25T00:20:03.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:03.778+0000 7fc0d79a5640 1 -- 172.21.15.45:0/3070128563 wait complete. 2024-04-25T00:20:03.789 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-04-25T00:20:03.267469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600590,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":149,"num_read_kb":131,"num_write":344,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385868,"kb_used_data":11756,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555956,"statfs":{"total":1151940427776,"available":1151545298944,"internally_reserved":0,"allocated":12038144,"data_stored":7567909,"data_compressed":284160,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5723,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":116,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542178+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:19:39.542178+0000","last_peered":"2024-04-25T00:19:39.542178+0000","last_clean":"2024-04-25T00:19:39.542178+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:19:39.542178+0000","last_undegraded":"2024-04-25T00:19:39.542178+0000","last_fullsized":"2024-04-25T00:19:39.542178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541616+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:19:39.541616+0000","last_peered":"2024-04-25T00:19:39.541616+0000","last_clean":"2024-04-25T00:19:39.541616+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:19:39.541616+0000","last_undegraded":"2024-04-25T00:19:39.541616+0000","last_fullsized":"2024-04-25T00:19:39.541616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541497+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:19:39.541497+0000","last_peered":"2024-04-25T00:19:39.541497+0000","last_clean":"2024-04-25T00:19:39.541497+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:19:39.541497+0000","last_undegraded":"2024-04-25T00:19:39.541497+0000","last_fullsized":"2024-04-25T00:19:39.541497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542700+0000","last_change":"2024-04-25T00:19:33.880302+0000","last_active":"2024-04-25T00:19:39.542700+0000","last_peered":"2024-04-25T00:19:39.542700+0000","last_clean":"2024-04-25T00:19:39.542700+0000","last_became_active":"2024-04-25T00:19:33.879849+0000","last_became_peered":"2024-04-25T00:19:33.879849+0000","last_unstale":"2024-04-25T00:19:39.542700+0000","last_undegraded":"2024-04-25T00:19:39.542700+0000","last_fullsized":"2024-04-25T00:19:39.542700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:32.449680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045744+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:19:39.045744+0000","last_peered":"2024-04-25T00:19:39.045744+0000","last_clean":"2024-04-25T00:19:39.045744+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:19:39.045744+0000","last_undegraded":"2024-04-25T00:19:39.045744+0000","last_fullsized":"2024-04-25T00:19:39.045744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891976+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:19:38.891976+0000","last_peered":"2024-04-25T00:19:38.891976+0000","last_clean":"2024-04-25T00:19:38.891976+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:19:38.891976+0000","last_undegraded":"2024-04-25T00:19:38.891976+0000","last_fullsized":"2024-04-25T00:19:38.891976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541604+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:19:39.541604+0000","last_peered":"2024-04-25T00:19:39.541604+0000","last_clean":"2024-04-25T00:19:39.541604+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:19:39.541604+0000","last_undegraded":"2024-04-25T00:19:39.541604+0000","last_fullsized":"2024-04-25T00:19:39.541604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542714+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:19:39.542714+0000","last_peered":"2024-04-25T00:19:39.542714+0000","last_clean":"2024-04-25T00:19:39.542714+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:19:39.542714+0000","last_undegraded":"2024-04-25T00:19:39.542714+0000","last_fullsized":"2024-04-25T00:19:39.542714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542461+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:19:39.542461+0000","last_peered":"2024-04-25T00:19:39.542461+0000","last_clean":"2024-04-25T00:19:39.542461+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:19:39.542461+0000","last_undegraded":"2024-04-25T00:19:39.542461+0000","last_fullsized":"2024-04-25T00:19:39.542461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.892228+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:19:38.892228+0000","last_peered":"2024-04-25T00:19:38.892228+0000","last_clean":"2024-04-25T00:19:38.892228+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:19:38.892228+0000","last_undegraded":"2024-04-25T00:19:38.892228+0000","last_fullsized":"2024-04-25T00:19:38.892228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542824+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:19:39.542824+0000","last_peered":"2024-04-25T00:19:39.542824+0000","last_clean":"2024-04-25T00:19:39.542824+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:19:39.542824+0000","last_undegraded":"2024-04-25T00:19:39.542824+0000","last_fullsized":"2024-04-25T00:19:39.542824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542568+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:19:39.542568+0000","last_peered":"2024-04-25T00:19:39.542568+0000","last_clean":"2024-04-25T00:19:39.542568+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:19:39.542568+0000","last_undegraded":"2024-04-25T00:19:39.542568+0000","last_fullsized":"2024-04-25T00:19:39.542568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542324+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:19:39.542324+0000","last_peered":"2024-04-25T00:19:39.542324+0000","last_clean":"2024-04-25T00:19:39.542324+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:19:39.542324+0000","last_undegraded":"2024-04-25T00:19:39.542324+0000","last_fullsized":"2024-04-25T00:19:39.542324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541185+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:19:39.541185+0000","last_peered":"2024-04-25T00:19:39.541185+0000","last_clean":"2024-04-25T00:19:39.541185+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:19:39.541185+0000","last_undegraded":"2024-04-25T00:19:39.541185+0000","last_fullsized":"2024-04-25T00:19:39.541185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541199+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:19:39.541199+0000","last_peered":"2024-04-25T00:19:39.541199+0000","last_clean":"2024-04-25T00:19:39.541199+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:19:39.541199+0000","last_undegraded":"2024-04-25T00:19:39.541199+0000","last_fullsized":"2024-04-25T00:19:39.541199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541173+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:19:39.541173+0000","last_peered":"2024-04-25T00:19:39.541173+0000","last_clean":"2024-04-25T00:19:39.541173+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:19:39.541173+0000","last_undegraded":"2024-04-25T00:19:39.541173+0000","last_fullsized":"2024-04-25T00:19:39.541173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542005+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:19:39.542005+0000","last_peered":"2024-04-25T00:19:39.542005+0000","last_clean":"2024-04-25T00:19:39.542005+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:19:39.542005+0000","last_undegraded":"2024-04-25T00:19:39.542005+0000","last_fullsized":"2024-04-25T00:19:39.542005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891843+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:19:38.891843+0000","last_peered":"2024-04-25T00:19:38.891843+0000","last_clean":"2024-04-25T00:19:38.891843+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:19:38.891843+0000","last_undegraded":"2024-04-25T00:19:38.891843+0000","last_fullsized":"2024-04-25T00:19:38.891843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891543+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:19:38.891543+0000","last_peered":"2024-04-25T00:19:38.891543+0000","last_clean":"2024-04-25T00:19:38.891543+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:19:38.891543+0000","last_undegraded":"2024-04-25T00:19:38.891543+0000","last_fullsized":"2024-04-25T00:19:38.891543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891296+0000","last_change":"2024-04-25T00:19:37.914577+0000","last_active":"2024-04-25T00:19:38.891296+0000","last_peered":"2024-04-25T00:19:38.891296+0000","last_clean":"2024-04-25T00:19:38.891296+0000","last_became_active":"2024-04-25T00:19:37.914014+0000","last_became_peered":"2024-04-25T00:19:37.914014+0000","last_unstale":"2024-04-25T00:19:38.891296+0000","last_undegraded":"2024-04-25T00:19:38.891296+0000","last_fullsized":"2024-04-25T00:19:38.891296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:55.580561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891881+0000","last_change":"2024-04-25T00:19:32.869791+0000","last_active":"2024-04-25T00:19:38.891881+0000","last_peered":"2024-04-25T00:19:38.891881+0000","last_clean":"2024-04-25T00:19:38.891881+0000","last_became_active":"2024-04-25T00:19:32.869432+0000","last_became_peered":"2024-04-25T00:19:32.869432+0000","last_unstale":"2024-04-25T00:19:38.891881+0000","last_undegraded":"2024-04-25T00:19:38.891881+0000","last_fullsized":"2024-04-25T00:19:38.891881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:52.026951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543614+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:19:39.543614+0000","last_peered":"2024-04-25T00:19:39.543614+0000","last_clean":"2024-04-25T00:19:39.543614+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:19:39.543614+0000","last_undegraded":"2024-04-25T00:19:39.543614+0000","last_fullsized":"2024-04-25T00:19:39.543614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541546+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:19:39.541546+0000","last_peered":"2024-04-25T00:19:39.541546+0000","last_clean":"2024-04-25T00:19:39.541546+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:19:39.541546+0000","last_undegraded":"2024-04-25T00:19:39.541546+0000","last_fullsized":"2024-04-25T00:19:39.541546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543785+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:19:39.543785+0000","last_peered":"2024-04-25T00:19:39.543785+0000","last_clean":"2024-04-25T00:19:39.543785+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:19:39.543785+0000","last_undegraded":"2024-04-25T00:19:39.543785+0000","last_fullsized":"2024-04-25T00:19:39.543785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541382+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:19:39.541382+0000","last_peered":"2024-04-25T00:19:39.541382+0000","last_clean":"2024-04-25T00:19:39.541382+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:19:39.541382+0000","last_undegraded":"2024-04-25T00:19:39.541382+0000","last_fullsized":"2024-04-25T00:19:39.541382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543053+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:19:39.543053+0000","last_peered":"2024-04-25T00:19:39.543053+0000","last_clean":"2024-04-25T00:19:39.543053+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:19:39.543053+0000","last_undegraded":"2024-04-25T00:19:39.543053+0000","last_fullsized":"2024-04-25T00:19:39.543053+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991690+0000","last_change":"2024-04-25T00:19:32.874816+0000","last_active":"2024-04-25T00:20:00.991690+0000","last_peered":"2024-04-25T00:20:00.991690+0000","last_clean":"2024-04-25T00:20:00.991690+0000","last_became_active":"2024-04-25T00:19:32.874180+0000","last_became_peered":"2024-04-25T00:19:32.874180+0000","last_unstale":"2024-04-25T00:20:00.991690+0000","last_undegraded":"2024-04-25T00:20:00.991690+0000","last_fullsized":"2024-04-25T00:20:00.991690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:38.437038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542061+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:19:39.542061+0000","last_peered":"2024-04-25T00:19:39.542061+0000","last_clean":"2024-04-25T00:19:39.542061+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:19:39.542061+0000","last_undegraded":"2024-04-25T00:19:39.542061+0000","last_fullsized":"2024-04-25T00:19:39.542061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542126+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:19:39.542126+0000","last_peered":"2024-04-25T00:19:39.542126+0000","last_clean":"2024-04-25T00:19:39.542126+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:19:39.542126+0000","last_undegraded":"2024-04-25T00:19:39.542126+0000","last_fullsized":"2024-04-25T00:19:39.542126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991117+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:20:00.991117+0000","last_peered":"2024-04-25T00:20:00.991117+0000","last_clean":"2024-04-25T00:20:00.991117+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:20:00.991117+0000","last_undegraded":"2024-04-25T00:20:00.991117+0000","last_fullsized":"2024-04-25T00:20:00.991117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541985+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:19:39.541985+0000","last_peered":"2024-04-25T00:19:39.541985+0000","last_clean":"2024-04-25T00:19:39.541985+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:19:39.541985+0000","last_undegraded":"2024-04-25T00:19:39.541985+0000","last_fullsized":"2024-04-25T00:19:39.541985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891537+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:19:38.891537+0000","last_peered":"2024-04-25T00:19:38.891537+0000","last_clean":"2024-04-25T00:19:38.891537+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:19:38.891537+0000","last_undegraded":"2024-04-25T00:19:38.891537+0000","last_fullsized":"2024-04-25T00:19:38.891537+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.963884+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:19:57.963884+0000","last_peered":"2024-04-25T00:19:57.963884+0000","last_clean":"2024-04-25T00:19:57.963884+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:19:57.963884+0000","last_undegraded":"2024-04-25T00:19:57.963884+0000","last_fullsized":"2024-04-25T00:19:57.963884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045745+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:19:39.045745+0000","last_peered":"2024-04-25T00:19:39.045745+0000","last_clean":"2024-04-25T00:19:39.045745+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:19:39.045745+0000","last_undegraded":"2024-04-25T00:19:39.045745+0000","last_fullsized":"2024-04-25T00:19:39.045745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891366+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:19:38.891366+0000","last_peered":"2024-04-25T00:19:38.891366+0000","last_clean":"2024-04-25T00:19:38.891366+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:19:38.891366+0000","last_undegraded":"2024-04-25T00:19:38.891366+0000","last_fullsized":"2024-04-25T00:19:38.891366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.971937+0000","last_change":"2024-04-25T00:19:32.869204+0000","last_active":"2024-04-25T00:19:58.971937+0000","last_peered":"2024-04-25T00:19:58.971937+0000","last_clean":"2024-04-25T00:19:58.971937+0000","last_became_active":"2024-04-25T00:19:32.868826+0000","last_became_peered":"2024-04-25T00:19:32.868826+0000","last_unstale":"2024-04-25T00:19:58.971937+0000","last_undegraded":"2024-04-25T00:19:58.971937+0000","last_fullsized":"2024-04-25T00:19:58.971937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:01.439105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542054+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:19:39.542054+0000","last_peered":"2024-04-25T00:19:39.542054+0000","last_clean":"2024-04-25T00:19:39.542054+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:19:39.542054+0000","last_undegraded":"2024-04-25T00:19:39.542054+0000","last_fullsized":"2024-04-25T00:19:39.542054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541896+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:19:39.541896+0000","last_peered":"2024-04-25T00:19:39.541896+0000","last_clean":"2024-04-25T00:19:39.541896+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:19:39.541896+0000","last_undegraded":"2024-04-25T00:19:39.541896+0000","last_fullsized":"2024-04-25T00:19:39.541896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.972710+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:19:58.972710+0000","last_peered":"2024-04-25T00:19:58.972710+0000","last_clean":"2024-04-25T00:19:58.972710+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:19:58.972710+0000","last_undegraded":"2024-04-25T00:19:58.972710+0000","last_fullsized":"2024-04-25T00:19:58.972710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542261+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:19:39.542261+0000","last_peered":"2024-04-25T00:19:39.542261+0000","last_clean":"2024-04-25T00:19:39.542261+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:19:39.542261+0000","last_undegraded":"2024-04-25T00:19:39.542261+0000","last_fullsized":"2024-04-25T00:19:39.542261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541608+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:19:39.541608+0000","last_peered":"2024-04-25T00:19:39.541608+0000","last_clean":"2024-04-25T00:19:39.541608+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:19:39.541608+0000","last_undegraded":"2024-04-25T00:19:39.541608+0000","last_fullsized":"2024-04-25T00:19:39.541608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.981752+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:19:59.981752+0000","last_peered":"2024-04-25T00:19:59.981752+0000","last_clean":"2024-04-25T00:19:59.981752+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:19:59.981752+0000","last_undegraded":"2024-04-25T00:19:59.981752+0000","last_fullsized":"2024-04-25T00:19:59.981752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891176+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:19:38.891176+0000","last_peered":"2024-04-25T00:19:38.891176+0000","last_clean":"2024-04-25T00:19:38.891176+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:19:38.891176+0000","last_undegraded":"2024-04-25T00:19:38.891176+0000","last_fullsized":"2024-04-25T00:19:38.891176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543105+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:19:39.543105+0000","last_peered":"2024-04-25T00:19:39.543105+0000","last_clean":"2024-04-25T00:19:39.543105+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:19:39.543105+0000","last_undegraded":"2024-04-25T00:19:39.543105+0000","last_fullsized":"2024-04-25T00:19:39.543105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961812+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:19:57.961812+0000","last_peered":"2024-04-25T00:19:57.961812+0000","last_clean":"2024-04-25T00:19:57.961812+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:19:57.961812+0000","last_undegraded":"2024-04-25T00:19:57.961812+0000","last_fullsized":"2024-04-25T00:19:57.961812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542662+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:19:39.542662+0000","last_peered":"2024-04-25T00:19:39.542662+0000","last_clean":"2024-04-25T00:19:39.542662+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:19:39.542662+0000","last_undegraded":"2024-04-25T00:19:39.542662+0000","last_fullsized":"2024-04-25T00:19:39.542662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891025+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:19:38.891025+0000","last_peered":"2024-04-25T00:19:38.891025+0000","last_clean":"2024-04-25T00:19:38.891025+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:19:38.891025+0000","last_undegraded":"2024-04-25T00:19:38.891025+0000","last_fullsized":"2024-04-25T00:19:38.891025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961669+0000","last_change":"2024-04-25T00:19:32.874259+0000","last_active":"2024-04-25T00:19:57.961669+0000","last_peered":"2024-04-25T00:19:57.961669+0000","last_clean":"2024-04-25T00:19:57.961669+0000","last_became_active":"2024-04-25T00:19:32.873885+0000","last_became_peered":"2024-04-25T00:19:32.873885+0000","last_unstale":"2024-04-25T00:19:57.961669+0000","last_undegraded":"2024-04-25T00:19:57.961669+0000","last_fullsized":"2024-04-25T00:19:57.961669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:45.132551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541629+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:19:39.541629+0000","last_peered":"2024-04-25T00:19:39.541629+0000","last_clean":"2024-04-25T00:19:39.541629+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:19:39.541629+0000","last_undegraded":"2024-04-25T00:19:39.541629+0000","last_fullsized":"2024-04-25T00:19:39.541629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542880+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:19:39.542880+0000","last_peered":"2024-04-25T00:19:39.542880+0000","last_clean":"2024-04-25T00:19:39.542880+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:19:39.542880+0000","last_undegraded":"2024-04-25T00:19:39.542880+0000","last_fullsized":"2024-04-25T00:19:39.542880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.959859+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:19:57.959859+0000","last_peered":"2024-04-25T00:19:57.959859+0000","last_clean":"2024-04-25T00:19:57.959859+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:19:57.959859+0000","last_undegraded":"2024-04-25T00:19:57.959859+0000","last_fullsized":"2024-04-25T00:19:57.959859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542194+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:19:39.542194+0000","last_peered":"2024-04-25T00:19:39.542194+0000","last_clean":"2024-04-25T00:19:39.542194+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:19:39.542194+0000","last_undegraded":"2024-04-25T00:19:39.542194+0000","last_fullsized":"2024-04-25T00:19:39.542194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891265+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:19:38.891265+0000","last_peered":"2024-04-25T00:19:38.891265+0000","last_clean":"2024-04-25T00:19:38.891265+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:19:38.891265+0000","last_undegraded":"2024-04-25T00:19:38.891265+0000","last_fullsized":"2024-04-25T00:19:38.891265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961673+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:19:57.961673+0000","last_peered":"2024-04-25T00:19:57.961673+0000","last_clean":"2024-04-25T00:19:57.961673+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:19:57.961673+0000","last_undegraded":"2024-04-25T00:19:57.961673+0000","last_fullsized":"2024-04-25T00:19:57.961673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045944+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:19:39.045944+0000","last_peered":"2024-04-25T00:19:39.045944+0000","last_clean":"2024-04-25T00:19:39.045944+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:19:39.045944+0000","last_undegraded":"2024-04-25T00:19:39.045944+0000","last_fullsized":"2024-04-25T00:19:39.045944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542209+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:19:39.542209+0000","last_peered":"2024-04-25T00:19:39.542209+0000","last_clean":"2024-04-25T00:19:39.542209+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:19:39.542209+0000","last_undegraded":"2024-04-25T00:19:39.542209+0000","last_fullsized":"2024-04-25T00:19:39.542209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:01.841927+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:20:01.841927+0000","last_peered":"2024-04-25T00:20:01.841927+0000","last_clean":"2024-04-25T00:20:01.841927+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:20:01.841927+0000","last_undegraded":"2024-04-25T00:20:01.841927+0000","last_fullsized":"2024-04-25T00:20:01.841927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542087+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:19:39.542087+0000","last_peered":"2024-04-25T00:19:39.542087+0000","last_clean":"2024-04-25T00:19:39.542087+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:19:39.542087+0000","last_undegraded":"2024-04-25T00:19:39.542087+0000","last_fullsized":"2024-04-25T00:19:39.542087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541932+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:19:39.541932+0000","last_peered":"2024-04-25T00:19:39.541932+0000","last_clean":"2024-04-25T00:19:39.541932+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:19:39.541932+0000","last_undegraded":"2024-04-25T00:19:39.541932+0000","last_fullsized":"2024-04-25T00:19:39.541932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.981411+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:19:59.981411+0000","last_peered":"2024-04-25T00:19:59.981411+0000","last_clean":"2024-04-25T00:19:59.981411+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:19:59.981411+0000","last_undegraded":"2024-04-25T00:19:59.981411+0000","last_fullsized":"2024-04-25T00:19:59.981411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542158+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:19:39.542158+0000","last_peered":"2024-04-25T00:19:39.542158+0000","last_clean":"2024-04-25T00:19:39.542158+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:19:39.542158+0000","last_undegraded":"2024-04-25T00:19:39.542158+0000","last_fullsized":"2024-04-25T00:19:39.542158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891681+0000","last_change":"2024-04-25T00:19:37.915144+0000","last_active":"2024-04-25T00:19:38.891681+0000","last_peered":"2024-04-25T00:19:38.891681+0000","last_clean":"2024-04-25T00:19:38.891681+0000","last_became_active":"2024-04-25T00:19:37.914736+0000","last_became_peered":"2024-04-25T00:19:37.914736+0000","last_unstale":"2024-04-25T00:19:38.891681+0000","last_undegraded":"2024-04-25T00:19:38.891681+0000","last_fullsized":"2024-04-25T00:19:38.891681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.900136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.980949+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:19:59.980949+0000","last_peered":"2024-04-25T00:19:59.980949+0000","last_clean":"2024-04-25T00:19:59.980949+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:19:59.980949+0000","last_undegraded":"2024-04-25T00:19:59.980949+0000","last_fullsized":"2024-04-25T00:19:59.980949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541948+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:19:39.541948+0000","last_peered":"2024-04-25T00:19:39.541948+0000","last_clean":"2024-04-25T00:19:39.541948+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:19:39.541948+0000","last_undegraded":"2024-04-25T00:19:39.541948+0000","last_fullsized":"2024-04-25T00:19:39.541948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542716+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:19:39.542716+0000","last_peered":"2024-04-25T00:19:39.542716+0000","last_clean":"2024-04-25T00:19:39.542716+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:19:39.542716+0000","last_undegraded":"2024-04-25T00:19:39.542716+0000","last_fullsized":"2024-04-25T00:19:39.542716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.991123+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:20:00.991123+0000","last_peered":"2024-04-25T00:20:00.991123+0000","last_clean":"2024-04-25T00:20:00.991123+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:20:00.991123+0000","last_undegraded":"2024-04-25T00:20:00.991123+0000","last_fullsized":"2024-04-25T00:20:00.991123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541483+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:19:39.541483+0000","last_peered":"2024-04-25T00:19:39.541483+0000","last_clean":"2024-04-25T00:19:39.541483+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:19:39.541483+0000","last_undegraded":"2024-04-25T00:19:39.541483+0000","last_fullsized":"2024-04-25T00:19:39.541483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541945+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:19:39.541945+0000","last_peered":"2024-04-25T00:19:39.541945+0000","last_clean":"2024-04-25T00:19:39.541945+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:19:39.541945+0000","last_undegraded":"2024-04-25T00:19:39.541945+0000","last_fullsized":"2024-04-25T00:19:39.541945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.968016+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:20:00.968016+0000","last_peered":"2024-04-25T00:20:00.968016+0000","last_clean":"2024-04-25T00:20:00.968016+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:20:00.968016+0000","last_undegraded":"2024-04-25T00:20:00.968016+0000","last_fullsized":"2024-04-25T00:20:00.968016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541810+0000","last_change":"2024-04-25T00:19:33.880006+0000","last_active":"2024-04-25T00:19:39.541810+0000","last_peered":"2024-04-25T00:19:39.541810+0000","last_clean":"2024-04-25T00:19:39.541810+0000","last_became_active":"2024-04-25T00:19:33.879054+0000","last_became_peered":"2024-04-25T00:19:33.879054+0000","last_unstale":"2024-04-25T00:19:39.541810+0000","last_undegraded":"2024-04-25T00:19:39.541810+0000","last_fullsized":"2024-04-25T00:19:39.541810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:43.232276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045832+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:19:39.045832+0000","last_peered":"2024-04-25T00:19:39.045832+0000","last_clean":"2024-04-25T00:19:39.045832+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:19:39.045832+0000","last_undegraded":"2024-04-25T00:19:39.045832+0000","last_fullsized":"2024-04-25T00:19:39.045832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891695+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:19:38.891695+0000","last_peered":"2024-04-25T00:19:38.891695+0000","last_clean":"2024-04-25T00:19:38.891695+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:19:38.891695+0000","last_undegraded":"2024-04-25T00:19:38.891695+0000","last_fullsized":"2024-04-25T00:19:38.891695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.892108+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:19:38.892108+0000","last_peered":"2024-04-25T00:19:38.892108+0000","last_clean":"2024-04-25T00:19:38.892108+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:19:38.892108+0000","last_undegraded":"2024-04-25T00:19:38.892108+0000","last_fullsized":"2024-04-25T00:19:38.892108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542466+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:19:39.542466+0000","last_peered":"2024-04-25T00:19:39.542466+0000","last_clean":"2024-04-25T00:19:39.542466+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:19:39.542466+0000","last_undegraded":"2024-04-25T00:19:39.542466+0000","last_fullsized":"2024-04-25T00:19:39.542466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542583+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:19:39.542583+0000","last_peered":"2024-04-25T00:19:39.542583+0000","last_clean":"2024-04-25T00:19:39.542583+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:19:39.542583+0000","last_undegraded":"2024-04-25T00:19:39.542583+0000","last_fullsized":"2024-04-25T00:19:39.542583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891818+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:19:38.891818+0000","last_peered":"2024-04-25T00:19:38.891818+0000","last_clean":"2024-04-25T00:19:38.891818+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:19:38.891818+0000","last_undegraded":"2024-04-25T00:19:38.891818+0000","last_fullsized":"2024-04-25T00:19:38.891818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542458+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:19:39.542458+0000","last_peered":"2024-04-25T00:19:39.542458+0000","last_clean":"2024-04-25T00:19:39.542458+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:19:39.542458+0000","last_undegraded":"2024-04-25T00:19:39.542458+0000","last_fullsized":"2024-04-25T00:19:39.542458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542189+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:19:39.542189+0000","last_peered":"2024-04-25T00:19:39.542189+0000","last_clean":"2024-04-25T00:19:39.542189+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:19:39.542189+0000","last_undegraded":"2024-04-25T00:19:39.542189+0000","last_fullsized":"2024-04-25T00:19:39.542189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.974682+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:19:58.974682+0000","last_peered":"2024-04-25T00:19:58.974682+0000","last_clean":"2024-04-25T00:19:58.974682+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:19:58.974682+0000","last_undegraded":"2024-04-25T00:19:58.974682+0000","last_fullsized":"2024-04-25T00:19:58.974682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542723+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:19:39.542723+0000","last_peered":"2024-04-25T00:19:39.542723+0000","last_clean":"2024-04-25T00:19:39.542723+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:19:39.542723+0000","last_undegraded":"2024-04-25T00:19:39.542723+0000","last_fullsized":"2024-04-25T00:19:39.542723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542915+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:19:39.542915+0000","last_peered":"2024-04-25T00:19:39.542915+0000","last_clean":"2024-04-25T00:19:39.542915+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:19:39.542915+0000","last_undegraded":"2024-04-25T00:19:39.542915+0000","last_fullsized":"2024-04-25T00:19:39.542915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.982155+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:19:59.982155+0000","last_peered":"2024-04-25T00:19:59.982155+0000","last_clean":"2024-04-25T00:19:59.982155+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:19:59.982155+0000","last_undegraded":"2024-04-25T00:19:59.982155+0000","last_fullsized":"2024-04-25T00:19:59.982155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541817+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:19:39.541817+0000","last_peered":"2024-04-25T00:19:39.541817+0000","last_clean":"2024-04-25T00:19:39.541817+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:19:39.541817+0000","last_undegraded":"2024-04-25T00:19:39.541817+0000","last_fullsized":"2024-04-25T00:19:39.541817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541827+0000","last_change":"2024-04-25T00:19:37.967364+0000","last_active":"2024-04-25T00:19:39.541827+0000","last_peered":"2024-04-25T00:19:39.541827+0000","last_clean":"2024-04-25T00:19:39.541827+0000","last_became_active":"2024-04-25T00:19:37.966868+0000","last_became_peered":"2024-04-25T00:19:37.966868+0000","last_unstale":"2024-04-25T00:19:39.541827+0000","last_undegraded":"2024-04-25T00:19:39.541827+0000","last_fullsized":"2024-04-25T00:19:39.541827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:52:39.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541450+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:19:39.541450+0000","last_peered":"2024-04-25T00:19:39.541450+0000","last_clean":"2024-04-25T00:19:39.541450+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:19:39.541450+0000","last_undegraded":"2024-04-25T00:19:39.541450+0000","last_fullsized":"2024-04-25T00:19:39.541450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:58.974295+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:19:58.974295+0000","last_peered":"2024-04-25T00:19:58.974295+0000","last_clean":"2024-04-25T00:19:58.974295+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:19:58.974295+0000","last_undegraded":"2024-04-25T00:19:58.974295+0000","last_fullsized":"2024-04-25T00:19:58.974295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541762+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:19:39.541762+0000","last_peered":"2024-04-25T00:19:39.541762+0000","last_clean":"2024-04-25T00:19:39.541762+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:19:39.541762+0000","last_undegraded":"2024-04-25T00:19:39.541762+0000","last_fullsized":"2024-04-25T00:19:39.541762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.990678+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:20:00.990678+0000","last_peered":"2024-04-25T00:20:00.990678+0000","last_clean":"2024-04-25T00:20:00.990678+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:20:00.990678+0000","last_undegraded":"2024-04-25T00:20:00.990678+0000","last_fullsized":"2024-04-25T00:20:00.990678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542396+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:19:39.542396+0000","last_peered":"2024-04-25T00:19:39.542396+0000","last_clean":"2024-04-25T00:19:39.542396+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:19:39.542396+0000","last_undegraded":"2024-04-25T00:19:39.542396+0000","last_fullsized":"2024-04-25T00:19:39.542396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542205+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:19:39.542205+0000","last_peered":"2024-04-25T00:19:39.542205+0000","last_clean":"2024-04-25T00:19:39.542205+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:19:39.542205+0000","last_undegraded":"2024-04-25T00:19:39.542205+0000","last_fullsized":"2024-04-25T00:19:39.542205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891705+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:19:38.891705+0000","last_peered":"2024-04-25T00:19:38.891705+0000","last_clean":"2024-04-25T00:19:38.891705+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:19:38.891705+0000","last_undegraded":"2024-04-25T00:19:38.891705+0000","last_fullsized":"2024-04-25T00:19:38.891705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543492+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:19:39.543492+0000","last_peered":"2024-04-25T00:19:39.543492+0000","last_clean":"2024-04-25T00:19:39.543492+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:19:39.543492+0000","last_undegraded":"2024-04-25T00:19:39.543492+0000","last_fullsized":"2024-04-25T00:19:39.543492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891162+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:19:38.891162+0000","last_peered":"2024-04-25T00:19:38.891162+0000","last_clean":"2024-04-25T00:19:38.891162+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:19:38.891162+0000","last_undegraded":"2024-04-25T00:19:38.891162+0000","last_fullsized":"2024-04-25T00:19:38.891162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.960962+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:19:57.960962+0000","last_peered":"2024-04-25T00:19:57.960962+0000","last_clean":"2024-04-25T00:19:57.960962+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:19:57.960962+0000","last_undegraded":"2024-04-25T00:19:57.960962+0000","last_fullsized":"2024-04-25T00:19:57.960962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541725+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:19:39.541725+0000","last_peered":"2024-04-25T00:19:39.541725+0000","last_clean":"2024-04-25T00:19:39.541725+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:19:39.541725+0000","last_undegraded":"2024-04-25T00:19:39.541725+0000","last_fullsized":"2024-04-25T00:19:39.541725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542977+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:19:39.542977+0000","last_peered":"2024-04-25T00:19:39.542977+0000","last_clean":"2024-04-25T00:19:39.542977+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:19:39.542977+0000","last_undegraded":"2024-04-25T00:19:39.542977+0000","last_fullsized":"2024-04-25T00:19:39.542977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.961745+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:19:57.961745+0000","last_peered":"2024-04-25T00:19:57.961745+0000","last_clean":"2024-04-25T00:19:57.961745+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:19:57.961745+0000","last_undegraded":"2024-04-25T00:19:57.961745+0000","last_fullsized":"2024-04-25T00:19:57.961745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.045667+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:19:39.045667+0000","last_peered":"2024-04-25T00:19:39.045667+0000","last_clean":"2024-04-25T00:19:39.045667+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:19:39.045667+0000","last_undegraded":"2024-04-25T00:19:39.045667+0000","last_fullsized":"2024-04-25T00:19:39.045667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891910+0000","last_change":"2024-04-25T00:19:37.915452+0000","last_active":"2024-04-25T00:19:38.891910+0000","last_peered":"2024-04-25T00:19:38.891910+0000","last_clean":"2024-04-25T00:19:38.891910+0000","last_became_active":"2024-04-25T00:19:37.915009+0000","last_became_peered":"2024-04-25T00:19:37.915009+0000","last_unstale":"2024-04-25T00:19:38.891910+0000","last_undegraded":"2024-04-25T00:19:38.891910+0000","last_fullsized":"2024-04-25T00:19:38.891910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:04.882939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:00.990152+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:20:00.990152+0000","last_peered":"2024-04-25T00:20:00.990152+0000","last_clean":"2024-04-25T00:20:00.990152+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:20:00.990152+0000","last_undegraded":"2024-04-25T00:20:00.990152+0000","last_fullsized":"2024-04-25T00:20:00.990152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541326+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:19:39.541326+0000","last_peered":"2024-04-25T00:19:39.541326+0000","last_clean":"2024-04-25T00:19:39.541326+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:19:39.541326+0000","last_undegraded":"2024-04-25T00:19:39.541326+0000","last_fullsized":"2024-04-25T00:19:39.541326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542680+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:19:39.542680+0000","last_peered":"2024-04-25T00:19:39.542680+0000","last_clean":"2024-04-25T00:19:39.542680+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:19:39.542680+0000","last_undegraded":"2024-04-25T00:19:39.542680+0000","last_fullsized":"2024-04-25T00:19:39.542680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.957340+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:19:59.957340+0000","last_peered":"2024-04-25T00:19:59.957340+0000","last_clean":"2024-04-25T00:19:59.957340+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:19:59.957340+0000","last_undegraded":"2024-04-25T00:19:59.957340+0000","last_fullsized":"2024-04-25T00:19:59.957340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543184+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:19:39.543184+0000","last_peered":"2024-04-25T00:19:39.543184+0000","last_clean":"2024-04-25T00:19:39.543184+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:19:39.543184+0000","last_undegraded":"2024-04-25T00:19:39.543184+0000","last_fullsized":"2024-04-25T00:19:39.543184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542625+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:19:39.542625+0000","last_peered":"2024-04-25T00:19:39.542625+0000","last_clean":"2024-04-25T00:19:39.542625+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:19:39.542625+0000","last_undegraded":"2024-04-25T00:19:39.542625+0000","last_fullsized":"2024-04-25T00:19:39.542625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:57.962110+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:19:57.962110+0000","last_peered":"2024-04-25T00:19:57.962110+0000","last_clean":"2024-04-25T00:19:57.962110+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:19:57.962110+0000","last_undegraded":"2024-04-25T00:19:57.962110+0000","last_fullsized":"2024-04-25T00:19:57.962110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542699+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:19:39.542699+0000","last_peered":"2024-04-25T00:19:39.542699+0000","last_clean":"2024-04-25T00:19:39.542699+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:19:39.542699+0000","last_undegraded":"2024-04-25T00:19:39.542699+0000","last_fullsized":"2024-04-25T00:19:39.542699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541453+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:19:39.541453+0000","last_peered":"2024-04-25T00:19:39.541453+0000","last_clean":"2024-04-25T00:19:39.541453+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:19:39.541453+0000","last_undegraded":"2024-04-25T00:19:39.541453+0000","last_fullsized":"2024-04-25T00:19:39.541453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:20:01.845521+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:20:01.845521+0000","last_peered":"2024-04-25T00:20:01.845521+0000","last_clean":"2024-04-25T00:20:01.845521+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:20:01.845521+0000","last_undegraded":"2024-04-25T00:20:01.845521+0000","last_fullsized":"2024-04-25T00:20:01.845521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542854+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:19:39.542854+0000","last_peered":"2024-04-25T00:19:39.542854+0000","last_clean":"2024-04-25T00:19:39.542854+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:19:39.542854+0000","last_undegraded":"2024-04-25T00:19:39.542854+0000","last_fullsized":"2024-04-25T00:19:39.542854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541295+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:19:39.541295+0000","last_peered":"2024-04-25T00:19:39.541295+0000","last_clean":"2024-04-25T00:19:39.541295+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:19:39.541295+0000","last_undegraded":"2024-04-25T00:19:39.541295+0000","last_fullsized":"2024-04-25T00:19:39.541295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:59.982994+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:19:59.982994+0000","last_peered":"2024-04-25T00:19:59.982994+0000","last_clean":"2024-04-25T00:19:59.982994+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:19:59.982994+0000","last_undegraded":"2024-04-25T00:19:59.982994+0000","last_fullsized":"2024-04-25T00:19:59.982994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543008+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:19:39.543008+0000","last_peered":"2024-04-25T00:19:39.543008+0000","last_clean":"2024-04-25T00:19:39.543008+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:19:39.543008+0000","last_undegraded":"2024-04-25T00:19:39.543008+0000","last_fullsized":"2024-04-25T00:19:39.543008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541306+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:19:39.541306+0000","last_peered":"2024-04-25T00:19:39.541306+0000","last_clean":"2024-04-25T00:19:39.541306+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:19:39.541306+0000","last_undegraded":"2024-04-25T00:19:39.541306+0000","last_fullsized":"2024-04-25T00:19:39.541306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542586+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:19:39.542586+0000","last_peered":"2024-04-25T00:19:39.542586+0000","last_clean":"2024-04-25T00:19:39.542586+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:19:39.542586+0000","last_undegraded":"2024-04-25T00:19:39.542586+0000","last_fullsized":"2024-04-25T00:19:39.542586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543217+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:19:39.543217+0000","last_peered":"2024-04-25T00:19:39.543217+0000","last_clean":"2024-04-25T00:19:39.543217+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:19:39.543217+0000","last_undegraded":"2024-04-25T00:19:39.543217+0000","last_fullsized":"2024-04-25T00:19:39.543217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541452+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:19:39.541452+0000","last_peered":"2024-04-25T00:19:39.541452+0000","last_clean":"2024-04-25T00:19:39.541452+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:19:39.541452+0000","last_undegraded":"2024-04-25T00:19:39.541452+0000","last_fullsized":"2024-04-25T00:19:39.541452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891500+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:19:38.891500+0000","last_peered":"2024-04-25T00:19:38.891500+0000","last_clean":"2024-04-25T00:19:38.891500+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:19:38.891500+0000","last_undegraded":"2024-04-25T00:19:38.891500+0000","last_fullsized":"2024-04-25T00:19:38.891500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891130+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:19:38.891130+0000","last_peered":"2024-04-25T00:19:38.891130+0000","last_clean":"2024-04-25T00:19:38.891130+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:19:38.891130+0000","last_undegraded":"2024-04-25T00:19:38.891130+0000","last_fullsized":"2024-04-25T00:19:38.891130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541374+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:19:39.541374+0000","last_peered":"2024-04-25T00:19:39.541374+0000","last_clean":"2024-04-25T00:19:39.541374+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:19:39.541374+0000","last_undegraded":"2024-04-25T00:19:39.541374+0000","last_fullsized":"2024-04-25T00:19:39.541374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542476+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:19:39.542476+0000","last_peered":"2024-04-25T00:19:39.542476+0000","last_clean":"2024-04-25T00:19:39.542476+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:19:39.542476+0000","last_undegraded":"2024-04-25T00:19:39.542476+0000","last_fullsized":"2024-04-25T00:19:39.542476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542106+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:19:39.542106+0000","last_peered":"2024-04-25T00:19:39.542106+0000","last_clean":"2024-04-25T00:19:39.542106+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:19:39.542106+0000","last_undegraded":"2024-04-25T00:19:39.542106+0000","last_fullsized":"2024-04-25T00:19:39.542106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:38.891636+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:19:38.891636+0000","last_peered":"2024-04-25T00:19:38.891636+0000","last_clean":"2024-04-25T00:19:38.891636+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:19:38.891636+0000","last_undegraded":"2024-04-25T00:19:38.891636+0000","last_fullsized":"2024-04-25T00:19:38.891636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.543595+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:19:39.543595+0000","last_peered":"2024-04-25T00:19:39.543595+0000","last_clean":"2024-04-25T00:19:39.543595+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:19:39.543595+0000","last_undegraded":"2024-04-25T00:19:39.543595+0000","last_fullsized":"2024-04-25T00:19:39.543595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541537+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:19:39.541537+0000","last_peered":"2024-04-25T00:19:39.541537+0000","last_clean":"2024-04-25T00:19:39.541537+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:19:39.541537+0000","last_undegraded":"2024-04-25T00:19:39.541537+0000","last_fullsized":"2024-04-25T00:19:39.541537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542861+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:19:39.542861+0000","last_peered":"2024-04-25T00:19:39.542861+0000","last_clean":"2024-04-25T00:19:39.542861+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:19:39.542861+0000","last_undegraded":"2024-04-25T00:19:39.542861+0000","last_fullsized":"2024-04-25T00:19:39.542861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.541848+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:19:39.541848+0000","last_peered":"2024-04-25T00:19:39.541848+0000","last_clean":"2024-04-25T00:19:39.541848+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:19:39.541848+0000","last_undegraded":"2024-04-25T00:19:39.541848+0000","last_fullsized":"2024-04-25T00:19:39.541848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542227+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:19:39.542227+0000","last_peered":"2024-04-25T00:19:39.542227+0000","last_clean":"2024-04-25T00:19:39.542227+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:19:39.542227+0000","last_undegraded":"2024-04-25T00:19:39.542227+0000","last_fullsized":"2024-04-25T00:19:39.542227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T00:19:39.542109+0000","last_change":"2024-04-25T00:19:33.879484+0000","last_active":"2024-04-25T00:19:39.542109+0000","last_peered":"2024-04-25T00:19:39.542109+0000","last_clean":"2024-04-25T00:19:39.542109+0000","last_became_active":"2024-04-25T00:19:33.879046+0000","last_became_peered":"2024-04-25T00:19:33.879046+0000","last_unstale":"2024-04-25T00:19:39.542109+0000","last_undegraded":"2024-04-25T00:19:39.542109+0000","last_fullsized":"2024-04-25T00:19:39.542109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:06:43.540889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284395,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10222,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":90575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":478407,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1073003,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":477999,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":478715,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1069375,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":477305,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":519438,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":477909,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":490020,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":477645,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":478272,"data_compressed":14639,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1069821,"data_compressed":50803,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":47423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:20:03.796 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:20:03.796 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T00:20:03.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T00:20:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:04.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:03 smithi045 ceph-mon[29194]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:04.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4120542987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:03 smithi122 ceph-mon[32959]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:03 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4120542987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:04.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:03 smithi071 ceph-mon[32667]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:20:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:03 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4120542987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:04 smithi122 ceph-mon[32959]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T00:20:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:04 smithi122 ceph-mon[32959]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:04 smithi122 ceph-mon[32959]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:04 smithi045 ceph-mon[29194]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T00:20:05.351 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:04 smithi045 ceph-mon[29194]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:04 smithi045 ceph-mon[29194]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:04 smithi071 ceph-mon[32667]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-25T00:20:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:04 smithi071 ceph-mon[32667]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:04 smithi071 ceph-mon[32667]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:20:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.887+0000 7fe792b2b700 1 -- 172.21.15.45:0/3330336923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 msgr2=0x7fe78c102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3330336923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c102700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe7800099c0 tx=0x7fe780009cd0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 -- 172.21.15.45:0/3330336923 shutdown_connections 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3330336923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 0x7fe78c10a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3330336923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c102700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3330336923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 -- 172.21.15.45:0/3330336923 >> 172.21.15.45:0/3330336923 conn(0x7fe78c0fd880 msgr2=0x7fe78c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 -- 172.21.15.45:0/3330336923 shutdown_connections 2024-04-25T00:20:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 -- 172.21.15.45:0/3330336923 wait complete. 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 Processor -- start 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.888+0000 7fe792b2b700 1 -- start start 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 0x7fe78c074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10deb0 con 0x7fe78c1022e0 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10dd30 con 0x7fe78c1034e0 2024-04-25T00:20:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe792b2b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c10e030 con 0x7fe78c103e80 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe7908c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe7908c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54326/0 (socket says 172.21.15.45:54326) 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe7908c7700 1 -- 172.21.15.45:0/3552295944 learned_addr learned my addr 172.21.15.45:0/3552295944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe7910c8700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 0x7fe78c074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 -- 172.21.15.45:0/3552295944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 msgr2=0x7fe78c074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 0x7fe78c074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 -- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 msgr2=0x7fe78c075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.889+0000 7fe78bfff700 1 -- 172.21.15.45:0/3552295944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c075410 con 0x7fe78c1034e0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe78bfff700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c074620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e9f0 tx=0x7fe77800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77800c9c0 con 0x7fe78c1034e0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c1a2560 con 0x7fe78c1034e0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77800cb60 con 0x7fe78c1034e0 2024-04-25T00:20:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778018680 con 0x7fe78c1034e0 2024-04-25T00:20:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.890+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1a2aa0 con 0x7fe78c1034e0 2024-04-25T00:20:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.891+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c0507a0 con 0x7fe78c1034e0 2024-04-25T00:20:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.892+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe778010430 con 0x7fe78c1034e0 2024-04-25T00:20:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.893+0000 7fe789ffb700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 0x7fe77c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:05.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.893+0000 7fe7908c7700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 0x7fe77c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:05.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.893+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe7780cf830 con 0x7fe78c1034e0 2024-04-25T00:20:05.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.893+0000 7fe7908c7700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 0x7fe77c0a5220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe780000c00 tx=0x7fe780005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:05.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:05.897+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe77809c3f0 con 0x7fe78c1034e0 2024-04-25T00:20:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.189+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe78c062e40 con 0x7fe78c1034e0 2024-04-25T00:20:06.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.194+0000 7fe789ffb700 1 -- 172.21.15.45:0/3552295944 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe77809bcf0 con 0x7fe78c1034e0 2024-04-25T00:20:06.196 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T00:20:06.196 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQC2oSlmBjd3CxAAE2jAKEU2C4ZvLzKapJEifA== 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 msgr2=0x7fe77c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 0x7fe77c0a5220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe780000c00 tx=0x7fe780005fb0 comp rx=0 tx=0).stop 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 msgr2=0x7fe78c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c074620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e9f0 tx=0x7fe77800ed00 comp rx=0 tx=0).stop 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 shutdown_connections 2024-04-25T00:20:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe78c103e80 0x7fe78c074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2d60 0x7fe77c0a5220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe78c1022e0 0x7fe78c075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 --2- 172.21.15.45:0/3552295944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe78c1034e0 0x7fe78c074620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.196+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 >> 172.21.15.45:0/3552295944 conn(0x7fe78c0fd880 msgr2=0x7fe78c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.197+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 shutdown_connections 2024-04-25T00:20:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:06.197+0000 7fe792b2b700 1 -- 172.21.15.45:0/3552295944 wait complete. 2024-04-25T00:20:06.974 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:20:06.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T00:20:06.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:20:07.014 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:20:07.014 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:20:07.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T00:20:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:06 smithi122 ceph-mon[32959]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-25T00:20:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3552295944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:06 smithi122 ceph-mon[32959]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:06 smithi122 ceph-mon[32959]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:20:07.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:06 smithi071 ceph-mon[32667]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-25T00:20:07.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3552295944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:06 smithi071 ceph-mon[32667]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:06 smithi071 ceph-mon[32667]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:20:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:06 smithi045 ceph-mon[29194]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-25T00:20:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3552295944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:06 smithi045 ceph-mon[29194]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:20:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:06 smithi045 ceph-mon[29194]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- 172.21.15.45:0/11781394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c107190 msgr2=0x7fbe4c06a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 --2- 172.21.15.45:0/11781394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c107190 0x7fbe4c06a9e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009990 tx=0x7fbe48009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- 172.21.15.45:0/11781394 shutdown_connections 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 --2- 172.21.15.45:0/11781394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 --2- 172.21.15.45:0/11781394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c107190 0x7fbe4c06a9e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 --2- 172.21.15.45:0/11781394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 0x7fbe4c06b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- 172.21.15.45:0/11781394 >> 172.21.15.45:0/11781394 conn(0x7fbe4c0781d0 msgr2=0x7fbe4c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- 172.21.15.45:0/11781394 shutdown_connections 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- 172.21.15.45:0/11781394 wait complete. 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 Processor -- start 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.123+0000 7fbe54583700 1 -- start start 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 0x7fbe4c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 0x7fbe4c1a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c118b50 con 0x7fbe4c104ca0 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1189d0 con 0x7fbe4c107190 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe54583700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c118cd0 con 0x7fbe4c06af20 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe51b1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe5231f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 0x7fbe4c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 0x7fbe4c1a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe51b1e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54342/0 (socket says 172.21.15.45:54342) 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe51b1e700 1 -- 172.21.15.45:0/496605648 learned_addr learned my addr 172.21.15.45:0/496605648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 -- 172.21.15.45:0/496605648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 msgr2=0x7fbe4c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 0x7fbe4c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 -- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 msgr2=0x7fbe4c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 -- 172.21.15.45:0/496605648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c1a3f40 con 0x7fbe4c107190 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe51b1e700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe52b20700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 0x7fbe4c1a3800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400efb0 tx=0x7fbe4400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.124+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44003a00 con 0x7fbe4c107190 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.125+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe44003ba0 con 0x7fbe4c107190 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.125+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c1a4230 con 0x7fbe4c107190 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.125+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44004850 con 0x7fbe4c107190 2024-04-25T00:20:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.125+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c1abab0 con 0x7fbe4c107190 2024-04-25T00:20:09.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.125+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4c068f20 con 0x7fbe4c107190 2024-04-25T00:20:09.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.127+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe44012050 con 0x7fbe4c107190 2024-04-25T00:20:09.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.127+0000 7fbe437fe700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 0x7fbe380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:09.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.127+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbe440cef40 con 0x7fbe4c107190 2024-04-25T00:20:09.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.127+0000 7fbe5231f700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 0x7fbe380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:09.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.128+0000 7fbe5231f700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 0x7fbe380a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009750 tx=0x7fbe48005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:09.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.131+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe44022060 con 0x7fbe4c107190 2024-04-25T00:20:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:08 smithi122 ceph-mon[32959]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2024-04-25T00:20:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:08 smithi071 ceph-mon[32667]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2024-04-25T00:20:09.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.395+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbe4c1abc80 con 0x7fbe4c107190 2024-04-25T00:20:09.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:08 smithi045 ceph-mon[29194]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 20 op/s 2024-04-25T00:20:09.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.400+0000 7fbe437fe700 1 -- 172.21.15.45:0/496605648 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7fbe44010040 con 0x7fbe4c107190 2024-04-25T00:20:09.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:09.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 msgr2=0x7fbe380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 0x7fbe380a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009750 tx=0x7fbe48005960 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 msgr2=0x7fbe4c1a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 0x7fbe4c1a3800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400efb0 tx=0x7fbe4400c3a0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 shutdown_connections 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c06af20 0x7fbe4c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbe380a2cb0 0x7fbe380a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe4c107190 0x7fbe4c1a3800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 --2- 172.21.15.45:0/496605648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe4c104ca0 0x7fbe4c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 >> 172.21.15.45:0/496605648 conn(0x7fbe4c0781d0 msgr2=0x7fbe4c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 shutdown_connections 2024-04-25T00:20:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:09.404+0000 7fbe54583700 1 -- 172.21.15.45:0/496605648 wait complete. 2024-04-25T00:20:10.038 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3305016128},{"type":"v1","addr":"172.21.15.45:6801","nonce":3305016128}]},"active_addr":"172.21.15.45:6801/3305016128","active_change":"2024-04-25T00:12:25.175567+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3329362383}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":937209566}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1588770344}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2421422338}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3646793309}]}]} 2024-04-25T00:20:10.045 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T00:20:10.046 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T00:20:10.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:20:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:10.312 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:10 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/496605648' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:20:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:10 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/496605648' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:20:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:10 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/496605648' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:20:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:11 smithi122 ceph-mon[32959]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-25T00:20:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:11 smithi071 ceph-mon[32667]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-25T00:20:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:11 smithi045 ceph-mon[29194]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-25T00:20:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 -- 172.21.15.45:0/622237510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc102ec0 msgr2=0x7fc4fc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:11.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 --2- 172.21.15.45:0/622237510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc106f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0099c0 tx=0x7fc4ec009cd0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 -- 172.21.15.45:0/622237510 shutdown_connections 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 --2- 172.21.15.45:0/622237510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 0x7fc4fc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 --2- 172.21.15.45:0/622237510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 --2- 172.21.15.45:0/622237510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc106f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 -- 172.21.15.45:0/622237510 >> 172.21.15.45:0/622237510 conn(0x7fc4fc0793e0 msgr2=0x7fc4fc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:11.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 -- 172.21.15.45:0/622237510 shutdown_connections 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.959+0000 7fc50371c700 1 -- 172.21.15.45:0/622237510 wait complete. 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 Processor -- start 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 -- start start 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 0x7fc4fc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:11.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1125c0 con 0x7fc4fc1075e0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc112440 con 0x7fc4fc102ec0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc50371c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc112740 con 0x7fc4fc102510 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc501cb9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc501cb9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54350/0 (socket says 172.21.15.45:54350) 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc501cb9700 1 -- 172.21.15.45:0/960695284 learned_addr learned my addr 172.21.15.45:0/960695284 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc5014b8700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 0x7fc4fc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.960+0000 7fc500cb7700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 -- 172.21.15.45:0/960695284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 msgr2=0x7fc4fc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 0x7fc4fc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 -- 172.21.15.45:0/960695284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc102ec0 msgr2=0x7fc4fc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 -- 172.21.15.45:0/960695284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc198fc0 con 0x7fc4fc1075e0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc501cb9700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4007ee0 tx=0x7fc4f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4004bb0 con 0x7fc4fc1075e0 2024-04-25T00:20:11.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f4004d50 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc1992b0 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4005960 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.961+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc1a5570 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.962+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc068f20 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.963+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc4f401a020 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.964+0000 7fc4f27fc700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 0x7fc4e80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.964+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc4f40cea80 con 0x7fc4fc1075e0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.964+0000 7fc5014b8700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 0x7fc4e80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.964+0000 7fc5014b8700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 0x7fc4e80a51f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc1999a0 tx=0x7fc4f800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:11.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:11.968+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc4f4010070 con 0x7fc4fc1075e0 2024-04-25T00:20:12.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.173+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4fc02db30 con 0x7fc4fc1075e0 2024-04-25T00:20:12.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.174+0000 7fc4f27fc700 1 -- 172.21.15.45:0/960695284 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7fc4f409b640 con 0x7fc4fc1075e0 2024-04-25T00:20:12.175 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:12.176 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:20:12.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.176+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 msgr2=0x7fc4e80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:12.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.176+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 0x7fc4e80a51f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc1999a0 tx=0x7fc4f800b3c0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.176+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 msgr2=0x7fc4fc198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:12.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.176+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4007ee0 tx=0x7fc4f400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 shutdown_connections 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4fc102510 0x7fc4fc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc4e80a2d30 0x7fc4e80a51f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc1075e0 0x7fc4fc198860 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 --2- 172.21.15.45:0/960695284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4fc102ec0 0x7fc4fc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 >> 172.21.15.45:0/960695284 conn(0x7fc4fc0793e0 msgr2=0x7fc4fc1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 shutdown_connections 2024-04-25T00:20:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:12.177+0000 7fc50371c700 1 -- 172.21.15.45:0/960695284 wait complete. 2024-04-25T00:20:12.945 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T00:20:12.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:20:13.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:13 smithi122 ceph-mon[32959]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-25T00:20:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:13 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/960695284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:13.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:13 smithi045 ceph-mon[29194]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-25T00:20:13.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:13 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/960695284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:13 smithi071 ceph-mon[32667]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-25T00:20:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:13 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/960695284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 -- 172.21.15.45:0/902001606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300108410 msgr2=0x7fd3001087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 --2- 172.21.15.45:0/902001606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300108410 0x7fd3001087f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009990 tx=0x7fd2f0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 -- 172.21.15.45:0/902001606 shutdown_connections 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 --2- 172.21.15.45:0/902001606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300101620 0x7fd30010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 --2- 172.21.15.45:0/902001606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300108410 0x7fd3001087f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 --2- 172.21.15.45:0/902001606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd3001010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 -- 172.21.15.45:0/902001606 >> 172.21.15.45:0/902001606 conn(0x7fd3000763a0 msgr2=0x7fd3000767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 -- 172.21.15.45:0/902001606 shutdown_connections 2024-04-25T00:20:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.959+0000 7fd3055be700 1 -- 172.21.15.45:0/902001606 wait complete. 2024-04-25T00:20:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 Processor -- start 2024-04-25T00:20:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 -- start start 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd30019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 0x7fd300198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd300112510 con 0x7fd300101620 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd300112390 con 0x7fd300108410 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd3055be700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd300112690 con 0x7fd300100c60 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40918/0 (socket says 172.21.15.45:40918) 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 -- 172.21.15.45:0/2934455008 learned_addr learned my addr 172.21.15.45:0/2934455008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2ff7fe700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 0x7fd300198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2feffd700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd30019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 -- 172.21.15.45:0/2934455008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 msgr2=0x7fd30019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd30019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 -- 172.21.15.45:0/2934455008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 msgr2=0x7fd300198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 0x7fd300198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.960+0000 7fd2fe7fc700 1 -- 172.21.15.45:0/2934455008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd300198ff0 con 0x7fd300101620 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2feffd700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd30019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2ff7fe700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 0x7fd300198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2fe7fc700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400c9b0 tx=0x7fd2f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4007b00 con 0x7fd300101620 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f4007ca0 con 0x7fd300101620 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3001992e0 con 0x7fd300101620 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4018780 con 0x7fd300101620 2024-04-25T00:20:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.961+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3001a55c0 con 0x7fd300101620 2024-04-25T00:20:14.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.962+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30006a400 con 0x7fd300101620 2024-04-25T00:20:14.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.963+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2f400f450 con 0x7fd300101620 2024-04-25T00:20:14.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.964+0000 7fd2e7fff700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 0x7fd2e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:14.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.964+0000 7fd2feffd700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 0x7fd2e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:14.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.964+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd2f40cf480 con 0x7fd300101620 2024-04-25T00:20:14.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.964+0000 7fd2feffd700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 0x7fd2e80a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000b5c0 tx=0x7fd2f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:14.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:14.968+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd2f4014030 con 0x7fd300101620 2024-04-25T00:20:15.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:15 smithi045 ceph-mon[29194]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-25T00:20:15.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.178+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3001a5790 con 0x7fd300101620 2024-04-25T00:20:15.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.179+0000 7fd2e7fff700 1 -- 172.21.15.45:0/2934455008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24116 (secure 0 0 0) 0x7fd2f409c040 con 0x7fd300101620 2024-04-25T00:20:15.180 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:20:15.180 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:19:38.264259+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:20:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.181+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 msgr2=0x7fd2e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.181+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 0x7fd2e80a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000b5c0 tx=0x7fd2f0005fb0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.181+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 msgr2=0x7fd30019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.181+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400c9b0 tx=0x7fd2f400cd70 comp rx=0 tx=0).stop 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 shutdown_connections 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd300100c60 0x7fd30019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd2e80a2cb0 0x7fd2e80a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd300108410 0x7fd300198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 --2- 172.21.15.45:0/2934455008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd300101620 0x7fd30019eb80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 >> 172.21.15.45:0/2934455008 conn(0x7fd3000763a0 msgr2=0x7fd3000ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 shutdown_connections 2024-04-25T00:20:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:15.182+0000 7fd3055be700 1 -- 172.21.15.45:0/2934455008 wait complete. 2024-04-25T00:20:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:15 smithi122 ceph-mon[32959]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-25T00:20:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:15 smithi071 ceph-mon[32667]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2024-04-25T00:20:15.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T00:20:15.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T00:20:15.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T00:20:15.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T00:20:15.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T00:20:15.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T00:20:15.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T00:20:15.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T00:20:15.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T00:20:15.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T00:20:15.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T00:20:15.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T00:20:16.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:16 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2934455008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:16.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:16 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2934455008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:16 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2934455008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:20:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.723 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.724 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.731 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:16.731 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:17.119 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:17 smithi045 ceph-mon[29194]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-25T00:20:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:17 smithi122 ceph-mon[32959]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-25T00:20:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:17 smithi071 ceph-mon[32667]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 17 op/s 2024-04-25T00:20:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:19 smithi045 ceph-mon[29194]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-25T00:20:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:19 smithi122 ceph-mon[32959]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-25T00:20:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:19 smithi071 ceph-mon[32667]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-25T00:20:21.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:21 smithi045 ceph-mon[29194]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:20:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:21 smithi122 ceph-mon[32959]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:20:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:21 smithi071 ceph-mon[32667]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:20:23.286 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:23 smithi045 ceph-mon[29194]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-25T00:20:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:23 smithi122 ceph-mon[32959]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-25T00:20:23.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:23 smithi071 ceph-mon[32667]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-25T00:20:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:25 smithi045 ceph-mon[29194]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T00:20:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:25 smithi122 ceph-mon[32959]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T00:20:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:25 smithi071 ceph-mon[32667]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T00:20:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:20:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:20:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T00:20:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:20:26.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T00:20:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-25T00:20:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-04-25T00:20:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:20:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:20:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T00:20:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:20:26.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T00:20:26.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-25T00:20:26.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-04-25T00:20:26.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:20:26.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-25T00:20:26.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-04-25T00:20:26.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]: dispatch 2024-04-25T00:20:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-25T00:20:27.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-25T00:20:27.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-25T00:20:27.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-04-25T00:20:27.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:27 smithi122 ceph-mon[32959]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:20:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T00:20:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-25T00:20:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-25T00:20:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-25T00:20:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-04-25T00:20:27.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:27 smithi071 ceph-mon[32667]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:20:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:20:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-25T00:20:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-04-25T00:20:27.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 10]}]': finished 2024-04-25T00:20:27.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:27 smithi045 ceph-mon[29194]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:20:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:28 smithi045 ceph-mon[29194]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:20:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:28 smithi122 ceph-mon[32959]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:20:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:28 smithi071 ceph-mon[32667]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:20:28.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.725+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/393414870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee0102e90 msgr2=0x7f3ee0106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.725+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/393414870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee0102e90 0x7f3ee0106e90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009990 tx=0x7f3ed4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/393414870 shutdown_connections 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/393414870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ee0107570 0x7f3ee010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/393414870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee01024e0 0x7f3ee01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/393414870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee0102e90 0x7f3ee0106e90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/393414870 >> 172.21.15.45:0/393414870 conn(0x7f3ee0079390 msgr2=0x7f3ee00797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/393414870 shutdown_connections 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/393414870 wait complete. 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 Processor -- start 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.726+0000 7f3ee4ef8700 1 -- start start 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 0x7f3ee019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee0102e90 0x7f3ee019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ee0107570 0x7f3ee0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee0112530 con 0x7f3ee01024e0 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01123b0 con 0x7f3ee0102e90 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ee4ef8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee01126b0 con 0x7f3ee0107570 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 0x7f3ee019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 0x7f3ee019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45798/0 (socket says 172.21.15.45:45798) 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 -- 172.21.15.45:0/51953228 learned_addr learned my addr 172.21.15.45:0/51953228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3eddd9b700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee0102e90 0x7f3ee019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3eded9d700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ee0107570 0x7f3ee0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 -- 172.21.15.45:0/51953228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ee0107570 msgr2=0x7f3ee0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ee0107570 0x7f3ee0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 -- 172.21.15.45:0/51953228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee0102e90 msgr2=0x7f3ee019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee0102e90 0x7f3ee019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee0198ff0 con 0x7f3ee01024e0 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3eddd9b700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee0102e90 0x7f3ee019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:28.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ede59c700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 0x7f3ee019e690 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d950 tx=0x7f3ed000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.727+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0016070 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.728+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee01992e0 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.728+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0009b40 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.728+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed000f620 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.728+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee01a54f0 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.728+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ee00629c0 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.730+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ed0019070 con 0x7f3ee01024e0 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.730+0000 7f3ecf7fe700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3ec80a2cb0 0x7f3ec80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.730+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3ed00cf6b0 con 0x7f3ee01024e0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3eddd9b700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3ec80a2cb0 0x7f3ec80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3ecf7fe700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] conn(0x7f3ec80ad970 0x7f3ec80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ed0001d30 con 0x7f3ec80ad970 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f3ed00cfb00 con 0x7f3ee01024e0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3eded9d700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] conn(0x7f3ec80ad970 0x7f3ec80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3eddd9b700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3ec80a2cb0 0x7f3ec80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4006010 tx=0x7f3ed400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.731+0000 7f3eded9d700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] conn(0x7f3ec80ad970 0x7f3ec80afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.732+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== osd.6 v2:172.21.15.45:6818/4007679117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ed0001d30 con 0x7f3ec80ad970 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.748+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 --> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ee0062dc0 con 0x7f3ec80ad970 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.748+0000 7f3ecf7fe700 1 -- 172.21.15.45:0/51953228 <== osd.6 v2:172.21.15.45:6818/4007679117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3ee0062dc0 con 0x7f3ec80ad970 2024-04-25T00:20:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] conn(0x7f3ec80ad970 msgr2=0x7f3ec80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] conn(0x7f3ec80ad970 0x7f3ec80afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3ec80a2cb0 msgr2=0x7f3ec80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3ec80a2cb0 0x7f3ec80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4006010 tx=0x7f3ed400b540 comp rx=0 tx=0).stop 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 msgr2=0x7f3ee019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 --2- 172.21.15.45:0/51953228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ee01024e0 0x7f3ee019e690 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d950 tx=0x7f3ed000dd10 comp rx=0 tx=0).stop 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3eded9d700 1 -- 172.21.15.45:0/51953228 reap_dead start 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 shutdown_connections 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 >> 172.21.15.45:0/51953228 conn(0x7f3ee0079390 msgr2=0x7f3ee0101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 shutdown_connections 2024-04-25T00:20:28.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:24.749+0000 7f3ee4ef8700 1 -- 172.21.15.45:0/51953228 wait complete. 2024-04-25T00:20:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 -- 172.21.15.45:0/3830043125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad801082b0 msgr2=0x7fad80108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 --2- 172.21.15.45:0/3830043125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad801082b0 0x7fad80108690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009990 tx=0x7fad7c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 -- 172.21.15.45:0/3830043125 shutdown_connections 2024-04-25T00:20:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 --2- 172.21.15.45:0/3830043125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad8006b320 0x7fad8010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 --2- 172.21.15.45:0/3830043125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad801082b0 0x7fad80108690 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 --2- 172.21.15.45:0/3830043125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad8006a980 0x7fad8006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 -- 172.21.15.45:0/3830043125 >> 172.21.15.45:0/3830043125 conn(0x7fad80078100 msgr2=0x7fad80078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 -- 172.21.15.45:0/3830043125 shutdown_connections 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 -- 172.21.15.45:0/3830043125 wait complete. 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.850+0000 7fad87e2c700 1 Processor -- start 2024-04-25T00:20:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 -- start start 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad8006a980 0x7fad80101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 0x7fad80101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad801082b0 0x7fad80101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad80112510 con 0x7fad8006b320 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad80112390 con 0x7fad8006a980 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad87e2c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad80112690 con 0x7fad801082b0 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 0x7fad80101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 0x7fad80101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45812/0 (socket says 172.21.15.45:45812) 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 -- 172.21.15.45:0/471761262 learned_addr learned my addr 172.21.15.45:0/471761262 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad863c9700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad801082b0 0x7fad80101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad85bc8700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad8006a980 0x7fad80101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 -- 172.21.15.45:0/471761262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad801082b0 msgr2=0x7fad80101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad801082b0 0x7fad80101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 -- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad8006a980 msgr2=0x7fad80101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad8006a980 0x7fad80101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.851+0000 7fad853c7700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801a6f50 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad853c7700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 0x7fad80101770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c9b0 tx=0x7fad7000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70007b00 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70007ca0 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801a7240 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70018730 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.852+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a7780 con 0x7fad8006b320 2024-04-25T00:20:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.853+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad80062940 con 0x7fad8006b320 2024-04-25T00:20:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.854+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad700188d0 con 0x7fad8006b320 2024-04-25T00:20:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad76ffd700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fad6c0a2cb0 0x7fad6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad85bc8700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fad6c0a2cb0 0x7fad6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fad700cf960 con 0x7fad8006b320 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad76ffd700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] conn(0x7fad6c0add90 0x7fad6c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad6c0b0900 con 0x7fad6c0add90 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fad700cfd10 con 0x7fad8006b320 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad85bc8700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fad6c0a2cb0 0x7fad6c0a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009750 tx=0x7fad7c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.855+0000 7fad863c9700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] conn(0x7fad6c0add90 0x7fad6c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.856+0000 7fad863c9700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] conn(0x7fad6c0add90 0x7fad6c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.857+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== osd.1 v2:172.21.15.71:6800/3734145643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad6c0b0900 con 0x7fad6c0add90 2024-04-25T00:20:28.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.873+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 --> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad80062d40 con 0x7fad6c0add90 2024-04-25T00:20:28.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad76ffd700 1 -- 172.21.15.45:0/471761262 <== osd.1 v2:172.21.15.71:6800/3734145643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fad80062d40 con 0x7fad6c0add90 2024-04-25T00:20:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] conn(0x7fad6c0add90 msgr2=0x7fad6c0b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] conn(0x7fad6c0add90 0x7fad6c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fad6c0a2cb0 msgr2=0x7fad6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fad6c0a2cb0 0x7fad6c0a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009750 tx=0x7fad7c00b540 comp rx=0 tx=0).stop 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 msgr2=0x7fad80101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 --2- 172.21.15.45:0/471761262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8006b320 0x7fad80101770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c9b0 tx=0x7fad7000ccc0 comp rx=0 tx=0).stop 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad863c9700 1 -- 172.21.15.45:0/471761262 reap_dead start 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 shutdown_connections 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 >> 172.21.15.45:0/471761262 conn(0x7fad80078100 msgr2=0x7fad800ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 shutdown_connections 2024-04-25T00:20:28.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:28.874+0000 7fad87e2c700 1 -- 172.21.15.45:0/471761262 wait complete. 2024-04-25T00:20:29.246 INFO:teuthology.orchestra.run.smithi045.stdout:197568495666 2024-04-25T00:20:29.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T00:20:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:29 smithi122 ceph-mon[32959]: pgmap v322: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:20:29.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:29 smithi045 ceph-mon[29194]: pgmap v322: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:20:29.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:29 smithi071 ceph-mon[32667]: pgmap v322: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:20:29.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.788+0000 7f83dcd76700 1 -- 172.21.15.45:0/1518647068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 msgr2=0x7f83d810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.788+0000 7f83dcd76700 1 --2- 172.21.15.45:0/1518647068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d810e020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f83d000b0d0 tx=0x7f83d000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 -- 172.21.15.45:0/1518647068 shutdown_connections 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 --2- 172.21.15.45:0/1518647068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83d81013c0 0x7f83d81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 --2- 172.21.15.45:0/1518647068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d810e020 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 --2- 172.21.15.45:0/1518647068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83d8101d70 0x7f83d8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 -- 172.21.15.45:0/1518647068 >> 172.21.15.45:0/1518647068 conn(0x7f83d8078100 msgr2=0x7f83d8078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 -- 172.21.15.45:0/1518647068 shutdown_connections 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 -- 172.21.15.45:0/1518647068 wait complete. 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.789+0000 7f83dcd76700 1 Processor -- start 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 -- start start 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83d81013c0 0x7f83d8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83d8101d70 0x7f83d8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d81a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8111490 con 0x7f83d8106450 2024-04-25T00:20:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8111310 con 0x7f83d8101d70 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83dcd76700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8111610 con 0x7f83d81013c0 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d81a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d81a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45828/0 (socket says 172.21.15.45:45828) 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 -- 172.21.15.45:0/3876259514 learned_addr learned my addr 172.21.15.45:0/3876259514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d5d9b700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83d8101d70 0x7f83d8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d659c700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83d81013c0 0x7f83d8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 -- 172.21.15.45:0/3876259514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83d81013c0 msgr2=0x7f83d8199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83d81013c0 0x7f83d8199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 -- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83d8101d70 msgr2=0x7f83d8199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83d8101d70 0x7f83d8199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.790+0000 7f83d6d9d700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d81a4e30 con 0x7f83d8106450 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83d6d9d700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d81a48f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83d0006070 tx=0x7f83d0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0021070 con 0x7f83d8106450 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d81a5000 con 0x7f83d8106450 2024-04-25T00:20:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d81a54b0 con 0x7f83d8106450 2024-04-25T00:20:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0003c00 con 0x7f83d8106450 2024-04-25T00:20:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.791+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d001d450 con 0x7f83d8106450 2024-04-25T00:20:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.792+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83b8001030 con 0x7f83d8106450 2024-04-25T00:20:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.793+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f83d0003db0 con 0x7f83d8106450 2024-04-25T00:20:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.794+0000 7f83c77fe700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f83c00a2d30 0x7f83c00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.794+0000 7f83d659c700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f83c00a2d30 0x7f83c00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.794+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f83d00d0740 con 0x7f83d8106450 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83c77fe700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] conn(0x7f83c00ade10 0x7f83c00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83d000aea0 con 0x7f83c00ade10 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f83d00d0b90 con 0x7f83d8106450 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83d659c700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f83c00a2d30 0x7f83c00a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005950 tx=0x7f83c80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83d5d9b700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] conn(0x7f83c00ade10 0x7f83c00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.795+0000 7f83d5d9b700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] conn(0x7f83c00ade10 0x7f83c00b02b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:29.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.796+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== osd.10 v2:172.21.15.71:6824/2371437490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83d000aea0 con 0x7f83c00ade10 2024-04-25T00:20:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 --> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83b8002e60 con 0x7f83c00ade10 2024-04-25T00:20:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83c77fe700 1 -- 172.21.15.45:0/3876259514 <== osd.10 v2:172.21.15.71:6824/2371437490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f83b8002e60 con 0x7f83c00ade10 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] conn(0x7f83c00ade10 msgr2=0x7f83c00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] conn(0x7f83c00ade10 0x7f83c00b02b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f83c00a2d30 msgr2=0x7f83c00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f83c00a2d30 0x7f83c00a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005950 tx=0x7f83c80058e0 comp rx=0 tx=0).stop 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 msgr2=0x7f83d81a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.813+0000 7f83dcd76700 1 --2- 172.21.15.45:0/3876259514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83d8106450 0x7f83d81a48f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83d0006070 tx=0x7f83d0004d50 comp rx=0 tx=0).stop 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.814+0000 7f83d6d9d700 1 -- 172.21.15.45:0/3876259514 reap_dead start 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.814+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 shutdown_connections 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.814+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 >> 172.21.15.45:0/3876259514 conn(0x7f83d8078100 msgr2=0x7f83d80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.814+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 shutdown_connections 2024-04-25T00:20:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:29.814+0000 7f83dcd76700 1 -- 172.21.15.45:0/3876259514 wait complete. 2024-04-25T00:20:31.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:31 smithi045 ceph-mon[29194]: pgmap v323: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:20:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:31 smithi071 ceph-mon[32667]: pgmap v323: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:20:31.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:31 smithi122 ceph-mon[32959]: pgmap v323: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T00:20:31.661 INFO:teuthology.orchestra.run.smithi045.stdout:55834574924 2024-04-25T00:20:31.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T00:20:32.261 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:32 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:32.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:32 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:32 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:32.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 -- 172.21.15.45:0/3795544898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a960 msgr2=0x7f164c06adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 --2- 172.21.15.45:0/3795544898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a960 0x7f164c06adc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1648009990 tx=0x7f1648009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 -- 172.21.15.45:0/3795544898 shutdown_connections 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 --2- 172.21.15.45:0/3795544898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f164c1082b0 0x7f164c108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 --2- 172.21.15.45:0/3795544898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c06b300 0x7f164c10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 --2- 172.21.15.45:0/3795544898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06a960 0x7f164c06adc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 -- 172.21.15.45:0/3795544898 >> 172.21.15.45:0/3795544898 conn(0x7f164c0780e0 msgr2=0x7f164c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:32.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 -- 172.21.15.45:0/3795544898 shutdown_connections 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.771+0000 7f165452e700 1 -- 172.21.15.45:0/3795544898 wait complete. 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 Processor -- start 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 -- start start 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f164c06a960 0x7f164c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 0x7f164c101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c1082b0 0x7f164c101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f164c112510 con 0x7f164c06b300 2024-04-25T00:20:32.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f164c112390 con 0x7f164c1082b0 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f165452e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f164c112690 con 0x7f164c06a960 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 0x7f164c101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f16522ca700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f164c06a960 0x7f164c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 0x7f164c101770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45860/0 (socket says 172.21.15.45:45860) 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 -- 172.21.15.45:0/3155339333 learned_addr learned my addr 172.21.15.45:0/3155339333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1652acb700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c1082b0 0x7f164c101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 -- 172.21.15.45:0/3155339333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f164c06a960 msgr2=0x7f164c101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f164c06a960 0x7f164c101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 -- 172.21.15.45:0/3155339333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c1082b0 msgr2=0x7f164c101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164c1082b0 0x7f164c101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.772+0000 7f1651ac9700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c1a6fd0 con 0x7f164c06b300 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f1651ac9700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 0x7f164c101770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1648000c00 tx=0x7f164800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648027070 con 0x7f164c06b300 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1648003d70 con 0x7f164c06b300 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c1a7260 con 0x7f164c06b300 2024-04-25T00:20:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164800f460 con 0x7f164c06b300 2024-04-25T00:20:32.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.773+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c1a7710 con 0x7f164c06b300 2024-04-25T00:20:32.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.774+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f164c062940 con 0x7f164c06b300 2024-04-25T00:20:32.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.775+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f164802a070 con 0x7f164c06b300 2024-04-25T00:20:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16437fe700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f16380a2d30 0x7f16380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16522ca700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f16380a2d30 0x7f16380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f16480d0790 con 0x7f164c06b300 2024-04-25T00:20:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16437fe700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] conn(0x7f16380ade10 0x7f16380b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:32.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16480043d0 con 0x7f16380ade10 2024-04-25T00:20:32.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f16480d0b40 con 0x7f164c06b300 2024-04-25T00:20:32.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f1652acb700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] conn(0x7f16380ade10 0x7f16380b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:32.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.776+0000 7f16522ca700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f16380a2d30 0x7f16380a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f163c009be0 tx=0x7f163c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:32.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.777+0000 7f1652acb700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] conn(0x7f16380ade10 0x7f16380b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:32.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.778+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== osd.0 v2:172.21.15.45:6802/2748889100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16480043d0 con 0x7f16380ade10 2024-04-25T00:20:32.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.795+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 --> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f164c062d40 con 0x7f16380ade10 2024-04-25T00:20:32.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f16437fe700 1 -- 172.21.15.45:0/3155339333 <== osd.0 v2:172.21.15.45:6802/2748889100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f164c062d40 con 0x7f16380ade10 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] conn(0x7f16380ade10 msgr2=0x7f16380b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] conn(0x7f16380ade10 0x7f16380b02b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f16380a2d30 msgr2=0x7f16380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f16380a2d30 0x7f16380a51f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f163c009be0 tx=0x7f163c009260 comp rx=0 tx=0).stop 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 msgr2=0x7f164c101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 --2- 172.21.15.45:0/3155339333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f164c06b300 0x7f164c101770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1648000c00 tx=0x7f164800bab0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f1652acb700 1 -- 172.21.15.45:0/3155339333 reap_dead start 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 shutdown_connections 2024-04-25T00:20:32.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 >> 172.21.15.45:0/3155339333 conn(0x7f164c0780e0 msgr2=0x7f164c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 shutdown_connections 2024-04-25T00:20:32.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.796+0000 7f165452e700 1 -- 172.21.15.45:0/3155339333 wait complete. 2024-04-25T00:20:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 -- 172.21.15.45:0/4087231303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c8101770 msgr2=0x7f98c8101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 --2- 172.21.15.45:0/4087231303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c8101770 0x7f98c8101b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f98b80099c0 tx=0x7f98b8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:20:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 -- 172.21.15.45:0/4087231303 shutdown_connections 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 --2- 172.21.15.45:0/4087231303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c81064c0 0x7f98c810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 --2- 172.21.15.45:0/4087231303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c8101770 0x7f98c8101b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 --2- 172.21.15.45:0/4087231303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 -- 172.21.15.45:0/4087231303 >> 172.21.15.45:0/4087231303 conn(0x7f98c80780e0 msgr2=0x7f98c80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 -- 172.21.15.45:0/4087231303 shutdown_connections 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.998+0000 7f98cd94e700 1 -- 172.21.15.45:0/4087231303 wait complete. 2024-04-25T00:20:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 Processor -- start 2024-04-25T00:20:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 -- start start 2024-04-25T00:20:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 0x7f98c8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c81064c0 0x7f98c819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81114a0 con 0x7f98c81064c0 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8111320 con 0x7f98c8102120 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.999+0000 7f98cd94e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8111620 con 0x7f98c8101770 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 0x7f98c8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c81064c0 0x7f98c819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 0x7f98c8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:34482/0 (socket says 172.21.15.45:34482) 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 -- 172.21.15.45:0/1072142646 learned_addr learned my addr 172.21.15.45:0/1072142646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:33032/0 (socket says 172.21.15.45:33032) 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 -- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 msgr2=0x7f98c8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 -- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c81064c0 msgr2=0x7f98c819d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98c81064c0 0x7f98c819d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819daa0 con 0x7f98c8101770 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c67fc700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c8102120 0x7f98c8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98c6ffd700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 0x7f98c8198aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98b80059d0 tx=0x7f98b80038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8027070 con 0x7f98c8101770 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b8003e40 con 0x7f98c8101770 2024-04-25T00:20:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819dd30 con 0x7f98c8101770 2024-04-25T00:20:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8022d00 con 0x7f98c8101770 2024-04-25T00:20:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.000+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a5650 con 0x7f98c8101770 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.001+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98c8062900 con 0x7f98c8101770 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.003+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f98b802a070 con 0x7f98c8101770 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98cc94c700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98b00a2cb0 0x7f98b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f98b80d0ca0 con 0x7f98c8101770 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98cc94c700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] conn(0x7f98b00add90 0x7f98b00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98c67fc700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98b00a2cb0 0x7f98b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98b80095d0 con 0x7f98b00add90 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f98b80d1050 con 0x7f98c8101770 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.004+0000 7f98c77fe700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] conn(0x7f98b00add90 0x7f98b00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.005+0000 7f98c67fc700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98b00a2cb0 0x7f98b00a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f98c81078e0 tx=0x7f98bc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.005+0000 7f98c77fe700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] conn(0x7f98b00add90 0x7f98b00b0230 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.006+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== osd.4 v2:172.21.15.71:6808/2155408782 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98b80095d0 con 0x7f98b00add90 2024-04-25T00:20:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.023+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 --> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98c8062d00 con 0x7f98b00add90 2024-04-25T00:20:33.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cc94c700 1 -- 172.21.15.45:0/1072142646 <== osd.4 v2:172.21.15.71:6808/2155408782 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f98c8062d00 con 0x7f98b00add90 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] conn(0x7f98b00add90 msgr2=0x7f98b00b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] conn(0x7f98b00add90 0x7f98b00b0230 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98b00a2cb0 msgr2=0x7f98b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98b00a2cb0 0x7f98b00a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f98c81078e0 tx=0x7f98bc0073d0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 msgr2=0x7f98c8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 --2- 172.21.15.45:0/1072142646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98c8101770 0x7f98c8198aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98b80059d0 tx=0x7f98b80038b0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98c77fe700 1 -- 172.21.15.45:0/1072142646 reap_dead start 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 shutdown_connections 2024-04-25T00:20:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 >> 172.21.15.45:0/1072142646 conn(0x7f98c80780e0 msgr2=0x7f98c806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 shutdown_connections 2024-04-25T00:20:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:33.024+0000 7f98cd94e700 1 -- 172.21.15.45:0/1072142646 wait complete. 2024-04-25T00:20:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 -- 172.21.15.45:0/1977149772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd8101130 msgr2=0x7f1bd81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/1977149772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd8101130 0x7f1bd81015b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009990 tx=0x7f1bc8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 -- 172.21.15.45:0/1977149772 shutdown_connections 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/1977149772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd81071a0 0x7f1bd8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/1977149772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 0x7f1bd810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/1977149772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd8101130 0x7f1bd81015b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.280+0000 7f1bdfe90700 1 -- 172.21.15.45:0/1977149772 >> 172.21.15.45:0/1977149772 conn(0x7f1bd80780e0 msgr2=0x7f1bd80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.281+0000 7f1bdfe90700 1 -- 172.21.15.45:0/1977149772 shutdown_connections 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.281+0000 7f1bdfe90700 1 -- 172.21.15.45:0/1977149772 wait complete. 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.281+0000 7f1bdfe90700 1 Processor -- start 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.281+0000 7f1bdfe90700 1 -- start start 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8101130 0x7f1bd819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 0x7f1bd819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 0x7f1bd81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8111410 con 0x7f1bd81071a0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8111290 con 0x7f1bd8101af0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdfe90700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8111590 con 0x7f1bd8101130 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bde42d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 0x7f1bd81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bde42d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 0x7f1bd81977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45844/0 (socket says 172.21.15.45:45844) 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bde42d700 1 -- 172.21.15.45:0/4126222283 learned_addr learned my addr 172.21.15.45:0/4126222283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bddc2c700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8101130 0x7f1bd819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 0x7f1bd819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 -- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8101130 msgr2=0x7f1bd819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8101130 0x7f1bd819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 -- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 msgr2=0x7f1bd81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 0x7f1bd81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bdd42b700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd8197fb0 con 0x7f1bd8101af0 2024-04-25T00:20:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.282+0000 7f1bde42d700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd81071a0 0x7f1bd81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bdd42b700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 0x7f1bd819db40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80059d0 tx=0x7f1bc800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8027070 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc8003d70 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800f460 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd8198240 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.283+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd8069f10 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.284+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bd806a440 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.285+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1bc802a070 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bceffd700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bc40a2cb0 0x7f1bc40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bddc2c700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bc40a2cb0 0x7f1bc40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f1bc80d0ef0 con 0x7f1bd8101af0 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bceffd700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] conn(0x7f1bc40add90 0x7f1bc40b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bc8010ea0 con 0x7f1bc40add90 2024-04-25T00:20:33.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f1bc80d12a0 con 0x7f1bd8101af0 2024-04-25T00:20:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bde42d700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] conn(0x7f1bc40add90 0x7f1bc40b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bddc2c700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bc40a2cb0 0x7f1bc40a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4007b60 tx=0x7f1bd40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.286+0000 7f1bde42d700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] conn(0x7f1bc40add90 0x7f1bc40b0230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.287+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== osd.11 v2:172.21.15.122:6824/1901931479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bc8010ea0 con 0x7f1bc40add90 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 --> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1bd806a840 con 0x7f1bc40add90 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bceffd700 1 -- 172.21.15.45:0/4126222283 <== osd.11 v2:172.21.15.122:6824/1901931479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1bd806a840 con 0x7f1bc40add90 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] conn(0x7f1bc40add90 msgr2=0x7f1bc40b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] conn(0x7f1bc40add90 0x7f1bc40b0230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bc40a2cb0 msgr2=0x7f1bc40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bc40a2cb0 0x7f1bc40a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4007b60 tx=0x7f1bd40073d0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 msgr2=0x7f1bd819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 --2- 172.21.15.45:0/4126222283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bd8101af0 0x7f1bd819db40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80059d0 tx=0x7f1bc800bab0 comp rx=0 tx=0).stop 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bde42d700 1 -- 172.21.15.45:0/4126222283 reap_dead start 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 shutdown_connections 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.304+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 >> 172.21.15.45:0/4126222283 conn(0x7f1bd80780e0 msgr2=0x7f1bd80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.305+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 shutdown_connections 2024-04-25T00:20:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:32.305+0000 7f1bdfe90700 1 -- 172.21.15.45:0/4126222283 wait complete. 2024-04-25T00:20:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:33 smithi071 ceph-mon[32667]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:33 smithi045 ceph-mon[29194]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:33 smithi122 ceph-mon[32959]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:33.796 INFO:teuthology.orchestra.run.smithi045.stdout:343597383706 2024-04-25T00:20:33.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T00:20:34.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:34 smithi045 ceph-mon[29194]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:34.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:34.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:34.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:34.758 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:34 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:34.787 INFO:teuthology.orchestra.run.smithi045.stdout:141733920830 2024-04-25T00:20:34.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T00:20:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:34 smithi122 ceph-mon[32959]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:34 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:34 smithi071 ceph-mon[32667]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7 op/s; 83 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T00:20:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:34 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:20:35.632 INFO:teuthology.orchestra.run.smithi045.stdout:309237645343 2024-04-25T00:20:35.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.798+0000 7fb318763700 1 -- 172.21.15.45:0/2876123326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 msgr2=0x7fb310106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.798+0000 7fb318763700 1 --2- 172.21.15.45:0/2876123326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb310106ec0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb300009990 tx=0x7fb300009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 -- 172.21.15.45:0/2876123326 shutdown_connections 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 --2- 172.21.15.45:0/2876123326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3101024e0 0x7fb3101028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 --2- 172.21.15.45:0/2876123326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3101075a0 0x7fb31010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 --2- 172.21.15.45:0/2876123326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb310106ec0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 -- 172.21.15.45:0/2876123326 >> 172.21.15.45:0/2876123326 conn(0x7fb3100793e0 msgr2=0x7fb3100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 -- 172.21.15.45:0/2876123326 shutdown_connections 2024-04-25T00:20:35.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.799+0000 7fb318763700 1 -- 172.21.15.45:0/2876123326 wait complete. 2024-04-25T00:20:35.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 Processor -- start 2024-04-25T00:20:35.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 -- start start 2024-04-25T00:20:35.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3101024e0 0x7fb31019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb31019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3101075a0 0x7fb310198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb310112560 con 0x7fb310102e90 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101123e0 con 0x7fb3101075a0 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb318763700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101126e0 con 0x7fb3101024e0 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb31019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb31019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39878/0 (socket says 172.21.15.45:39878) 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 -- 172.21.15.45:0/173518309 learned_addr learned my addr 172.21.15.45:0/173518309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb316d00700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3101075a0 0x7fb310198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb3164ff700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3101024e0 0x7fb31019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 -- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3101024e0 msgr2=0x7fb31019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3101024e0 0x7fb31019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 -- 172.21.15.45:0/173518309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3101075a0 msgr2=0x7fb310198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3101075a0 0x7fb310198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.800+0000 7fb315cfe700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb310199000 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb315cfe700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb31019eb90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb300004580 tx=0x7fb3000045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300026070 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb300004ce0 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb310199290 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300005320 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.801+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb310199690 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.802+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3100629c0 con 0x7fb310102e90 2024-04-25T00:20:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.803+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb300029070 con 0x7fb310102e90 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.804+0000 7fb3077fe700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb2fc0a2d30 0x7fb2fc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.804+0000 7fb3164ff700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb2fc0a2d30 0x7fb2fc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.804+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fb3000d09e0 con 0x7fb310102e90 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.805+0000 7fb3077fe700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] conn(0x7fb2fc0ade10 0x7fb2fc0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.805+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2fc0b0980 con 0x7fb2fc0ade10 2024-04-25T00:20:35.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.805+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb3000d0d90 con 0x7fb310102e90 2024-04-25T00:20:35.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.805+0000 7fb316d00700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] conn(0x7fb2fc0ade10 0x7fb2fc0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.805+0000 7fb3164ff700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb2fc0a2d30 0x7fb2fc0a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb30c005950 tx=0x7fb30c005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.806+0000 7fb316d00700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] conn(0x7fb2fc0ade10 0x7fb2fc0b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.806+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== osd.2 v2:172.21.15.122:6800/3829837658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2fc0b0980 con 0x7fb2fc0ade10 2024-04-25T00:20:35.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.825+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 --> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb310062dc0 con 0x7fb2fc0ade10 2024-04-25T00:20:35.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.826+0000 7fb3077fe700 1 -- 172.21.15.45:0/173518309 <== osd.2 v2:172.21.15.122:6800/3829837658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb310062dc0 con 0x7fb2fc0ade10 2024-04-25T00:20:35.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.826+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] conn(0x7fb2fc0ade10 msgr2=0x7fb2fc0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.826+0000 7fb318763700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] conn(0x7fb2fc0ade10 0x7fb2fc0b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.826+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb2fc0a2d30 msgr2=0x7fb2fc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.826+0000 7fb318763700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb2fc0a2d30 0x7fb2fc0a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb30c005950 tx=0x7fb30c005cf0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 msgr2=0x7fb31019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 --2- 172.21.15.45:0/173518309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb310102e90 0x7fb31019eb90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb300004580 tx=0x7fb3000045b0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb316d00700 1 -- 172.21.15.45:0/173518309 reap_dead start 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 shutdown_connections 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 >> 172.21.15.45:0/173518309 conn(0x7fb3100793e0 msgr2=0x7fb310100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 shutdown_connections 2024-04-25T00:20:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:35.827+0000 7fb318763700 1 -- 172.21.15.45:0/173518309 wait complete. 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/4065835566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102ec0 msgr2=0x7f7eb0106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/4065835566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102ec0 0x7f7eb0106eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009990 tx=0x7f7ea0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/4065835566 shutdown_connections 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/4065835566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 0x7f7eb010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/4065835566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb0102510 0x7f7eb01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/4065835566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102ec0 0x7f7eb0106eb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/4065835566 >> 172.21.15.45:0/4065835566 conn(0x7f7eb00793a0 msgr2=0x7f7eb00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/4065835566 shutdown_connections 2024-04-25T00:20:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/4065835566 wait complete. 2024-04-25T00:20:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.396+0000 7f7eb5ad2700 1 Processor -- start 2024-04-25T00:20:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 -- start start 2024-04-25T00:20:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 0x7f7eb019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb0102ec0 0x7f7eb019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 0x7f7eb0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0112530 con 0x7f7eb0102510 2024-04-25T00:20:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01123b0 con 0x7f7eb0102ec0 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eb5ad2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01126b0 con 0x7f7eb0107590 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 0x7f7eb019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 0x7f7eb019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39892/0 (socket says 172.21.15.45:39892) 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 -- 172.21.15.45:0/3280918521 learned_addr learned my addr 172.21.15.45:0/3280918521 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaeffd700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb0102ec0 0x7f7eb019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaffff700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 0x7f7eb0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 -- 172.21.15.45:0/3280918521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 msgr2=0x7f7eb0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 0x7f7eb0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 -- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb0102ec0 msgr2=0x7f7eb019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb0102ec0 0x7f7eb019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaf7fe700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0198ff0 con 0x7f7eb0102510 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.397+0000 7f7eaffff700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0107590 0x7f7eb0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eaf7fe700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 0x7f7eb019e690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d950 tx=0x7f7e9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98016070 con 0x7f7eb0102510 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e98009e20 con 0x7f7eb0102510 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01992e0 con 0x7f7eb0102510 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800f620 con 0x7f7eb0102510 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.398+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01a54f0 con 0x7f7eb0102510 2024-04-25T00:20:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.399+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7eb00629c0 con 0x7f7eb0102510 2024-04-25T00:20:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.400+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e98019070 con 0x7f7eb0102510 2024-04-25T00:20:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eacff9700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e9c0a2cb0 0x7f7e9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eaeffd700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e9c0a2cb0 0x7f7e9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7e980cf640 con 0x7f7eb0102510 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eacff9700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] conn(0x7f7e9c0add90 0x7f7e9c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e98010e50 con 0x7f7e9c0add90 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7e980cf9f0 con 0x7f7eb0102510 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eaeffd700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e9c0a2cb0 0x7f7e9c0a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000b5c0 tx=0x7f7ea0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.401+0000 7f7eaffff700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] conn(0x7f7e9c0add90 0x7f7e9c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.402+0000 7f7eaffff700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] conn(0x7f7e9c0add90 0x7f7e9c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.403+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== osd.7 v2:172.21.15.71:6816/2471128651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e98010e50 con 0x7f7e9c0add90 2024-04-25T00:20:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.419+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 --> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7eb0062dc0 con 0x7f7e9c0add90 2024-04-25T00:20:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.419+0000 7f7eacff9700 1 -- 172.21.15.45:0/3280918521 <== osd.7 v2:172.21.15.71:6816/2471128651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7eb0062dc0 con 0x7f7e9c0add90 2024-04-25T00:20:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] conn(0x7f7e9c0add90 msgr2=0x7f7e9c0b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] conn(0x7f7e9c0add90 0x7f7e9c0b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:36.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e9c0a2cb0 msgr2=0x7f7e9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7e9c0a2cb0 0x7f7e9c0a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000b5c0 tx=0x7f7ea0005960 comp rx=0 tx=0).stop 2024-04-25T00:20:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 msgr2=0x7f7eb019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 --2- 172.21.15.45:0/3280918521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7eb0102510 0x7f7eb019e690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d950 tx=0x7f7e9800dc60 comp rx=0 tx=0).stop 2024-04-25T00:20:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eaffff700 1 -- 172.21.15.45:0/3280918521 reap_dead start 2024-04-25T00:20:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 shutdown_connections 2024-04-25T00:20:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 >> 172.21.15.45:0/3280918521 conn(0x7f7eb00793a0 msgr2=0x7f7eb0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 shutdown_connections 2024-04-25T00:20:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:36.420+0000 7f7eb5ad2700 1 -- 172.21.15.45:0/3280918521 wait complete. 2024-04-25T00:20:36.830 INFO:teuthology.orchestra.run.smithi045.stdout:77309411400 2024-04-25T00:20:36.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T00:20:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:36 smithi045 ceph-mon[29194]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.027+0000 7f3814b04700 1 -- 172.21.15.45:0/715210223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38101071a0 msgr2=0x7f3810107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.027+0000 7f3814b04700 1 --2- 172.21.15.45:0/715210223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38101071a0 0x7f3810107580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3800009990 tx=0x7f3800009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 -- 172.21.15.45:0/715210223 shutdown_connections 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 --2- 172.21.15.45:0/715210223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3810101af0 0x7f381010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 --2- 172.21.15.45:0/715210223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38101071a0 0x7f3810107580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 --2- 172.21.15.45:0/715210223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3810101130 0x7f38101015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 -- 172.21.15.45:0/715210223 >> 172.21.15.45:0/715210223 conn(0x7f38100780e0 msgr2=0x7f38100784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 -- 172.21.15.45:0/715210223 shutdown_connections 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 -- 172.21.15.45:0/715210223 wait complete. 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 Processor -- start 2024-04-25T00:20:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.028+0000 7f3814b04700 1 -- start start 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3810101130 0x7f38101997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 0x7f3810199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38101071a0 0x7f38101a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3810111490 con 0x7f3810101af0 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3810111310 con 0x7f38101071a0 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3810111610 con 0x7f3810101130 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 0x7f3810199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 0x7f3810199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39906/0 (socket says 172.21.15.45:39906) 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 -- 172.21.15.45:0/3024992836 learned_addr learned my addr 172.21.15.45:0/3024992836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380e59c700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3810101130 0x7f38101997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380ed9d700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38101071a0 0x7f38101a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 -- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3810101130 msgr2=0x7f38101997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3810101130 0x7f38101997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 -- 172.21.15.45:0/3024992836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38101071a0 msgr2=0x7f38101a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38101071a0 0x7f38101a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101a4d70 con 0x7f3810101af0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f380dd9b700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 0x7f3810199cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3804007c70 tx=0x7f3804007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400e2e0 con 0x7f3810101af0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3810197600 con 0x7f3810101af0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.029+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380400e480 con 0x7f3810101af0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.030+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380400cce0 con 0x7f3810101af0 2024-04-25T00:20:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.030+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3810197b90 con 0x7f3810101af0 2024-04-25T00:20:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.031+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3810100520 con 0x7f3810101af0 2024-04-25T00:20:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.032+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3804010660 con 0x7f3810101af0 2024-04-25T00:20:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.032+0000 7f37ff7fe700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f37f80a2c10 0x7f37f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.032+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f38040d27a0 con 0x7f3810101af0 2024-04-25T00:20:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.032+0000 7f380e59c700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f37f80a2c10 0x7f37f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.032+0000 7f37ff7fe700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] conn(0x7f37f80adcf0 0x7f37f80b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.033+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3804022e50 con 0x7f37f80adcf0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.033+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f38040d9050 con 0x7f3810101af0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.033+0000 7f380ed9d700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] conn(0x7f37f80adcf0 0x7f37f80b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.033+0000 7f380e59c700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f37f80a2c10 0x7f37f80a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f380000b5c0 tx=0x7f3800005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.033+0000 7f380ed9d700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] conn(0x7f37f80adcf0 0x7f37f80b0190 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.034+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== osd.8 v2:172.21.15.122:6816/2592930089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3804022e50 con 0x7f37f80adcf0 2024-04-25T00:20:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.051+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 --> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3810102850 con 0x7f37f80adcf0 2024-04-25T00:20:37.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.051+0000 7f37ff7fe700 1 -- 172.21.15.45:0/3024992836 <== osd.8 v2:172.21.15.122:6816/2592930089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3810102850 con 0x7f37f80adcf0 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.051+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] conn(0x7f37f80adcf0 msgr2=0x7f37f80b0190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.051+0000 7f3814b04700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] conn(0x7f37f80adcf0 0x7f37f80b0190 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f37f80a2c10 msgr2=0x7f37f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f37f80a2c10 0x7f37f80a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f380000b5c0 tx=0x7f3800005960 comp rx=0 tx=0).stop 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 msgr2=0x7f3810199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 --2- 172.21.15.45:0/3024992836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3810101af0 0x7f3810199cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3804007c70 tx=0x7f3804007f80 comp rx=0 tx=0).stop 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f380ed9d700 1 -- 172.21.15.45:0/3024992836 reap_dead start 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 shutdown_connections 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 >> 172.21.15.45:0/3024992836 conn(0x7f38100780e0 msgr2=0x7f38100ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 shutdown_connections 2024-04-25T00:20:37.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.052+0000 7f3814b04700 1 -- 172.21.15.45:0/3024992836 wait complete. 2024-04-25T00:20:37.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:36 smithi122 ceph-mon[32959]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T00:20:37.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:36 smithi071 ceph-mon[32667]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 72 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T00:20:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 -- 172.21.15.45:0/626840573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352006a8e0 msgr2=0x7f352010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 --2- 172.21.15.45:0/626840573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352006a8e0 0x7f352010f0f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f351800b0a0 tx=0x7f351800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 -- 172.21.15.45:0/626840573 shutdown_connections 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 --2- 172.21.15.45:0/626840573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35201082b0 0x7f3520108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 --2- 172.21.15.45:0/626840573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352006a8e0 0x7f352010f0f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 --2- 172.21.15.45:0/626840573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3520069f40 0x7f352006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 -- 172.21.15.45:0/626840573 >> 172.21.15.45:0/626840573 conn(0x7f35200780e0 msgr2=0x7f35200784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 -- 172.21.15.45:0/626840573 shutdown_connections 2024-04-25T00:20:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.639+0000 7f35277be700 1 -- 172.21.15.45:0/626840573 wait complete. 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.640+0000 7f35277be700 1 Processor -- start 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.640+0000 7f35277be700 1 -- start start 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 0x7f352019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352006a8e0 0x7f352019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35201082b0 0x7f3520198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112500 con 0x7f3520069f40 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112380 con 0x7f35201082b0 2024-04-25T00:20:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35277be700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3520112680 con 0x7f352006a8e0 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 0x7f352019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 0x7f352019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39932/0 (socket says 172.21.15.45:39932) 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 -- 172.21.15.45:0/4115287060 learned_addr learned my addr 172.21.15.45:0/4115287060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f3524d59700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352006a8e0 0x7f352019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f3525d5b700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35201082b0 0x7f3520198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 -- 172.21.15.45:0/4115287060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352006a8e0 msgr2=0x7f352019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f352006a8e0 0x7f352019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 -- 172.21.15.45:0/4115287060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35201082b0 msgr2=0x7f3520198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35201082b0 0x7f3520198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3520199020 con 0x7f3520069f40 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f352555a700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 0x7f352019e620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f351c00d950 tx=0x7f351c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.641+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c016070 con 0x7f3520069f40 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.642+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351c009e20 con 0x7f3520069f40 2024-04-25T00:20:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.642+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3520199310 con 0x7f3520069f40 2024-04-25T00:20:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.642+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00f620 con 0x7f3520069f40 2024-04-25T00:20:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.642+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201a55a0 con 0x7f3520069f40 2024-04-25T00:20:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.642+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3520062940 con 0x7f3520069f40 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.644+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f351c019070 con 0x7f3520069f40 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.644+0000 7f35167fc700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f350c0a2d30 0x7f350c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f3524d59700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f350c0a2d30 0x7f350c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f351c0cf550 con 0x7f3520069f40 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f35167fc700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] conn(0x7f350c0ade10 0x7f350c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f351c010e50 con 0x7f350c0ade10 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f351c0cf900 con 0x7f3520069f40 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f3525d5b700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] conn(0x7f350c0ade10 0x7f350c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f3524d59700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f350c0a2d30 0x7f350c0a51f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3510005ea0 tx=0x7f3510005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.645+0000 7f3525d5b700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] conn(0x7f350c0ade10 0x7f350c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.646+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== osd.9 v2:172.21.15.45:6826/3783590854 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f351c010e50 con 0x7f350c0ade10 2024-04-25T00:20:37.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.663+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 --> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3520062d80 con 0x7f350c0ade10 2024-04-25T00:20:37.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.663+0000 7f35167fc700 1 -- 172.21.15.45:0/4115287060 <== osd.9 v2:172.21.15.45:6826/3783590854 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3520062d80 con 0x7f350c0ade10 2024-04-25T00:20:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] conn(0x7f350c0ade10 msgr2=0x7f350c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] conn(0x7f350c0ade10 0x7f350c0b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f350c0a2d30 msgr2=0x7f350c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f350c0a2d30 0x7f350c0a51f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3510005ea0 tx=0x7f3510005e30 comp rx=0 tx=0).stop 2024-04-25T00:20:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 msgr2=0x7f352019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 --2- 172.21.15.45:0/4115287060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3520069f40 0x7f352019e620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f351c00d950 tx=0x7f351c00dc60 comp rx=0 tx=0).stop 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f3525d5b700 1 -- 172.21.15.45:0/4115287060 reap_dead start 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 shutdown_connections 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 >> 172.21.15.45:0/4115287060 conn(0x7f35200780e0 msgr2=0x7f35200ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 shutdown_connections 2024-04-25T00:20:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:37.664+0000 7f35277be700 1 -- 172.21.15.45:0/4115287060 wait complete. 2024-04-25T00:20:38.430 INFO:teuthology.orchestra.run.smithi045.stdout:219043332143 2024-04-25T00:20:38.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T00:20:38.921 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:38 smithi071 ceph-mon[32667]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 65 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:38.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:38 smithi045 ceph-mon[29194]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 65 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:39.003 INFO:teuthology.orchestra.run.smithi045.stdout:253403070506 2024-04-25T00:20:39.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T00:20:39.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:38 smithi122 ceph-mon[32959]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 65 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:39.676 INFO:teuthology.orchestra.run.smithi045.stdout:34359738449 2024-04-25T00:20:39.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T00:20:40.564 INFO:teuthology.orchestra.run.smithi045.stdout:287762808869 2024-04-25T00:20:40.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T00:20:40.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.741+0000 7f70480f3700 1 -- 172.21.15.45:0/2497444678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704006a860 msgr2=0x7f704010e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.741+0000 7f70480f3700 1 --2- 172.21.15.45:0/2497444678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704006a860 0x7f704010e7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f703c009990 tx=0x7f703c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 -- 172.21.15.45:0/2497444678 shutdown_connections 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 --2- 172.21.15.45:0/2497444678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 0x7f704006a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 --2- 172.21.15.45:0/2497444678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704010ece0 0x7f70401110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 --2- 172.21.15.45:0/2497444678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704006a860 0x7f704010e7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 -- 172.21.15.45:0/2497444678 >> 172.21.15.45:0/2497444678 conn(0x7f70400780e0 msgr2=0x7f70400784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:40.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 -- 172.21.15.45:0/2497444678 shutdown_connections 2024-04-25T00:20:40.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.742+0000 7f70480f3700 1 -- 172.21.15.45:0/2497444678 wait complete. 2024-04-25T00:20:40.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 Processor -- start 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 -- start start 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 0x7f70401a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704006a860 0x7f70401a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704010ece0 0x7f704019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70401147b0 con 0x7f704010ece0 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7040114630 con 0x7f704006a860 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.743+0000 7f70480f3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7040114930 con 0x7f7040069f40 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7046690700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704010ece0 0x7f704019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 0x7f70401a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7046690700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704010ece0 0x7f704019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39970/0 (socket says 172.21.15.45:39970) 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7046690700 1 -- 172.21.15.45:0/2829621189 learned_addr learned my addr 172.21.15.45:0/2829621189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f704568e700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704006a860 0x7f70401a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 -- 172.21.15.45:0/2829621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704006a860 msgr2=0x7f70401a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704006a860 0x7f70401a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 -- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704010ece0 msgr2=0x7f704019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704010ece0 0x7f704019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704019b450 con 0x7f7040069f40 2024-04-25T00:20:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f704568e700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f704006a860 0x7f70401a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:40.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.744+0000 7f7045e8f700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 0x7f70401a0a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f703000e9f0 tx=0x7f703000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.745+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703000cbd0 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.745+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7030004500 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.745+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030005280 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.745+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704019b740 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.745+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a79d0 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.746+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7040062900 con 0x7f7040069f40 2024-04-25T00:20:40.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.747+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7030005420 con 0x7f7040069f40 2024-04-25T00:20:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7036ffd700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f702c0a2d30 0x7f702c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f70300cf790 con 0x7f7040069f40 2024-04-25T00:20:40.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f704568e700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f702c0a2d30 0x7f702c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7036ffd700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] conn(0x7f702c0ade10 0x7f702c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f703000be50 con 0x7f702c0ade10 2024-04-25T00:20:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f70300cfb40 con 0x7f7040069f40 2024-04-25T00:20:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.748+0000 7f7046690700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] conn(0x7f702c0ade10 0x7f702c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:40.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.749+0000 7f704568e700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f702c0a2d30 0x7f702c0a51f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f704019c270 tx=0x7f703c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.749+0000 7f7046690700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] conn(0x7f702c0ade10 0x7f702c0b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:40.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.750+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== osd.3 v2:172.21.15.45:6810/2950579455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f703000be50 con 0x7f702c0ade10 2024-04-25T00:20:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.774+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 --> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7040062d80 con 0x7f702c0ade10 2024-04-25T00:20:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.775+0000 7f7036ffd700 1 -- 172.21.15.45:0/2829621189 <== osd.3 v2:172.21.15.45:6810/2950579455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7040062d80 con 0x7f702c0ade10 2024-04-25T00:20:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.775+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] conn(0x7f702c0ade10 msgr2=0x7f702c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.775+0000 7f70480f3700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] conn(0x7f702c0ade10 0x7f702c0b02b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f702c0a2d30 msgr2=0x7f702c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f702c0a2d30 0x7f702c0a51f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f704019c270 tx=0x7f703c01a040 comp rx=0 tx=0).stop 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 msgr2=0x7f70401a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 --2- 172.21.15.45:0/2829621189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7040069f40 0x7f70401a0a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f703000e9f0 tx=0x7f703000ed00 comp rx=0 tx=0).stop 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f7046690700 1 -- 172.21.15.45:0/2829621189 reap_dead start 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 shutdown_connections 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 >> 172.21.15.45:0/2829621189 conn(0x7f70400780e0 msgr2=0x7f70401101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 shutdown_connections 2024-04-25T00:20:40.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.776+0000 7f70480f3700 1 -- 172.21.15.45:0/2829621189 wait complete. 2024-04-25T00:20:40.791 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:40 smithi045 ceph-mon[29194]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:40 smithi122 ceph-mon[32959]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:41.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:40 smithi071 ceph-mon[32667]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T00:20:41.459 INFO:teuthology.orchestra.run.smithi045.stdout:107374182468 2024-04-25T00:20:41.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T00:20:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 -- 172.21.15.45:0/2437989961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101120 msgr2=0x7f3a501015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 --2- 172.21.15.45:0/2437989961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101120 0x7f3a501015a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009990 tx=0x7f3a4c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 -- 172.21.15.45:0/2437989961 shutdown_connections 2024-04-25T00:20:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 --2- 172.21.15.45:0/2437989961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a50107190 0x7f3a50107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 --2- 172.21.15.45:0/2437989961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50101ae0 0x7f3a5010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 --2- 172.21.15.45:0/2437989961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101120 0x7f3a501015a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 -- 172.21.15.45:0/2437989961 >> 172.21.15.45:0/2437989961 conn(0x7f3a500780a0 msgr2=0x7f3a500784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 -- 172.21.15.45:0/2437989961 shutdown_connections 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.178+0000 7f3a57a16700 1 -- 172.21.15.45:0/2437989961 wait complete. 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.179+0000 7f3a57a16700 1 Processor -- start 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.179+0000 7f3a57a16700 1 -- start start 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a50101120 0x7f3a5019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 0x7f3a5019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50107190 0x7f3a501976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50111420 con 0x7f3a50101ae0 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501112a0 con 0x7f3a50107190 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501115a0 con 0x7f3a50101120 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 0x7f3a5019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 0x7f3a5019db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39950/0 (socket says 172.21.15.45:39950) 2024-04-25T00:20:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 -- 172.21.15.45:0/3699782603 learned_addr learned my addr 172.21.15.45:0/3699782603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a557b2700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a50101120 0x7f3a5019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a55fb3700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50107190 0x7f3a501976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 -- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a50101120 msgr2=0x7f3a5019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a50101120 0x7f3a5019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 -- 172.21.15.45:0/3699782603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50107190 msgr2=0x7f3a501976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50107190 0x7f3a501976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a50197f80 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a55fb3700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a50107190 0x7f3a501976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a54fb1700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 0x7f3a5019db10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000c00 tx=0x7f3a4c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c027070 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a50198210 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c003d70 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.180+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00f6c0 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.181+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5006a7a0 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.181+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a5006ad50 con 0x7f3a50101ae0 2024-04-25T00:20:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3a4c02a070 con 0x7f3a50101ae0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3a3c0a2cb0 0x7f3a3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f3a4c0d0cb0 con 0x7f3a50101ae0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a557b2700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3a3c0a2cb0 0x7f3a3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] conn(0x7f3a3c0add90 0x7f3a3c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a4c0043d0 con 0x7f3a3c0add90 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.183+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3a4c0d1100 con 0x7f3a50101ae0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.184+0000 7f3a557b2700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3a3c0a2cb0 0x7f3a3c0a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a440045b0 tx=0x7f3a4400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.184+0000 7f3a55fb3700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] conn(0x7f3a3c0add90 0x7f3a3c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.184+0000 7f3a55fb3700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] conn(0x7f3a3c0add90 0x7f3a3c0b0230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.185+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== osd.5 v2:172.21.15.122:6808/2119403630 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a4c0043d0 con 0x7f3a3c0add90 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.201+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 --> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a5006b1d0 con 0x7f3a3c0add90 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.202+0000 7f3a427fc700 1 -- 172.21.15.45:0/3699782603 <== osd.5 v2:172.21.15.122:6808/2119403630 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3a5006b1d0 con 0x7f3a3c0add90 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.202+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] conn(0x7f3a3c0add90 msgr2=0x7f3a3c0b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.202+0000 7f3a57a16700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] conn(0x7f3a3c0add90 0x7f3a3c0b0230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3a3c0a2cb0 msgr2=0x7f3a3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3a3c0a2cb0 0x7f3a3c0a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a440045b0 tx=0x7f3a4400b3c0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 msgr2=0x7f3a5019db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 --2- 172.21.15.45:0/3699782603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a50101ae0 0x7f3a5019db10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c000c00 tx=0x7f3a4c00bab0 comp rx=0 tx=0).stop 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a55fb3700 1 -- 172.21.15.45:0/3699782603 reap_dead start 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 shutdown_connections 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 >> 172.21.15.45:0/3699782603 conn(0x7f3a500780a0 msgr2=0x7f3a500ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 shutdown_connections 2024-04-25T00:20:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:40.203+0000 7f3a57a16700 1 -- 172.21.15.45:0/3699782603 wait complete. 2024-04-25T00:20:42.234 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:42.236 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:42.238 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:42.238 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:42.248 INFO:teuthology.orchestra.run.smithi045.stdout:163208757306 2024-04-25T00:20:42.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T00:20:42.264 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:42 smithi045 ceph-mon[29194]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T00:20:43.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:42 smithi122 ceph-mon[32959]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T00:20:43.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:42 smithi071 ceph-mon[32667]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T00:20:44.924 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:44 smithi045 ceph-mon[29194]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:45.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:44 smithi122 ceph-mon[32959]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:45.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:44 smithi071 ceph-mon[32667]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:47.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:46 smithi045 ceph-mon[29194]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:47.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:46 smithi122 ceph-mon[32959]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:46 smithi071 ceph-mon[32667]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:48.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 -- 172.21.15.45:0/2901962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92001082a0 msgr2=0x7f9200108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 --2- 172.21.15.45:0/2901962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92001082a0 0x7f9200108680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009990 tx=0x7f91fc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 -- 172.21.15.45:0/2901962648 shutdown_connections 2024-04-25T00:20:48.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 --2- 172.21.15.45:0/2901962648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006b170 0x7f920010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 --2- 172.21.15.45:0/2901962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92001082a0 0x7f9200108680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 --2- 172.21.15.45:0/2901962648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f920006a7d0 0x7f920006ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 -- 172.21.15.45:0/2901962648 >> 172.21.15.45:0/2901962648 conn(0x7f92000780a0 msgr2=0x7f92000784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 -- 172.21.15.45:0/2901962648 shutdown_connections 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.089+0000 7f92083e9700 1 -- 172.21.15.45:0/2901962648 wait complete. 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 Processor -- start 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 -- start start 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006a7d0 0x7f920019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 0x7f9200198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9200112500 con 0x7f920006b170 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9200112380 con 0x7f92001082a0 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f92083e9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9200112680 con 0x7f920006a7d0 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.090+0000 7f9205984700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57724/0 (socket says 172.21.15.45:57724) 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 -- 172.21.15.45:0/2948872650 learned_addr learned my addr 172.21.15.45:0/2948872650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9206986700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 0x7f9200198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9206185700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006a7d0 0x7f920019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 -- 172.21.15.45:0/2948872650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006a7d0 msgr2=0x7f920019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006a7d0 0x7f920019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 -- 172.21.15.45:0/2948872650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 msgr2=0x7f9200198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 0x7f9200198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 -- 172.21.15.45:0/2948872650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200198fe0 con 0x7f920006b170 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9206986700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 0x7f9200198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f9205984700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9a0 tx=0x7f91f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f400ca10 con 0x7f920006b170 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f400cbb0 con 0x7f920006b170 2024-04-25T00:20:48.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92001992d0 con 0x7f920006b170 2024-04-25T00:20:48.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.091+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4018680 con 0x7f920006b170 2024-04-25T00:20:48.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.092+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92001a54e0 con 0x7f920006b170 2024-04-25T00:20:48.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.093+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9200103950 con 0x7f920006b170 2024-04-25T00:20:48.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.094+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f91f4010430 con 0x7f920006b170 2024-04-25T00:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.095+0000 7f91f37fe700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 0x7f91ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.095+0000 7f9206185700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 0x7f91ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.095+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f91f40cfc00 con 0x7f920006b170 2024-04-25T00:20:48.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.095+0000 7f9206185700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 0x7f91ec0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b2e0 tx=0x7f91fc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.098+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f91f409c6c0 con 0x7f920006b170 2024-04-25T00:20:48.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.305+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f920002db30 con 0x7f920006b170 2024-04-25T00:20:48.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.305+0000 7f91f37fe700 1 -- 172.21.15.45:0/2948872650 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f91f409bfc0 con 0x7f920006b170 2024-04-25T00:20:48.306 INFO:teuthology.orchestra.run.smithi045.stdout:309237645346 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 msgr2=0x7f91ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 0x7f91ec0a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00b2e0 tx=0x7f91fc005960 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 msgr2=0x7f920019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91f400e9a0 tx=0x7f91f400ecb0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 shutdown_connections 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f920006a7d0 0x7f920019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f91ec0a2cb0 0x7f91ec0a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92001082a0 0x7f9200198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 --2- 172.21.15.45:0/2948872650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920006b170 0x7f920019ebc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 >> 172.21.15.45:0/2948872650 conn(0x7f92000780a0 msgr2=0x7f92000ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 shutdown_connections 2024-04-25T00:20:48.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:48.307+0000 7f92083e9700 1 -- 172.21.15.45:0/2948872650 wait complete. 2024-04-25T00:20:48.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:48 smithi045 ceph-mon[29194]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:48.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:48 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2948872650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:20:49.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:48 smithi122 ceph-mon[32959]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:49.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:48 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2948872650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:20:49.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:48 smithi071 ceph-mon[32667]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:49.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:48 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2948872650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:20:50.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645346 for osd.10 2024-04-25T00:20:50.181 DEBUG:teuthology.parallel:result is None 2024-04-25T00:20:50.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:50 smithi045 ceph-mon[29194]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:20:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:50 smithi122 ceph-mon[32959]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:20:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:50 smithi071 ceph-mon[32667]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:20:51.626 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:51.626 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:51.627 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:52.422 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:20:52.828 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:52 smithi045 ceph-mon[29194]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:20:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:52 smithi122 ceph-mon[32959]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:20:53.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:52 smithi071 ceph-mon[32667]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:20:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.788+0000 7fa540b66700 1 -- 172.21.15.45:0/603202638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c101af0 msgr2=0x7fa53c10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.788+0000 7fa540b66700 1 --2- 172.21.15.45:0/603202638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c101af0 0x7fa53c10dfe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa53400b0a0 tx=0x7fa53400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 -- 172.21.15.45:0/603202638 shutdown_connections 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 --2- 172.21.15.45:0/603202638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c101af0 0x7fa53c10dfe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 --2- 172.21.15.45:0/603202638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c1071a0 0x7fa53c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 --2- 172.21.15.45:0/603202638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 0x7fa53c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 -- 172.21.15.45:0/603202638 >> 172.21.15.45:0/603202638 conn(0x7fa53c0780e0 msgr2=0x7fa53c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 -- 172.21.15.45:0/603202638 shutdown_connections 2024-04-25T00:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 -- 172.21.15.45:0/603202638 wait complete. 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 Processor -- start 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.789+0000 7fa540b66700 1 -- start start 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 0x7fa53c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 0x7fa53c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 0x7fa53c1a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111410 con 0x7fa53c1071a0 2024-04-25T00:20:53.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111290 con 0x7fa53c101af0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa540b66700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111590 con 0x7fa53c101130 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa53ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 0x7fa53c1a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa53ad9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 0x7fa53c1a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56700/0 (socket says 172.21.15.45:56700) 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 0x7fa53c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa53ad9d700 1 -- 172.21.15.45:0/2970306147 learned_addr learned my addr 172.21.15.45:0/2970306147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa53a59c700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 0x7fa53c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 -- 172.21.15.45:0/2970306147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 msgr2=0x7fa53c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 0x7fa53c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 -- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 msgr2=0x7fa53c1a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 0x7fa53c1a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.790+0000 7fa539d9b700 1 -- 172.21.15.45:0/2970306147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c1a4dd0 con 0x7fa53c101af0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa539d9b700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 0x7fa53c199d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa53000d750 tx=0x7fa53000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530014030 con 0x7fa53c101af0 2024-04-25T00:20:53.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c1a4fa0 con 0x7fa53c101af0 2024-04-25T00:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530004500 con 0x7fa53c101af0 2024-04-25T00:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530005400 con 0x7fa53c101af0 2024-04-25T00:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.791+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c1a54a0 con 0x7fa53c101af0 2024-04-25T00:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.792+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53c068f20 con 0x7fa53c101af0 2024-04-25T00:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.794+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa530004750 con 0x7fa53c101af0 2024-04-25T00:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.794+0000 7fa52b7fe700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 0x7fa5240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.795+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa5300ceda0 con 0x7fa53c101af0 2024-04-25T00:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.795+0000 7fa53a59c700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 0x7fa5240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.795+0000 7fa53a59c700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 0x7fa5240a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005950 tx=0x7fa52c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.799+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa53009b860 con 0x7fa53c101af0 2024-04-25T00:20:53.958 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:53 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1051696702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:20:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.017+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa53c02dcf0 con 0x7fa53c101af0 2024-04-25T00:20:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.017+0000 7fa52b7fe700 1 -- 172.21.15.45:0/2970306147 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa53009b160 con 0x7fa53c101af0 2024-04-25T00:20:54.018 INFO:teuthology.orchestra.run.smithi045.stdout:55834574929 2024-04-25T00:20:54.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.019+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 msgr2=0x7fa5240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.019+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 0x7fa5240a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005950 tx=0x7fa52c00b410 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.019+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 msgr2=0x7fa53c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.019+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 0x7fa53c199d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa53000d750 tx=0x7fa53000da60 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 shutdown_connections 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa53c101130 0x7fa53c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5240a2cb0 0x7fa5240a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa53c1071a0 0x7fa53c1a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 --2- 172.21.15.45:0/2970306147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53c101af0 0x7fa53c199d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 >> 172.21.15.45:0/2970306147 conn(0x7fa53c0780e0 msgr2=0x7fa53c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 shutdown_connections 2024-04-25T00:20:54.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:54.020+0000 7fa540b66700 1 -- 172.21.15.45:0/2970306147 wait complete. 2024-04-25T00:20:54.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:53 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1051696702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:20:54.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:53 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1051696702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:20:54.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:54 smithi045 ceph-mon[29194]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:54.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:54 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1142535150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:20:54.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:54 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2970306147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:20:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:54 smithi122 ceph-mon[32959]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:54 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1142535150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:20:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:54 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2970306147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:20:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:54 smithi071 ceph-mon[32667]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:20:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:54 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1142535150' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:20:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:54 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2970306147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 -- 172.21.15.45:0/1565594048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 msgr2=0x7fd758105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1565594048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd758105d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0099c0 tx=0x7fd74c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 -- 172.21.15.45:0/1565594048 shutdown_connections 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1565594048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd758106450 0x7fd75810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1565594048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7581013c0 0x7fd7581017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1565594048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd758105d70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 -- 172.21.15.45:0/1565594048 >> 172.21.15.45:0/1565594048 conn(0x7fd7580780e0 msgr2=0x7fd7580784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 -- 172.21.15.45:0/1565594048 shutdown_connections 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.679+0000 7fd75fb53700 1 -- 172.21.15.45:0/1565594048 wait complete. 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 Processor -- start 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 -- start start 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7581013c0 0x7fd75819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd758106450 0x7fd758198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd758111410 con 0x7fd758101d70 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd758111290 con 0x7fd758106450 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75fb53700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd758111590 con 0x7fd7581013c0 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75d0ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75d0ee700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57742/0 (socket says 172.21.15.45:57742) 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75d0ee700 1 -- 172.21.15.45:0/1142535150 learned_addr learned my addr 172.21.15.45:0/1142535150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.680+0000 7fd75e0f0700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd758106450 0x7fd758198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d8ef700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7581013c0 0x7fd75819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stdout:77309411404 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 -- 172.21.15.45:0/1142535150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7581013c0 msgr2=0x7fd75819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7581013c0 0x7fd75819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 -- 172.21.15.45:0/1142535150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd758106450 msgr2=0x7fd758198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd758106450 0x7fd758198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 -- 172.21.15.45:0/1142535150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd758198fc0 con 0x7fd758101d70 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75d0ee700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c010 tx=0x7fd74c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c027070 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c003730 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd758199250 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c022d40 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.681+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7581a5570 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.682+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd758068f20 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.683+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd74c02a070 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.684+0000 7fd74affd700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 0x7fd7440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.684+0000 7fd75d8ef700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 0x7fd7440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.684+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd74c0d0990 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.684+0000 7fd75d8ef700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 0x7fd7440a51f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd754005950 tx=0x7fd7540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.687+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd74c031070 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.902+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd75802db30 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.903+0000 7fd74affd700 1 -- 172.21.15.45:0/1142535150 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd74c013070 con 0x7fd758101d70 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 msgr2=0x7fd7440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 0x7fd7440a51f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd754005950 tx=0x7fd7540058e0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 msgr2=0x7fd75819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00c010 tx=0x7fd74c0047b0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 shutdown_connections 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7581013c0 0x7fd75819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd7440a2d30 0x7fd7440a51f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd758106450 0x7fd758198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 --2- 172.21.15.45:0/1142535150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd758101d70 0x7fd75819ebe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 >> 172.21.15.45:0/1142535150 conn(0x7fd7580780e0 msgr2=0x7fd75806acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 shutdown_connections 2024-04-25T00:20:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.905+0000 7fd75fb53700 1 -- 172.21.15.45:0/1142535150 wait complete. 2024-04-25T00:20:56.129 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-04-25T00:20:56.129 DEBUG:teuthology.parallel:result is None 2024-04-25T00:20:56.890 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:56 smithi045 ceph-mon[29194]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:57.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:56 smithi122 ceph-mon[32959]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:57.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:56 smithi071 ceph-mon[32667]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:20:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 -- 172.21.15.45:0/1485211816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc168102e90 msgr2=0x7fc168106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 --2- 172.21.15.45:0/1485211816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc168102e90 0x7fc168106e90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc158009990 tx=0x7fc158009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 -- 172.21.15.45:0/1485211816 shutdown_connections 2024-04-25T00:20:57.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 --2- 172.21.15.45:0/1485211816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1681024e0 0x7fc1681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 --2- 172.21.15.45:0/1485211816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168107570 0x7fc16810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 --2- 172.21.15.45:0/1485211816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc168102e90 0x7fc168106e90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 -- 172.21.15.45:0/1485211816 >> 172.21.15.45:0/1485211816 conn(0x7fc168079390 msgr2=0x7fc1680797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 -- 172.21.15.45:0/1485211816 shutdown_connections 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.936+0000 7fc16f860700 1 -- 172.21.15.45:0/1485211816 wait complete. 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 Processor -- start 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 -- start start 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168102e90 0x7fc1681a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc168107570 0x7fc16819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc168112530 con 0x7fc1681024e0 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681123b0 con 0x7fc168102e90 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16f860700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681126b0 con 0x7fc168107570 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16d5fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16d5fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57730/0 (socket says 172.21.15.45:57730) 2024-04-25T00:20:57.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16d5fc700 1 -- 172.21.15.45:0/1051696702 learned_addr learned my addr 172.21.15.45:0/1051696702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16ddfd700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc168107570 0x7fc16819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.937+0000 7fc16cdfb700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168102e90 0x7fc1681a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 -- 172.21.15.45:0/1051696702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc168107570 msgr2=0x7fc16819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc168107570 0x7fc16819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 -- 172.21.15.45:0/1051696702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168102e90 msgr2=0x7fc1681a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stdout:343597383710 2024-04-25T00:20:57.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168102e90 0x7fc1681a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 -- 172.21.15.45:0/1051696702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16819b1b0 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16d5fc700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d980 tx=0x7fc16400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164017070 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc164005020 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16819b4a0 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400fbf0 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.938+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1681a7710 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.939+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc168068f20 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.940+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1640051d0 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.941+0000 7fc15e7fc700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 0x7fc1540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.941+0000 7fc16cdfb700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 0x7fc1540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.941+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc1640cf340 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.942+0000 7fc16cdfb700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 0x7fc1540a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc158006010 tx=0x7fc158005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:52.946+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc164020020 con 0x7fc1681024e0 2024-04-25T00:20:57.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.156+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc1681a78e0 con 0x7fc1681024e0 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.156+0000 7fc15e7fc700 1 -- 172.21.15.45:0/1051696702 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc16409be00 con 0x7fc1681024e0 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 msgr2=0x7fc1540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 0x7fc1540a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc158006010 tx=0x7fc158005960 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 msgr2=0x7fc1681a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d980 tx=0x7fc16400dc90 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 shutdown_connections 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc168107570 0x7fc16819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc1540a2cb0 0x7fc1540a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1681024e0 0x7fc1681a0840 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 --2- 172.21.15.45:0/1051696702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc168102e90 0x7fc1681a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 >> 172.21.15.45:0/1051696702 conn(0x7fc168079390 msgr2=0x7fc168101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.158+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 shutdown_connections 2024-04-25T00:20:57.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:53.159+0000 7fc16f860700 1 -- 172.21.15.45:0/1051696702 wait complete. 2024-04-25T00:20:58.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383706 got 343597383710 for osd.11 2024-04-25T00:20:58.377 DEBUG:teuthology.parallel:result is None 2024-04-25T00:20:58.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:58 smithi045 ceph-mon[29194]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 -- 172.21.15.45:0/1626307833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 msgr2=0x7f985806a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 --2- 172.21.15.45:0/1626307833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f985806a3a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f9848009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 -- 172.21.15.45:0/1626307833 shutdown_connections 2024-04-25T00:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 --2- 172.21.15.45:0/1626307833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 0x7f985810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 --2- 172.21.15.45:0/1626307833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f9858108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 --2- 172.21.15.45:0/1626307833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f985806a3a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.013+0000 7f985fb53700 1 -- 172.21.15.45:0/1626307833 >> 172.21.15.45:0/1626307833 conn(0x7f98580780e0 msgr2=0x7f98580784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- 172.21.15.45:0/1626307833 shutdown_connections 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- 172.21.15.45:0/1626307833 wait complete. 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 Processor -- start 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- start start 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 0x7f98581a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f985819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9858101ed0 con 0x7f9858069f40 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9858101d50 con 0x7f98581082b0 2024-04-25T00:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.014+0000 7f985fb53700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9858102050 con 0x7f985806a8e0 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56738/0 (socket says 172.21.15.45:56738) 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 -- 172.21.15.45:0/1355192425 learned_addr learned my addr 172.21.15.45:0/1355192425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d0ee700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 0x7f98581a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985e0f0700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f985819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 -- 172.21.15.45:0/1355192425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 msgr2=0x7f98581a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 0x7f98581a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 -- 172.21.15.45:0/1355192425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 msgr2=0x7f985819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f985819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 -- 172.21.15.45:0/1355192425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985819b100 con 0x7f9858069f40 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985e0f0700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f985819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985d8ef700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854016070 con 0x7f9858069f40 2024-04-25T00:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854009b40 con 0x7f9858069f40 2024-04-25T00:20:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400f620 con 0x7f9858069f40 2024-04-25T00:20:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985819b3f0 con 0x7f9858069f40 2024-04-25T00:20:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.015+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a76b0 con 0x7f9858069f40 2024-04-25T00:20:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.017+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9858068f20 con 0x7f9858069f40 2024-04-25T00:20:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.019+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9854019070 con 0x7f9858069f40 2024-04-25T00:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.019+0000 7f984effd700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 0x7f98440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.020+0000 7f985d0ee700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 0x7f98440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.020+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f98540cf4a0 con 0x7f9858069f40 2024-04-25T00:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.020+0000 7f985d0ee700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 0x7f98440a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9848005b40 tx=0x7f9848005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.022+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f985409bf60 con 0x7f9858069f40 2024-04-25T00:20:59.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:58 smithi122 ceph-mon[32959]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:58 smithi071 ceph-mon[32667]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:20:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.238+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f98581a7880 con 0x7f9858069f40 2024-04-25T00:20:59.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.238+0000 7f984effd700 1 -- 172.21.15.45:0/1355192425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f985409b860 con 0x7f9858069f40 2024-04-25T00:20:59.239 INFO:teuthology.orchestra.run.smithi045.stdout:107374182472 2024-04-25T00:20:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 msgr2=0x7f98440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 0x7f98440a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9848005b40 tx=0x7f9848005a90 comp rx=0 tx=0).stop 2024-04-25T00:20:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 msgr2=0x7f98581a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400dd10 comp rx=0 tx=0).stop 2024-04-25T00:20:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 shutdown_connections 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f98440a2cb0 0x7f98440a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98581082b0 0x7f985819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9858069f40 0x7f98581a07e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 --2- 172.21.15.45:0/1355192425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985806a8e0 0x7f98581a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 >> 172.21.15.45:0/1355192425 conn(0x7f98580780e0 msgr2=0x7f9858100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 shutdown_connections 2024-04-25T00:20:59.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.240+0000 7f985fb53700 1 -- 172.21.15.45:0/1355192425 wait complete. 2024-04-25T00:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 -- 172.21.15.45:0/3838602195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 msgr2=0x7f88281015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 --2- 172.21.15.45:0/3838602195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f88281015b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8810009990 tx=0x7f8810009ca0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 -- 172.21.15.45:0/3838602195 shutdown_connections 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 --2- 172.21.15.45:0/3838602195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 0x7f8828107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 --2- 172.21.15.45:0/3838602195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 --2- 172.21.15.45:0/3838602195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f88281015b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 -- 172.21.15.45:0/3838602195 >> 172.21.15.45:0/3838602195 conn(0x7f88280780e0 msgr2=0x7f88280784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 -- 172.21.15.45:0/3838602195 shutdown_connections 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.930+0000 7f882ca1f700 1 -- 172.21.15.45:0/3838602195 wait complete. 2024-04-25T00:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 Processor -- start 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 -- start start 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 0x7f8828198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8828111430 con 0x7f8828101130 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88281112b0 con 0x7f8828101af0 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882ca1f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88281115b0 con 0x7f88281071a0 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56714/0 (socket says 172.21.15.45:56714) 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 -- 172.21.15.45:0/2811304086 learned_addr learned my addr 172.21.15.45:0/2811304086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f8826d9d700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 0x7f8828198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f8825d9b700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 -- 172.21.15.45:0/2811304086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 msgr2=0x7f8828198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 0x7f8828198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 -- 172.21.15.45:0/2811304086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 msgr2=0x7f882819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.931+0000 7f882659c700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f882659c700 1 -- 172.21.15.45:0/2811304086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8828198fe0 con 0x7f8828101130 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f8825d9b700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f882659c700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f881800d980 tx=0x7f881800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818017070 con 0x7f8828101130 2024-04-25T00:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8818005020 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88281992d0 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818009bc0 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.932+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88281a54e0 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.933+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8828102850 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.934+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f88180051d0 con 0x7f8828101130 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.935+0000 7f881f7fe700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 0x7f88140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.935+0000 7f8825d9b700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 0x7f88140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.935+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f88180cf430 con 0x7f8828101130 2024-04-25T00:20:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.935+0000 7f8825d9b700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 0x7f88140a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f881000b2e0 tx=0x7f8810005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:58.940+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8818020020 con 0x7f8828101130 2024-04-25T00:20:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.155+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f88281a5770 con 0x7f8828101130 2024-04-25T00:20:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.155+0000 7f881f7fe700 1 -- 172.21.15.45:0/2811304086 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f881809bef0 con 0x7f8828101130 2024-04-25T00:20:59.560 INFO:teuthology.orchestra.run.smithi045.stdout:197568495673 2024-04-25T00:20:59.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 msgr2=0x7f88140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 0x7f88140a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f881000b2e0 tx=0x7f8810005e80 comp rx=0 tx=0).stop 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 msgr2=0x7f882819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f881800d980 tx=0x7f881800dc90 comp rx=0 tx=0).stop 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 shutdown_connections 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88281071a0 0x7f8828198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f88140a2cb0 0x7f88140a5170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8828101130 0x7f882819e680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 --2- 172.21.15.45:0/2811304086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8828101af0 0x7f882819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.157+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 >> 172.21.15.45:0/2811304086 conn(0x7f88280780e0 msgr2=0x7f88281003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.158+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 shutdown_connections 2024-04-25T00:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:20:59.158+0000 7f882ca1f700 1 -- 172.21.15.45:0/2811304086 wait complete. 2024-04-25T00:20:59.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2811304086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:20:59.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:20:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1355192425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:21:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2811304086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:21:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:20:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1355192425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:21:00.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2811304086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:21:00.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:20:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1355192425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:21:00.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- 172.21.15.45:0/442322022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f0102e90 msgr2=0x7f63f0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 --2- 172.21.15.45:0/442322022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f0102e90 0x7f63f0106ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009990 tx=0x7f63e0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- 172.21.15.45:0/442322022 shutdown_connections 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 --2- 172.21.15.45:0/442322022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f01075a0 0x7f63f010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 --2- 172.21.15.45:0/442322022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 0x7f63f01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 --2- 172.21.15.45:0/442322022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f0102e90 0x7f63f0106ec0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- 172.21.15.45:0/442322022 >> 172.21.15.45:0/442322022 conn(0x7f63f00793e0 msgr2=0x7f63f00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- 172.21.15.45:0/442322022 shutdown_connections 2024-04-25T00:21:00.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- 172.21.15.45:0/442322022 wait complete. 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 Processor -- start 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.281+0000 7f63f8503700 1 -- start start 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 0x7f63f019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f0102e90 0x7f63f019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0112540 con 0x7f63f01075a0 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01123c0 con 0x7f63f01024e0 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f8503700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63f01126c0 con 0x7f63f0102e90 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56762/0 (socket says 172.21.15.45:56762) 2024-04-25T00:21:00.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 -- 172.21.15.45:0/3035761588 learned_addr learned my addr 172.21.15.45:0/3035761588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f5a9e700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f0102e90 0x7f63f019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f629f700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 0x7f63f019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 -- 172.21.15.45:0/3035761588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f0102e90 msgr2=0x7f63f019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f0102e90 0x7f63f019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 -- 172.21.15.45:0/3035761588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 msgr2=0x7f63f019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 0x7f63f019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.282+0000 7f63f6aa0700 1 -- 172.21.15.45:0/3035761588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f0199040 con 0x7f63f01075a0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63f6aa0700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f63e800cca0 tx=0x7f63e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8016020 con 0x7f63f01075a0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f0199330 con 0x7f63f01075a0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e8012420 con 0x7f63f01075a0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80115e0 con 0x7f63f01075a0 2024-04-25T00:21:00.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.283+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a5610 con 0x7f63f01075a0 2024-04-25T00:21:00.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.284+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f01039a0 con 0x7f63f01075a0 2024-04-25T00:21:00.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.285+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f63e8019020 con 0x7f63f01075a0 2024-04-25T00:21:00.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.286+0000 7f63e77fe700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 0x7f63dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:00.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.286+0000 7f63f629f700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 0x7f63dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:00.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.286+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f63e80cf1b0 con 0x7f63f01075a0 2024-04-25T00:21:00.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.286+0000 7f63f629f700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 0x7f63dc0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00ac50 tx=0x7f63ec00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:00.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.290+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63e8020020 con 0x7f63f01075a0 2024-04-25T00:21:00.512 INFO:teuthology.orchestra.run.smithi045.stdout:163208757310 2024-04-25T00:21:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.510+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f63f0199c90 con 0x7f63f01075a0 2024-04-25T00:21:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.511+0000 7f63e77fe700 1 -- 172.21.15.45:0/3035761588 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f63e809bd80 con 0x7f63f01075a0 2024-04-25T00:21:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 msgr2=0x7f63dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 0x7f63dc0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00ac50 tx=0x7f63ec00a380 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 msgr2=0x7f63f0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f63e800cca0 tx=0x7f63e800a060 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 shutdown_connections 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f63dc0a2cb0 0x7f63dc0a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63f01075a0 0x7f63f0198760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63f01024e0 0x7f63f019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 --2- 172.21.15.45:0/3035761588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63f0102e90 0x7f63f019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 >> 172.21.15.45:0/3035761588 conn(0x7f63f00793e0 msgr2=0x7f63f0100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 shutdown_connections 2024-04-25T00:21:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:00.515+0000 7f63f8503700 1 -- 172.21.15.45:0/3035761588 wait complete. 2024-04-25T00:21:00.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182472 for osd.3 2024-04-25T00:21:00.841 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3035761588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:00 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3035761588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:21:01.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:01.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:01.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:00 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3035761588' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:00 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' 2024-04-25T00:21:01.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574929 for osd.1 2024-04-25T00:21:01.352 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 -- 172.21.15.45:0/3898931492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406b2a0 msgr2=0x7f3e2410f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 --2- 172.21.15.45:0/3898931492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406b2a0 0x7f3e2410f110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b0a0 tx=0x7f3e1c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 -- 172.21.15.45:0/3898931492 shutdown_connections 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 --2- 172.21.15.45:0/3898931492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e241086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 --2- 172.21.15.45:0/3898931492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406b2a0 0x7f3e2410f110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 --2- 172.21.15.45:0/3898931492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406a900 0x7f3e2406ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 -- 172.21.15.45:0/3898931492 >> 172.21.15.45:0/3898931492 conn(0x7f3e240780e0 msgr2=0x7f3e240784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 -- 172.21.15.45:0/3898931492 shutdown_connections 2024-04-25T00:21:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.949+0000 7f3e2c705700 1 -- 172.21.15.45:0/3898931492 wait complete. 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.950+0000 7f3e2c705700 1 Processor -- start 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.950+0000 7f3e2c705700 1 -- start start 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406b2a0 0x7f3e2419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e24198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24112540 con 0x7f3e2406a900 2024-04-25T00:21:01.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241123c0 con 0x7f3e2406b2a0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2c705700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241126c0 con 0x7f3e241082d0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56782/0 (socket says 172.21.15.45:56782) 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 -- 172.21.15.45:0/4213804559 learned_addr learned my addr 172.21.15.45:0/4213804559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e29ca0700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406b2a0 0x7f3e2419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2aca2700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e24198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 -- 172.21.15.45:0/4213804559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 msgr2=0x7f3e24198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e24198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 -- 172.21.15.45:0/4213804559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406b2a0 msgr2=0x7f3e2419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406b2a0 0x7f3e2419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2a4a1700 1 -- 172.21.15.45:0/4213804559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e24198fc0 con 0x7f3e2406a900 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.951+0000 7f3e2aca2700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e24198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e2a4a1700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d980 tx=0x7f3e1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14017070 con 0x7f3e2406a900 2024-04-25T00:21:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e241992b0 con 0x7f3e2406a900 2024-04-25T00:21:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14005020 con 0x7f3e2406a900 2024-04-25T00:21:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14009bc0 con 0x7f3e2406a900 2024-04-25T00:21:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.952+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a5570 con 0x7f3e2406a900 2024-04-25T00:21:01.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.953+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e24103980 con 0x7f3e2406a900 2024-04-25T00:21:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.954+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3e140051d0 con 0x7f3e2406a900 2024-04-25T00:21:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.955+0000 7f3e1b7fe700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 0x7f3e100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.955+0000 7f3e29ca0700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 0x7f3e100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.955+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f3e140cf860 con 0x7f3e2406a900 2024-04-25T00:21:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.955+0000 7f3e29ca0700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 0x7f3e100a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3e24078770 tx=0x7f3e2000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:01.959+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e14020020 con 0x7f3e2406a900 2024-04-25T00:21:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.172+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3e2402db30 con 0x7f3e2406a900 2024-04-25T00:21:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.173+0000 7f3e1b7fe700 1 -- 172.21.15.45:0/4213804559 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3e1409c2c0 con 0x7f3e2406a900 2024-04-25T00:21:02.174 INFO:teuthology.orchestra.run.smithi045.stdout:141733920836 2024-04-25T00:21:02.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 msgr2=0x7f3e100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 0x7f3e100a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3e24078770 tx=0x7f3e2000b410 comp rx=0 tx=0).stop 2024-04-25T00:21:02.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 msgr2=0x7f3e2419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d980 tx=0x7f3e1400dc90 comp rx=0 tx=0).stop 2024-04-25T00:21:02.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 shutdown_connections 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e241082d0 0x7f3e24198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f3e100a2cb0 0x7f3e100a5170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2406a900 0x7f3e2419e6a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 --2- 172.21.15.45:0/4213804559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e2406b2a0 0x7f3e2419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.176+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 >> 172.21.15.45:0/4213804559 conn(0x7f3e240780e0 msgr2=0x7f3e24100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.177+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 shutdown_connections 2024-04-25T00:21:02.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.177+0000 7f3e2c705700 1 -- 172.21.15.45:0/4213804559 wait complete. 2024-04-25T00:21:02.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 -- 172.21.15.45:0/2157564094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 msgr2=0x7f534810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2157564094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f534810dfe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f534000b0a0 tx=0x7f534000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 -- 172.21.15.45:0/2157564094 shutdown_connections 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2157564094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53481071a0 0x7f5348107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2157564094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f534810dfe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2157564094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5348101130 0x7f53481015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 -- 172.21.15.45:0/2157564094 >> 172.21.15.45:0/2157564094 conn(0x7f53480780e0 msgr2=0x7f53480784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 -- 172.21.15.45:0/2157564094 shutdown_connections 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.256+0000 7f534d3cd700 1 -- 172.21.15.45:0/2157564094 wait complete. 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 Processor -- start 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 -- start start 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5348101130 0x7f53481998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53481071a0 0x7f53481a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5348111430 con 0x7f5348101af0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53481112b0 con 0x7f53481071a0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f534d3cd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53481115b0 con 0x7f5348101130 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f53467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f53467fc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56810/0 (socket says 172.21.15.45:56810) 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f53467fc700 1 -- 172.21.15.45:0/2344292833 learned_addr learned my addr 172.21.15.45:0/2344292833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.257+0000 7f5346ffd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5348101130 0x7f53481998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 -- 172.21.15.45:0/2344292833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5348101130 msgr2=0x7f53481998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5348101130 0x7f53481998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53477fe700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53481071a0 0x7f53481a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 -- 172.21.15.45:0/2344292833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53481071a0 msgr2=0x7f53481a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53481071a0 0x7f53481a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 -- 172.21.15.45:0/2344292833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53481a4e70 con 0x7f5348101af0 2024-04-25T00:21:02.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f53467fc700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f533c00e9a0 tx=0x7f533c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c00cbd0 con 0x7f5348101af0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53481a5040 con 0x7f5348101af0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c00cd70 con 0x7f5348101af0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c0187c0 con 0x7f5348101af0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.258+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53481a5580 con 0x7f5348101af0 2024-04-25T00:21:02.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.259+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5348068f20 con 0x7f5348101af0 2024-04-25T00:21:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.261+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f533c010430 con 0x7f5348101af0 2024-04-25T00:21:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.262+0000 7f532ffff700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 0x7f53300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.262+0000 7f5346ffd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 0x7f53300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.262+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f533c0cfd10 con 0x7f5348101af0 2024-04-25T00:21:02.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.262+0000 7f5346ffd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 0x7f53300a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5338005fd0 tx=0x7f5338005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:02.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.265+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f533c09c770 con 0x7f5348101af0 2024-04-25T00:21:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.472+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f534806a500 con 0x7f5348101af0 2024-04-25T00:21:02.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.473+0000 7f532ffff700 1 -- 172.21.15.45:0/2344292833 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f533c09c070 con 0x7f5348101af0 2024-04-25T00:21:02.473 INFO:teuthology.orchestra.run.smithi045.stdout:287762808874 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 msgr2=0x7f53300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 0x7f53300a5170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5338005fd0 tx=0x7f5338005d90 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 msgr2=0x7f5348199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f533c00e9a0 tx=0x7f533c00ed60 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 shutdown_connections 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5348101130 0x7f53481998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53300a2cb0 0x7f53300a5170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53481071a0 0x7f53481a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 --2- 172.21.15.45:0/2344292833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5348101af0 0x7f5348199df0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 >> 172.21.15.45:0/2344292833 conn(0x7f53480780e0 msgr2=0x7f53480ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 shutdown_connections 2024-04-25T00:21:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:02.475+0000 7f534d3cd700 1 -- 172.21.15.45:0/2344292833 wait complete. 2024-04-25T00:21:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:02 smithi071 ceph-mon[32667]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4213804559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:21:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2344292833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:21:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:02 smithi045 ceph-mon[29194]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4213804559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:21:03.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2344292833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:21:03.202 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808869 got 287762808874 for osd.9 2024-04-25T00:21:03.202 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:02 smithi122 ceph-mon[32959]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4213804559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:21:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2344292833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:21:03.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757310 for osd.5 2024-04-25T00:21:03.748 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:04.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920836 for osd.4 2024-04-25T00:21:04.728 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:05.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:04 smithi045 ceph-mon[29194]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:05.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:04 smithi071 ceph-mon[32667]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- 172.21.15.45:0/1670972904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 msgr2=0x7eff9010f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 --2- 172.21.15.45:0/1670972904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff9010f1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff8800b0d0 tx=0x7eff8800b3e0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- 172.21.15.45:0/1670972904 shutdown_connections 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 --2- 172.21.15.45:0/1670972904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff90100e30 0x7eff90101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 --2- 172.21.15.45:0/1670972904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff9010f1b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 --2- 172.21.15.45:0/1670972904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff901017e0 0x7eff90106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- 172.21.15.45:0/1670972904 >> 172.21.15.45:0/1670972904 conn(0x7eff90076450 msgr2=0x7eff90076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- 172.21.15.45:0/1670972904 shutdown_connections 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- 172.21.15.45:0/1670972904 wait complete. 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 Processor -- start 2024-04-25T00:21:05.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.235+0000 7eff963fa700 1 -- start start 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 0x7eff90101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff901017e0 0x7eff901022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff901125d0 con 0x7eff90107650 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7eff90112450 con 0x7eff90100e30 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff963fa700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff90112750 con 0x7eff901017e0 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33616/0 (socket says 172.21.15.45:33616) 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 -- 172.21.15.45:0/4187672799 learned_addr learned my addr 172.21.15.45:0/4187672799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff8f7fe700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff901017e0 0x7eff901022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff8ffff700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 0x7eff90101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 -- 172.21.15.45:0/4187672799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff901017e0 msgr2=0x7eff901022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff901017e0 0x7eff901022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 -- 172.21.15.45:0/4187672799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 msgr2=0x7eff90101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 0x7eff90101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 -- 172.21.15.45:0/4187672799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff901a6e10 con 0x7eff90107650 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff8ffff700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 0x7eff90101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.236+0000 7eff94997700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7eff88006070 tx=0x7eff88004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.237+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88021070 con 0x7eff90107650 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.237+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff88007d00 con 0x7eff90107650 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.237+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff901a70a0 con 0x7eff90107650 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.237+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88005470 con 0x7eff90107650 2024-04-25T00:21:05.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.237+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff901a75a0 con 0x7eff90107650 2024-04-25T00:21:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.238+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff90068f20 con 0x7eff90107650 2024-04-25T00:21:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.239+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff88024070 con 0x7eff90107650 2024-04-25T00:21:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.239+0000 7eff8d7fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 0x7eff7c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.240+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7eff8800f020 con 0x7eff90107650 2024-04-25T00:21:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.240+0000 7eff8ffff700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 0x7eff7c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.240+0000 7eff8ffff700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 0x7eff7c0a51f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff800099d0 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:05.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.243+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff8802b020 con 0x7eff90107650 2024-04-25T00:21:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:04 smithi122 ceph-mon[32959]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.452+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7eff901a7960 con 0x7eff90107650 2024-04-25T00:21:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.452+0000 7eff8d7fa700 1 -- 172.21.15.45:0/4187672799 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7eff8809dab0 con 0x7eff90107650 2024-04-25T00:21:05.453 INFO:teuthology.orchestra.run.smithi045.stdout:34359738455 2024-04-25T00:21:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 msgr2=0x7eff7c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 0x7eff7c0a51f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff800099d0 tx=0x7eff80008040 comp rx=0 tx=0).stop 2024-04-25T00:21:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 msgr2=0x7eff90102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7eff88006070 tx=0x7eff88004e40 comp rx=0 tx=0).stop 2024-04-25T00:21:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 shutdown_connections 2024-04-25T00:21:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7eff7c0a2d30 0x7eff7c0a51f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff90107650 0x7eff90102830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff90100e30 0x7eff90101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 --2- 172.21.15.45:0/4187672799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff901017e0 0x7eff901022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 >> 172.21.15.45:0/4187672799 conn(0x7eff90076450 msgr2=0x7eff900feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 shutdown_connections 2024-04-25T00:21:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.454+0000 7eff963fa700 1 -- 172.21.15.45:0/4187672799 wait complete. 2024-04-25T00:21:05.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 -- 172.21.15.45:0/1629457276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580115380 msgr2=0x7fa580117770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 --2- 172.21.15.45:0/1629457276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580115380 0x7fa580117770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b0a0 tx=0x7fa57800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 -- 172.21.15.45:0/1629457276 shutdown_connections 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 --2- 172.21.15.45:0/1629457276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109230 0x7fa580109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 --2- 172.21.15.45:0/1629457276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580115380 0x7fa580117770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 --2- 172.21.15.45:0/1629457276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580109b50 0x7fa580114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 -- 172.21.15.45:0/1629457276 >> 172.21.15.45:0/1629457276 conn(0x7fa58006d3a0 msgr2=0x7fa58006d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 -- 172.21.15.45:0/1629457276 shutdown_connections 2024-04-25T00:21:05.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.962+0000 7fa584a86700 1 -- 172.21.15.45:0/1629457276 wait complete. 2024-04-25T00:21:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:05 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4187672799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 Processor -- start 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 -- start start 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109b50 0x7fa58010b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580115380 0x7fa58010b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa58011ade0 con 0x7fa580109230 2024-04-25T00:21:05.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa58011ac60 con 0x7fa580115380 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.963+0000 7fa584a86700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa58011af60 con 0x7fa580109b50 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33634/0 (socket says 172.21.15.45:33634) 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 -- 172.21.15.45:0/1649708145 learned_addr learned my addr 172.21.15.45:0/1649708145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57dd9b700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109b50 0x7fa58010b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57ed9d700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580115380 0x7fa58010b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 -- 172.21.15.45:0/1649708145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109b50 msgr2=0x7fa58010b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109b50 0x7fa58010b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 -- 172.21.15.45:0/1649708145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580115380 msgr2=0x7fa58010b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580115380 0x7fa58010b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 -- 172.21.15.45:0/1649708145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5801af6b0 con 0x7fa580109230 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa57e59c700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa57000c9b0 tx=0x7fa57000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570007b00 con 0x7fa580109230 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5801af9a0 con 0x7fa580109230 2024-04-25T00:21:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57000ce50 con 0x7fa580109230 2024-04-25T00:21:05.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570018660 con 0x7fa580109230 2024-04-25T00:21:05.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.964+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801aff30 con 0x7fa580109230 2024-04-25T00:21:05.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.965+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa580068f20 con 0x7fa580109230 2024-04-25T00:21:05.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.967+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa57000f450 con 0x7fa580109230 2024-04-25T00:21:05.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.967+0000 7fa56f7fe700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 0x7fa5680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:05.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.967+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa5700cf6b0 con 0x7fa580109230 2024-04-25T00:21:05.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.967+0000 7fa57dd9b700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 0x7fa5680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:05.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.969+0000 7fa57dd9b700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 0x7fa5680a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa580116e60 tx=0x7fa57400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:05.972+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa570014030 con 0x7fa580109230 2024-04-25T00:21:06.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:05 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4187672799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:21:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.181+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa5801b01c0 con 0x7fa580109230 2024-04-25T00:21:06.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.181+0000 7fa56f7fe700 1 -- 172.21.15.45:0/1649708145 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa57009c200 con 0x7fa580109230 2024-04-25T00:21:06.182 INFO:teuthology.orchestra.run.smithi045.stdout:219043332149 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 msgr2=0x7fa5680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 0x7fa5680a5170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa580116e60 tx=0x7fa57400a300 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 msgr2=0x7fa58010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa57000c9b0 tx=0x7fa57000ccc0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 shutdown_connections 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa5680a2cb0 0x7fa5680a5170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa580115380 0x7fa58010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa580109230 0x7fa58010aca0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 --2- 172.21.15.45:0/1649708145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa580109b50 0x7fa58010b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 >> 172.21.15.45:0/1649708145 conn(0x7fa58006d3a0 msgr2=0x7fa580070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 shutdown_connections 2024-04-25T00:21:06.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:06.184+0000 7fa584a86700 1 -- 172.21.15.45:0/1649708145 wait complete. 2024-04-25T00:21:06.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:05 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4187672799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:21:06.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495673 for osd.6 2024-04-25T00:21:06.724 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:06.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:06 smithi045 ceph-mon[29194]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:06.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1649708145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:21:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:06 smithi071 ceph-mon[32667]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1649708145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:21:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:06 smithi122 ceph-mon[32959]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1649708145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:21:07.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738455 for osd.0 2024-04-25T00:21:07.351 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:07.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332149 for osd.7 2024-04-25T00:21:07.978 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 -- 172.21.15.45:0/2981800880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8100820 msgr2=0x7f01f8100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 --2- 172.21.15.45:0/2981800880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8100820 0x7f01f8100c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009990 tx=0x7f01e8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 -- 172.21.15.45:0/2981800880 shutdown_connections 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 --2- 172.21.15.45:0/2981800880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f81011c0 0x7f01f810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 --2- 172.21.15.45:0/2981800880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 0x7f01f81086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 --2- 172.21.15.45:0/2981800880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f8100820 0x7f01f8100c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 -- 172.21.15.45:0/2981800880 >> 172.21.15.45:0/2981800880 conn(0x7f01f80777a0 msgr2=0x7f01f80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 -- 172.21.15.45:0/2981800880 shutdown_connections 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.192+0000 7f01ff52d700 1 -- 172.21.15.45:0/2981800880 wait complete. 2024-04-25T00:21:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 Processor -- start 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 -- start start 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f8100820 0x7f01f819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 0x7f01f8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8112540 con 0x7f01f81011c0 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81123c0 con 0x7f01f81082c0 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01ff52d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81126c0 con 0x7f01f8100820 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01fcac8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01fcac8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33658/0 (socket says 172.21.15.45:33658) 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01fcac8700 1 -- 172.21.15.45:0/306280485 learned_addr learned my addr 172.21.15.45:0/306280485 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.193+0000 7f01fd2c9700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f8100820 0x7f01f819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fdaca700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 0x7f01f8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 -- 172.21.15.45:0/306280485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f8100820 msgr2=0x7f01f819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f8100820 0x7f01f819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 -- 172.21.15.45:0/306280485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 msgr2=0x7f01f8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 0x7f01f8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 -- 172.21.15.45:0/306280485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f8199000 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01fcac8700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8027070 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f8199290 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f81a5500 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e8004050 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.194+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8031060 con 0x7f01f81011c0 2024-04-25T00:21:08.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.195+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc005330 con 0x7f01f81011c0 2024-04-25T00:21:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.196+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f01e802a070 con 0x7f01f81011c0 2024-04-25T00:21:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.197+0000 7f01ee7fc700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 0x7f01e40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.197+0000 7f01fd2c9700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 0x7f01e40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.197+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f01e80d0110 con 0x7f01f81011c0 2024-04-25T00:21:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.198+0000 7f01fd2c9700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 0x7f01e40a51f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01f400abd0 tx=0x7f01f400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.201+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01e8013070 con 0x7f01f81011c0 2024-04-25T00:21:08.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.408+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f01dc0058f0 con 0x7f01f81011c0 2024-04-25T00:21:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.408+0000 7f01ee7fc700 1 -- 172.21.15.45:0/306280485 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f01e809cc60 con 0x7f01f81011c0 2024-04-25T00:21:08.409 INFO:teuthology.orchestra.run.smithi045.stdout:253403070512 2024-04-25T00:21:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 msgr2=0x7f01e40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 0x7f01e40a51f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01f400abd0 tx=0x7f01f400a380 comp rx=0 tx=0).stop 2024-04-25T00:21:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 msgr2=0x7f01f819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e8003680 comp rx=0 tx=0).stop 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 shutdown_connections 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f8100820 0x7f01f819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f01e40a2d30 0x7f01e40a51f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f81082c0 0x7f01f8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 --2- 172.21.15.45:0/306280485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f81011c0 0x7f01f819ebe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.410+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 >> 172.21.15.45:0/306280485 conn(0x7f01f80777a0 msgr2=0x7f01f80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.411+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 shutdown_connections 2024-04-25T00:21:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:08.411+0000 7f01ff52d700 1 -- 172.21.15.45:0/306280485 wait complete. 2024-04-25T00:21:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:08 smithi045 ceph-mon[29194]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:08 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/306280485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:21:09.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070512 for osd.8 2024-04-25T00:21:09.242 DEBUG:teuthology.parallel:result is None 2024-04-25T00:21:09.242 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T00:21:09.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:21:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:08 smithi122 ceph-mon[32959]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:08 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/306280485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:21:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:08 smithi071 ceph-mon[32667]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:08 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/306280485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:21:09.499 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:10.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:10 smithi045 ceph-mon[29194]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 -- 172.21.15.45:0/1836879443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c101d70 msgr2=0x7fbf0c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/1836879443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c101d70 0x7fbf0c105d70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf000099c0 tx=0x7fbf00009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 -- 172.21.15.45:0/1836879443 shutdown_connections 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/1836879443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c106450 0x7fbf0c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/1836879443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c1013c0 0x7fbf0c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/1836879443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c101d70 0x7fbf0c105d70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 -- 172.21.15.45:0/1836879443 >> 172.21.15.45:0/1836879443 conn(0x7fbf0c078100 msgr2=0x7fbf0c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 -- 172.21.15.45:0/1836879443 shutdown_connections 2024-04-25T00:21:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.232+0000 7fbf12f8a700 1 -- 172.21.15.45:0/1836879443 wait complete. 2024-04-25T00:21:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 Processor -- start 2024-04-25T00:21:11.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 -- start start 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c1013c0 0x7fbf0c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c101d70 0x7fbf0c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111490 con 0x7fbf0c106450 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111310 con 0x7fbf0c101d70 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf12f8a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c111610 con 0x7fbf0c1013c0 2024-04-25T00:21:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33686/0 (socket says 172.21.15.45:33686) 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 -- 172.21.15.45:0/2350704774 learned_addr learned my addr 172.21.15.45:0/2350704774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf0bfff700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c101d70 0x7fbf0c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf10d26700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c1013c0 0x7fbf0c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 -- 172.21.15.45:0/2350704774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c1013c0 msgr2=0x7fbf0c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c1013c0 0x7fbf0c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 -- 172.21.15.45:0/2350704774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c101d70 msgr2=0x7fbf0c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c101d70 0x7fbf0c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.233+0000 7fbf11527700 1 -- 172.21.15.45:0/2350704774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c199000 con 0x7fbf0c106450 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf11527700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b970 tx=0x7fbf0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400d860 con 0x7fbf0c106450 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0400e430 con 0x7fbf0c106450 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1992f0 con 0x7fbf0c106450 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04017660 con 0x7fbf0c106450 2024-04-25T00:21:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.234+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a55b0 con 0x7fbf0c106450 2024-04-25T00:21:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.235+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0c068f20 con 0x7fbf0c106450 2024-04-25T00:21:11.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.237+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf0400e5e0 con 0x7fbf0c106450 2024-04-25T00:21:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.238+0000 7fbf09ffb700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 0x7fbef40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.238+0000 7fbf10d26700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 0x7fbef40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:11.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.238+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fbf040cf340 con 0x7fbf0c106450 2024-04-25T00:21:11.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.238+0000 7fbf10d26700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 0x7fbef40a5170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c1999e0 tx=0x7fbefc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:11.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.241+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbf04013030 con 0x7fbf0c106450 2024-04-25T00:21:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:10 smithi122 ceph-mon[32959]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:10 smithi071 ceph-mon[32667]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:11.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.442+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf0c062670 con 0x7fbef40a2cb0 2024-04-25T00:21:11.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.452+0000 7fbf09ffb700 1 -- 172.21.15.45:0/2350704774 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340690 (secure 0 0 0) 0x7fbf0c062670 con 0x7fbef40a2cb0 2024-04-25T00:21:11.458 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 msgr2=0x7fbef40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:11.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 0x7fbef40a5170 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c1999e0 tx=0x7fbefc0073d0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 msgr2=0x7fbf0c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b970 tx=0x7fbf0400bc80 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 shutdown_connections 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf0c1013c0 0x7fbf0c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbef40a2cb0 0x7fbef40a5170 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf0c106450 0x7fbf0c1988a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 --2- 172.21.15.45:0/2350704774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf0c101d70 0x7fbf0c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 >> 172.21.15.45:0/2350704774 conn(0x7fbf0c078100 msgr2=0x7fbf0c06ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 shutdown_connections 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:11.455+0000 7fbf12f8a700 1 -- 172.21.15.45:0/2350704774 wait complete. 2024-04-25T00:21:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:21:12.170 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-04-25T00:21:11.287303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658448,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":846,"num_read_kb":544,"num_write":421,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386480,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555344,"statfs":{"total":1151940427776,"available":1151544672256,"internally_reserved":0,"allocated":12591104,"data_stored":7960298,"data_compressed":355488,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003625"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044572+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:20:27.044572+0000","last_peered":"2024-04-25T00:20:27.044572+0000","last_clean":"2024-04-25T00:20:27.044572+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:20:27.044572+0000","last_undegraded":"2024-04-25T00:20:27.044572+0000","last_fullsized":"2024-04-25T00:20:27.044572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151575+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:20:27.151575+0000","last_peered":"2024-04-25T00:20:27.151575+0000","last_clean":"2024-04-25T00:20:27.151575+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:20:27.151575+0000","last_undegraded":"2024-04-25T00:20:27.151575+0000","last_fullsized":"2024-04-25T00:20:27.151575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151455+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:20:27.151455+0000","last_peered":"2024-04-25T00:20:27.151455+0000","last_clean":"2024-04-25T00:20:27.151455+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:20:27.151455+0000","last_undegraded":"2024-04-25T00:20:27.151455+0000","last_fullsized":"2024-04-25T00:20:27.151455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153002+0000","last_change":"2024-04-25T00:19:33.880302+0000","last_active":"2024-04-25T00:20:27.153002+0000","last_peered":"2024-04-25T00:20:27.153002+0000","last_clean":"2024-04-25T00:20:27.153002+0000","last_became_active":"2024-04-25T00:19:33.879849+0000","last_became_peered":"2024-04-25T00:19:33.879849+0000","last_unstale":"2024-04-25T00:20:27.153002+0000","last_undegraded":"2024-04-25T00:20:27.153002+0000","last_fullsized":"2024-04-25T00:20:27.153002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:32.449680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041494+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:20:27.041494+0000","last_peered":"2024-04-25T00:20:27.041494+0000","last_clean":"2024-04-25T00:20:27.041494+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:20:27.041494+0000","last_undegraded":"2024-04-25T00:20:27.041494+0000","last_fullsized":"2024-04-25T00:20:27.041494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046385+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:20:27.046385+0000","last_peered":"2024-04-25T00:20:27.046385+0000","last_clean":"2024-04-25T00:20:27.046385+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:20:27.046385+0000","last_undegraded":"2024-04-25T00:20:27.046385+0000","last_fullsized":"2024-04-25T00:20:27.046385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042227+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:20:27.042227+0000","last_peered":"2024-04-25T00:20:27.042227+0000","last_clean":"2024-04-25T00:20:27.042227+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:20:27.042227+0000","last_undegraded":"2024-04-25T00:20:27.042227+0000","last_fullsized":"2024-04-25T00:20:27.042227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135822+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:20:27.135822+0000","last_peered":"2024-04-25T00:20:27.135822+0000","last_clean":"2024-04-25T00:20:27.135822+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:20:27.135822+0000","last_undegraded":"2024-04-25T00:20:27.135822+0000","last_fullsized":"2024-04-25T00:20:27.135822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043011+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:20:27.043011+0000","last_peered":"2024-04-25T00:20:27.043011+0000","last_clean":"2024-04-25T00:20:27.043011+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:20:27.043011+0000","last_undegraded":"2024-04-25T00:20:27.043011+0000","last_fullsized":"2024-04-25T00:20:27.043011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.047266+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:20:27.047266+0000","last_peered":"2024-04-25T00:20:27.047266+0000","last_clean":"2024-04-25T00:20:27.047266+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:20:27.047266+0000","last_undegraded":"2024-04-25T00:20:27.047266+0000","last_fullsized":"2024-04-25T00:20:27.047266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044029+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:20:27.044029+0000","last_peered":"2024-04-25T00:20:27.044029+0000","last_clean":"2024-04-25T00:20:27.044029+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:20:27.044029+0000","last_undegraded":"2024-04-25T00:20:27.044029+0000","last_fullsized":"2024-04-25T00:20:27.044029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152857+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:20:27.152857+0000","last_peered":"2024-04-25T00:20:27.152857+0000","last_clean":"2024-04-25T00:20:27.152857+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:20:27.152857+0000","last_undegraded":"2024-04-25T00:20:27.152857+0000","last_fullsized":"2024-04-25T00:20:27.152857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048944+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:20:27.048944+0000","last_peered":"2024-04-25T00:20:27.048944+0000","last_clean":"2024-04-25T00:20:27.048944+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:20:27.048944+0000","last_undegraded":"2024-04-25T00:20:27.048944+0000","last_fullsized":"2024-04-25T00:20:27.048944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151234+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:20:27.151234+0000","last_peered":"2024-04-25T00:20:27.151234+0000","last_clean":"2024-04-25T00:20:27.151234+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:20:27.151234+0000","last_undegraded":"2024-04-25T00:20:27.151234+0000","last_fullsized":"2024-04-25T00:20:27.151234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151398+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:20:27.151398+0000","last_peered":"2024-04-25T00:20:27.151398+0000","last_clean":"2024-04-25T00:20:27.151398+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:20:27.151398+0000","last_undegraded":"2024-04-25T00:20:27.151398+0000","last_fullsized":"2024-04-25T00:20:27.151398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041714+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:20:27.041714+0000","last_peered":"2024-04-25T00:20:27.041714+0000","last_clean":"2024-04-25T00:20:27.041714+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:20:27.041714+0000","last_undegraded":"2024-04-25T00:20:27.041714+0000","last_fullsized":"2024-04-25T00:20:27.041714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044288+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:20:27.044288+0000","last_peered":"2024-04-25T00:20:27.044288+0000","last_clean":"2024-04-25T00:20:27.044288+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:20:27.044288+0000","last_undegraded":"2024-04-25T00:20:27.044288+0000","last_fullsized":"2024-04-25T00:20:27.044288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039592+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:20:27.039592+0000","last_peered":"2024-04-25T00:20:27.039592+0000","last_clean":"2024-04-25T00:20:27.039592+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:20:27.039592+0000","last_undegraded":"2024-04-25T00:20:27.039592+0000","last_fullsized":"2024-04-25T00:20:27.039592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041006+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:20:27.041006+0000","last_peered":"2024-04-25T00:20:27.041006+0000","last_clean":"2024-04-25T00:20:27.041006+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:20:27.041006+0000","last_undegraded":"2024-04-25T00:20:27.041006+0000","last_fullsized":"2024-04-25T00:20:27.041006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153842+0000","last_change":"2024-04-25T00:20:27.153842+0000","last_active":"2024-04-25T00:20:27.153842+0000","last_peered":"2024-04-25T00:20:27.153842+0000","last_clean":"2024-04-25T00:20:27.153842+0000","last_became_active":"2024-04-25T00:20:27.152421+0000","last_became_peered":"2024-04-25T00:20:27.152421+0000","last_unstale":"2024-04-25T00:20:27.153842+0000","last_undegraded":"2024-04-25T00:20:27.153842+0000","last_fullsized":"2024-04-25T00:20:27.153842+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:30.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:07.111686+0000","last_change":"2024-04-25T00:20:27.056346+0000","last_active":"2024-04-25T00:21:07.111686+0000","last_peered":"2024-04-25T00:21:07.111686+0000","last_clean":"2024-04-25T00:21:07.111686+0000","last_became_active":"2024-04-25T00:20:27.046470+0000","last_became_peered":"2024-04-25T00:20:27.046470+0000","last_unstale":"2024-04-25T00:21:07.111686+0000","last_undegraded":"2024-04-25T00:21:07.111686+0000","last_fullsized":"2024-04-25T00:21:07.111686+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:31.760140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135989+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:20:27.135989+0000","last_peered":"2024-04-25T00:20:27.135989+0000","last_clean":"2024-04-25T00:20:27.135989+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:20:27.135989+0000","last_undegraded":"2024-04-25T00:20:27.135989+0000","last_fullsized":"2024-04-25T00:20:27.135989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151867+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:20:27.151867+0000","last_peered":"2024-04-25T00:20:27.151867+0000","last_clean":"2024-04-25T00:20:27.151867+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:20:27.151867+0000","last_undegraded":"2024-04-25T00:20:27.151867+0000","last_fullsized":"2024-04-25T00:20:27.151867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136147+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:20:27.136147+0000","last_peered":"2024-04-25T00:20:27.136147+0000","last_clean":"2024-04-25T00:20:27.136147+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:20:27.136147+0000","last_undegraded":"2024-04-25T00:20:27.136147+0000","last_fullsized":"2024-04-25T00:20:27.136147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151599+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:20:27.151599+0000","last_peered":"2024-04-25T00:20:27.151599+0000","last_clean":"2024-04-25T00:20:27.151599+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:20:27.151599+0000","last_undegraded":"2024-04-25T00:20:27.151599+0000","last_fullsized":"2024-04-25T00:20:27.151599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152164+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:20:27.152164+0000","last_peered":"2024-04-25T00:20:27.152164+0000","last_clean":"2024-04-25T00:20:27.152164+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:20:27.152164+0000","last_undegraded":"2024-04-25T00:20:27.152164+0000","last_fullsized":"2024-04-25T00:20:27.152164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.173867+0000","last_change":"2024-04-25T00:20:27.173867+0000","last_active":"2024-04-25T00:20:27.173867+0000","last_peered":"2024-04-25T00:20:27.173867+0000","last_clean":"2024-04-25T00:20:27.173867+0000","last_became_active":"2024-04-25T00:20:27.154058+0000","last_became_peered":"2024-04-25T00:20:27.154058+0000","last_unstale":"2024-04-25T00:20:27.173867+0000","last_undegraded":"2024-04-25T00:20:27.173867+0000","last_fullsized":"2024-04-25T00:20:27.173867+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:55.519210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043781+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:20:27.043781+0000","last_peered":"2024-04-25T00:20:27.043781+0000","last_clean":"2024-04-25T00:20:27.043781+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:20:27.043781+0000","last_undegraded":"2024-04-25T00:20:27.043781+0000","last_fullsized":"2024-04-25T00:20:27.043781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151962+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:20:27.151962+0000","last_peered":"2024-04-25T00:20:27.151962+0000","last_clean":"2024-04-25T00:20:27.151962+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:20:27.151962+0000","last_undegraded":"2024-04-25T00:20:27.151962+0000","last_fullsized":"2024-04-25T00:20:27.151962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041832+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:20:27.041832+0000","last_peered":"2024-04-25T00:20:27.041832+0000","last_clean":"2024-04-25T00:20:27.041832+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:20:27.041832+0000","last_undegraded":"2024-04-25T00:20:27.041832+0000","last_fullsized":"2024-04-25T00:20:27.041832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152864+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:20:27.152864+0000","last_peered":"2024-04-25T00:20:27.152864+0000","last_clean":"2024-04-25T00:20:27.152864+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:20:27.152864+0000","last_undegraded":"2024-04-25T00:20:27.152864+0000","last_fullsized":"2024-04-25T00:20:27.152864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040722+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:20:27.040722+0000","last_peered":"2024-04-25T00:20:27.040722+0000","last_clean":"2024-04-25T00:20:27.040722+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:20:27.040722+0000","last_undegraded":"2024-04-25T00:20:27.040722+0000","last_fullsized":"2024-04-25T00:20:27.040722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:09.939517+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:21:09.939517+0000","last_peered":"2024-04-25T00:21:09.939517+0000","last_clean":"2024-04-25T00:21:09.939517+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:21:09.939517+0000","last_undegraded":"2024-04-25T00:21:09.939517+0000","last_fullsized":"2024-04-25T00:21:09.939517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041617+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:20:27.041617+0000","last_peered":"2024-04-25T00:20:27.041617+0000","last_clean":"2024-04-25T00:20:27.041617+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:20:27.041617+0000","last_undegraded":"2024-04-25T00:20:27.041617+0000","last_fullsized":"2024-04-25T00:20:27.041617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041203+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:20:27.041203+0000","last_peered":"2024-04-25T00:20:27.041203+0000","last_clean":"2024-04-25T00:20:27.041203+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:20:27.041203+0000","last_undegraded":"2024-04-25T00:20:27.041203+0000","last_fullsized":"2024-04-25T00:20:27.041203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.054883+0000","last_change":"2024-04-25T00:20:27.054883+0000","last_active":"2024-04-25T00:20:27.054883+0000","last_peered":"2024-04-25T00:20:27.054883+0000","last_clean":"2024-04-25T00:20:27.054883+0000","last_became_active":"2024-04-25T00:20:27.049595+0000","last_became_peered":"2024-04-25T00:20:27.049595+0000","last_unstale":"2024-04-25T00:20:27.054883+0000","last_undegraded":"2024-04-25T00:20:27.054883+0000","last_fullsized":"2024-04-25T00:20:27.054883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:05.358193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152019+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:20:27.152019+0000","last_peered":"2024-04-25T00:20:27.152019+0000","last_clean":"2024-04-25T00:20:27.152019+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:20:27.152019+0000","last_undegraded":"2024-04-25T00:20:27.152019+0000","last_fullsized":"2024-04-25T00:20:27.152019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044148+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:20:27.044148+0000","last_peered":"2024-04-25T00:20:27.044148+0000","last_clean":"2024-04-25T00:20:27.044148+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:20:27.044148+0000","last_undegraded":"2024-04-25T00:20:27.044148+0000","last_fullsized":"2024-04-25T00:20:27.044148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048330+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:20:27.048330+0000","last_peered":"2024-04-25T00:20:27.048330+0000","last_clean":"2024-04-25T00:20:27.048330+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:20:27.048330+0000","last_undegraded":"2024-04-25T00:20:27.048330+0000","last_fullsized":"2024-04-25T00:20:27.048330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135705+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:20:27.135705+0000","last_peered":"2024-04-25T00:20:27.135705+0000","last_clean":"2024-04-25T00:20:27.135705+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:20:27.135705+0000","last_undegraded":"2024-04-25T00:20:27.135705+0000","last_fullsized":"2024-04-25T00:20:27.135705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151520+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:20:27.151520+0000","last_peered":"2024-04-25T00:20:27.151520+0000","last_clean":"2024-04-25T00:20:27.151520+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:20:27.151520+0000","last_undegraded":"2024-04-25T00:20:27.151520+0000","last_fullsized":"2024-04-25T00:20:27.151520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151351+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:20:27.151351+0000","last_peered":"2024-04-25T00:20:27.151351+0000","last_clean":"2024-04-25T00:20:27.151351+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:20:27.151351+0000","last_undegraded":"2024-04-25T00:20:27.151351+0000","last_fullsized":"2024-04-25T00:20:27.151351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039075+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:20:27.039075+0000","last_peered":"2024-04-25T00:20:27.039075+0000","last_clean":"2024-04-25T00:20:27.039075+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:20:27.039075+0000","last_undegraded":"2024-04-25T00:20:27.039075+0000","last_fullsized":"2024-04-25T00:20:27.039075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136131+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:20:27.136131+0000","last_peered":"2024-04-25T00:20:27.136131+0000","last_clean":"2024-04-25T00:20:27.136131+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:20:27.136131+0000","last_undegraded":"2024-04-25T00:20:27.136131+0000","last_fullsized":"2024-04-25T00:20:27.136131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.937051+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:21:08.937051+0000","last_peered":"2024-04-25T00:21:08.937051+0000","last_clean":"2024-04-25T00:21:08.937051+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:21:08.937051+0000","last_undegraded":"2024-04-25T00:21:08.937051+0000","last_fullsized":"2024-04-25T00:21:08.937051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043211+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:20:27.043211+0000","last_peered":"2024-04-25T00:20:27.043211+0000","last_clean":"2024-04-25T00:20:27.043211+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:20:27.043211+0000","last_undegraded":"2024-04-25T00:20:27.043211+0000","last_fullsized":"2024-04-25T00:20:27.043211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.038805+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:20:27.038805+0000","last_peered":"2024-04-25T00:20:27.038805+0000","last_clean":"2024-04-25T00:20:27.038805+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:20:27.038805+0000","last_undegraded":"2024-04-25T00:20:27.038805+0000","last_fullsized":"2024-04-25T00:20:27.038805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.176907+0000","last_change":"2024-04-25T00:20:27.176907+0000","last_active":"2024-04-25T00:20:27.176907+0000","last_peered":"2024-04-25T00:20:27.176907+0000","last_clean":"2024-04-25T00:20:27.176907+0000","last_became_active":"2024-04-25T00:20:27.153755+0000","last_became_peered":"2024-04-25T00:20:27.153755+0000","last_unstale":"2024-04-25T00:20:27.176907+0000","last_undegraded":"2024-04-25T00:20:27.176907+0000","last_fullsized":"2024-04-25T00:20:27.176907+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:07.142383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151718+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:20:27.151718+0000","last_peered":"2024-04-25T00:20:27.151718+0000","last_clean":"2024-04-25T00:20:27.151718+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:20:27.151718+0000","last_undegraded":"2024-04-25T00:20:27.151718+0000","last_fullsized":"2024-04-25T00:20:27.151718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152321+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:20:27.152321+0000","last_peered":"2024-04-25T00:20:27.152321+0000","last_clean":"2024-04-25T00:20:27.152321+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:20:27.152321+0000","last_undegraded":"2024-04-25T00:20:27.152321+0000","last_fullsized":"2024-04-25T00:20:27.152321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152741+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:20:27.152741+0000","last_peered":"2024-04-25T00:20:27.152741+0000","last_clean":"2024-04-25T00:20:27.152741+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:20:27.152741+0000","last_undegraded":"2024-04-25T00:20:27.152741+0000","last_fullsized":"2024-04-25T00:20:27.152741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043874+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:20:27.043874+0000","last_peered":"2024-04-25T00:20:27.043874+0000","last_clean":"2024-04-25T00:20:27.043874+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:20:27.043874+0000","last_undegraded":"2024-04-25T00:20:27.043874+0000","last_fullsized":"2024-04-25T00:20:27.043874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.038902+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:20:27.038902+0000","last_peered":"2024-04-25T00:20:27.038902+0000","last_clean":"2024-04-25T00:20:27.038902+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:20:27.038902+0000","last_undegraded":"2024-04-25T00:20:27.038902+0000","last_fullsized":"2024-04-25T00:20:27.038902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":202,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.937052+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:21:08.937052+0000","last_peered":"2024-04-25T00:21:08.937052+0000","last_clean":"2024-04-25T00:21:08.937052+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:21:08.937052+0000","last_undegraded":"2024-04-25T00:21:08.937052+0000","last_fullsized":"2024-04-25T00:21:08.937052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60642,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":130,"num_write":35,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042381+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:20:27.042381+0000","last_peered":"2024-04-25T00:20:27.042381+0000","last_clean":"2024-04-25T00:20:27.042381+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:20:27.042381+0000","last_undegraded":"2024-04-25T00:20:27.042381+0000","last_fullsized":"2024-04-25T00:20:27.042381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153230+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:20:27.153230+0000","last_peered":"2024-04-25T00:20:27.153230+0000","last_clean":"2024-04-25T00:20:27.153230+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:20:27.153230+0000","last_undegraded":"2024-04-25T00:20:27.153230+0000","last_fullsized":"2024-04-25T00:20:27.153230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041552+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:20:27.041552+0000","last_peered":"2024-04-25T00:20:27.041552+0000","last_clean":"2024-04-25T00:20:27.041552+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:20:27.041552+0000","last_undegraded":"2024-04-25T00:20:27.041552+0000","last_fullsized":"2024-04-25T00:20:27.041552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048551+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:20:27.048551+0000","last_peered":"2024-04-25T00:20:27.048551+0000","last_clean":"2024-04-25T00:20:27.048551+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:20:27.048551+0000","last_undegraded":"2024-04-25T00:20:27.048551+0000","last_fullsized":"2024-04-25T00:20:27.048551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044108+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:20:27.044108+0000","last_peered":"2024-04-25T00:20:27.044108+0000","last_clean":"2024-04-25T00:20:27.044108+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:20:27.044108+0000","last_undegraded":"2024-04-25T00:20:27.044108+0000","last_fullsized":"2024-04-25T00:20:27.044108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151820+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:20:27.151820+0000","last_peered":"2024-04-25T00:20:27.151820+0000","last_clean":"2024-04-25T00:20:27.151820+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:20:27.151820+0000","last_undegraded":"2024-04-25T00:20:27.151820+0000","last_fullsized":"2024-04-25T00:20:27.151820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043367+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:20:27.043367+0000","last_peered":"2024-04-25T00:20:27.043367+0000","last_clean":"2024-04-25T00:20:27.043367+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:20:27.043367+0000","last_undegraded":"2024-04-25T00:20:27.043367+0000","last_fullsized":"2024-04-25T00:20:27.043367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153934+0000","last_change":"2024-04-25T00:20:27.153934+0000","last_active":"2024-04-25T00:20:27.153934+0000","last_peered":"2024-04-25T00:20:27.153934+0000","last_clean":"2024-04-25T00:20:27.153934+0000","last_became_active":"2024-04-25T00:20:27.152647+0000","last_became_peered":"2024-04-25T00:20:27.152647+0000","last_unstale":"2024-04-25T00:20:27.153934+0000","last_undegraded":"2024-04-25T00:20:27.153934+0000","last_fullsized":"2024-04-25T00:20:27.153934+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.389722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.943761+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:21:08.943761+0000","last_peered":"2024-04-25T00:21:08.943761+0000","last_clean":"2024-04-25T00:21:08.943761+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:21:08.943761+0000","last_undegraded":"2024-04-25T00:21:08.943761+0000","last_fullsized":"2024-04-25T00:21:08.943761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042198+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:20:27.042198+0000","last_peered":"2024-04-25T00:20:27.042198+0000","last_clean":"2024-04-25T00:20:27.042198+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:20:27.042198+0000","last_undegraded":"2024-04-25T00:20:27.042198+0000","last_fullsized":"2024-04-25T00:20:27.042198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048902+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:20:27.048902+0000","last_peered":"2024-04-25T00:20:27.048902+0000","last_clean":"2024-04-25T00:20:27.048902+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:20:27.048902+0000","last_undegraded":"2024-04-25T00:20:27.048902+0000","last_fullsized":"2024-04-25T00:20:27.048902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041326+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:20:27.041326+0000","last_peered":"2024-04-25T00:20:27.041326+0000","last_clean":"2024-04-25T00:20:27.041326+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:20:27.041326+0000","last_undegraded":"2024-04-25T00:20:27.041326+0000","last_fullsized":"2024-04-25T00:20:27.041326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151539+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:20:27.151539+0000","last_peered":"2024-04-25T00:20:27.151539+0000","last_clean":"2024-04-25T00:20:27.151539+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:20:27.151539+0000","last_undegraded":"2024-04-25T00:20:27.151539+0000","last_fullsized":"2024-04-25T00:20:27.151539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151405+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:20:27.151405+0000","last_peered":"2024-04-25T00:20:27.151405+0000","last_clean":"2024-04-25T00:20:27.151405+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:20:27.151405+0000","last_undegraded":"2024-04-25T00:20:27.151405+0000","last_fullsized":"2024-04-25T00:20:27.151405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043123+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:20:27.043123+0000","last_peered":"2024-04-25T00:20:27.043123+0000","last_clean":"2024-04-25T00:20:27.043123+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:20:27.043123+0000","last_undegraded":"2024-04-25T00:20:27.043123+0000","last_fullsized":"2024-04-25T00:20:27.043123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044585+0000","last_change":"2024-04-25T00:19:33.880006+0000","last_active":"2024-04-25T00:20:27.044585+0000","last_peered":"2024-04-25T00:20:27.044585+0000","last_clean":"2024-04-25T00:20:27.044585+0000","last_became_active":"2024-04-25T00:19:33.879054+0000","last_became_peered":"2024-04-25T00:19:33.879054+0000","last_unstale":"2024-04-25T00:20:27.044585+0000","last_undegraded":"2024-04-25T00:20:27.044585+0000","last_fullsized":"2024-04-25T00:20:27.044585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:43.232276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041495+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:20:27.041495+0000","last_peered":"2024-04-25T00:20:27.041495+0000","last_clean":"2024-04-25T00:20:27.041495+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:20:27.041495+0000","last_undegraded":"2024-04-25T00:20:27.041495+0000","last_fullsized":"2024-04-25T00:20:27.041495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039212+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:20:27.039212+0000","last_peered":"2024-04-25T00:20:27.039212+0000","last_clean":"2024-04-25T00:20:27.039212+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:20:27.039212+0000","last_undegraded":"2024-04-25T00:20:27.039212+0000","last_fullsized":"2024-04-25T00:20:27.039212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046537+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:20:27.046537+0000","last_peered":"2024-04-25T00:20:27.046537+0000","last_clean":"2024-04-25T00:20:27.046537+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:20:27.046537+0000","last_undegraded":"2024-04-25T00:20:27.046537+0000","last_fullsized":"2024-04-25T00:20:27.046537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048652+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:20:27.048652+0000","last_peered":"2024-04-25T00:20:27.048652+0000","last_clean":"2024-04-25T00:20:27.048652+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:20:27.048652+0000","last_undegraded":"2024-04-25T00:20:27.048652+0000","last_fullsized":"2024-04-25T00:20:27.048652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:09.115013+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:21:09.115013+0000","last_peered":"2024-04-25T00:21:09.115013+0000","last_clean":"2024-04-25T00:21:09.115013+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:21:09.115013+0000","last_undegraded":"2024-04-25T00:21:09.115013+0000","last_fullsized":"2024-04-25T00:21:09.115013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046853+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:20:27.046853+0000","last_peered":"2024-04-25T00:20:27.046853+0000","last_clean":"2024-04-25T00:20:27.046853+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:20:27.046853+0000","last_undegraded":"2024-04-25T00:20:27.046853+0000","last_fullsized":"2024-04-25T00:20:27.046853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048657+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:20:27.048657+0000","last_peered":"2024-04-25T00:20:27.048657+0000","last_clean":"2024-04-25T00:20:27.048657+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:20:27.048657+0000","last_undegraded":"2024-04-25T00:20:27.048657+0000","last_fullsized":"2024-04-25T00:20:27.048657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151515+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:20:27.151515+0000","last_peered":"2024-04-25T00:20:27.151515+0000","last_clean":"2024-04-25T00:20:27.151515+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:20:27.151515+0000","last_undegraded":"2024-04-25T00:20:27.151515+0000","last_fullsized":"2024-04-25T00:20:27.151515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.943047+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:21:08.943047+0000","last_peered":"2024-04-25T00:21:08.943047+0000","last_clean":"2024-04-25T00:21:08.943047+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:21:08.943047+0000","last_undegraded":"2024-04-25T00:21:08.943047+0000","last_fullsized":"2024-04-25T00:21:08.943047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043698+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:20:27.043698+0000","last_peered":"2024-04-25T00:20:27.043698+0000","last_clean":"2024-04-25T00:20:27.043698+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:20:27.043698+0000","last_undegraded":"2024-04-25T00:20:27.043698+0000","last_fullsized":"2024-04-25T00:20:27.043698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135744+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:20:27.135744+0000","last_peered":"2024-04-25T00:20:27.135744+0000","last_clean":"2024-04-25T00:20:27.135744+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:20:27.135744+0000","last_undegraded":"2024-04-25T00:20:27.135744+0000","last_fullsized":"2024-04-25T00:20:27.135744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152165+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:20:27.152165+0000","last_peered":"2024-04-25T00:20:27.152165+0000","last_clean":"2024-04-25T00:20:27.152165+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:20:27.152165+0000","last_undegraded":"2024-04-25T00:20:27.152165+0000","last_fullsized":"2024-04-25T00:20:27.152165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042393+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:20:27.042393+0000","last_peered":"2024-04-25T00:20:27.042393+0000","last_clean":"2024-04-25T00:20:27.042393+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:20:27.042393+0000","last_undegraded":"2024-04-25T00:20:27.042393+0000","last_fullsized":"2024-04-25T00:20:27.042393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.049784+0000","last_change":"2024-04-25T00:20:27.049784+0000","last_active":"2024-04-25T00:20:27.049784+0000","last_peered":"2024-04-25T00:20:27.049784+0000","last_clean":"2024-04-25T00:20:27.049784+0000","last_became_active":"2024-04-25T00:20:27.049413+0000","last_became_peered":"2024-04-25T00:20:27.049413+0000","last_unstale":"2024-04-25T00:20:27.049784+0000","last_undegraded":"2024-04-25T00:20:27.049784+0000","last_fullsized":"2024-04-25T00:20:27.049784+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:26.533987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048161+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:20:27.048161+0000","last_peered":"2024-04-25T00:20:27.048161+0000","last_clean":"2024-04-25T00:20:27.048161+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:20:27.048161+0000","last_undegraded":"2024-04-25T00:20:27.048161+0000","last_fullsized":"2024-04-25T00:20:27.048161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042081+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:20:27.042081+0000","last_peered":"2024-04-25T00:20:27.042081+0000","last_clean":"2024-04-25T00:20:27.042081+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:20:27.042081+0000","last_undegraded":"2024-04-25T00:20:27.042081+0000","last_fullsized":"2024-04-25T00:20:27.042081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044368+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:20:27.044368+0000","last_peered":"2024-04-25T00:20:27.044368+0000","last_clean":"2024-04-25T00:20:27.044368+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:20:27.044368+0000","last_undegraded":"2024-04-25T00:20:27.044368+0000","last_fullsized":"2024-04-25T00:20:27.044368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042424+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:20:27.042424+0000","last_peered":"2024-04-25T00:20:27.042424+0000","last_clean":"2024-04-25T00:20:27.042424+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:20:27.042424+0000","last_undegraded":"2024-04-25T00:20:27.042424+0000","last_fullsized":"2024-04-25T00:20:27.042424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048828+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:20:27.048828+0000","last_peered":"2024-04-25T00:20:27.048828+0000","last_clean":"2024-04-25T00:20:27.048828+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:20:27.048828+0000","last_undegraded":"2024-04-25T00:20:27.048828+0000","last_fullsized":"2024-04-25T00:20:27.048828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135926+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:20:27.135926+0000","last_peered":"2024-04-25T00:20:27.135926+0000","last_clean":"2024-04-25T00:20:27.135926+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:20:27.135926+0000","last_undegraded":"2024-04-25T00:20:27.135926+0000","last_fullsized":"2024-04-25T00:20:27.135926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039561+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:20:27.039561+0000","last_peered":"2024-04-25T00:20:27.039561+0000","last_clean":"2024-04-25T00:20:27.039561+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:20:27.039561+0000","last_undegraded":"2024-04-25T00:20:27.039561+0000","last_fullsized":"2024-04-25T00:20:27.039561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136253+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:20:27.136253+0000","last_peered":"2024-04-25T00:20:27.136253+0000","last_clean":"2024-04-25T00:20:27.136253+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:20:27.136253+0000","last_undegraded":"2024-04-25T00:20:27.136253+0000","last_fullsized":"2024-04-25T00:20:27.136253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039980+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:20:27.039980+0000","last_peered":"2024-04-25T00:20:27.039980+0000","last_clean":"2024-04-25T00:20:27.039980+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:20:27.039980+0000","last_undegraded":"2024-04-25T00:20:27.039980+0000","last_fullsized":"2024-04-25T00:20:27.039980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152851+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:20:27.152851+0000","last_peered":"2024-04-25T00:20:27.152851+0000","last_clean":"2024-04-25T00:20:27.152851+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:20:27.152851+0000","last_undegraded":"2024-04-25T00:20:27.152851+0000","last_fullsized":"2024-04-25T00:20:27.152851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151694+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:20:27.151694+0000","last_peered":"2024-04-25T00:20:27.151694+0000","last_clean":"2024-04-25T00:20:27.151694+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:20:27.151694+0000","last_undegraded":"2024-04-25T00:20:27.151694+0000","last_fullsized":"2024-04-25T00:20:27.151694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152702+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:20:27.152702+0000","last_peered":"2024-04-25T00:20:27.152702+0000","last_clean":"2024-04-25T00:20:27.152702+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:20:27.152702+0000","last_undegraded":"2024-04-25T00:20:27.152702+0000","last_fullsized":"2024-04-25T00:20:27.152702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041730+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:20:27.041730+0000","last_peered":"2024-04-25T00:20:27.041730+0000","last_clean":"2024-04-25T00:20:27.041730+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:20:27.041730+0000","last_undegraded":"2024-04-25T00:20:27.041730+0000","last_fullsized":"2024-04-25T00:20:27.041730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041358+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:20:27.041358+0000","last_peered":"2024-04-25T00:20:27.041358+0000","last_clean":"2024-04-25T00:20:27.041358+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:20:27.041358+0000","last_undegraded":"2024-04-25T00:20:27.041358+0000","last_fullsized":"2024-04-25T00:20:27.041358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153703+0000","last_change":"2024-04-25T00:20:27.153703+0000","last_active":"2024-04-25T00:20:27.153703+0000","last_peered":"2024-04-25T00:20:27.153703+0000","last_clean":"2024-04-25T00:20:27.153703+0000","last_became_active":"2024-04-25T00:20:27.152715+0000","last_became_peered":"2024-04-25T00:20:27.152715+0000","last_unstale":"2024-04-25T00:20:27.153703+0000","last_undegraded":"2024-04-25T00:20:27.153703+0000","last_fullsized":"2024-04-25T00:20:27.153703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:09.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135777+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:20:27.135777+0000","last_peered":"2024-04-25T00:20:27.135777+0000","last_clean":"2024-04-25T00:20:27.135777+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:20:27.135777+0000","last_undegraded":"2024-04-25T00:20:27.135777+0000","last_fullsized":"2024-04-25T00:20:27.135777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043801+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:20:27.043801+0000","last_peered":"2024-04-25T00:20:27.043801+0000","last_clean":"2024-04-25T00:20:27.043801+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:20:27.043801+0000","last_undegraded":"2024-04-25T00:20:27.043801+0000","last_fullsized":"2024-04-25T00:20:27.043801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152609+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:20:27.152609+0000","last_peered":"2024-04-25T00:20:27.152609+0000","last_clean":"2024-04-25T00:20:27.152609+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:20:27.152609+0000","last_undegraded":"2024-04-25T00:20:27.152609+0000","last_fullsized":"2024-04-25T00:20:27.152609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040231+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:20:27.040231+0000","last_peered":"2024-04-25T00:20:27.040231+0000","last_clean":"2024-04-25T00:20:27.040231+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:20:27.040231+0000","last_undegraded":"2024-04-25T00:20:27.040231+0000","last_fullsized":"2024-04-25T00:20:27.040231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136038+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:20:27.136038+0000","last_peered":"2024-04-25T00:20:27.136038+0000","last_clean":"2024-04-25T00:20:27.136038+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:20:27.136038+0000","last_undegraded":"2024-04-25T00:20:27.136038+0000","last_fullsized":"2024-04-25T00:20:27.136038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152710+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:20:27.152710+0000","last_peered":"2024-04-25T00:20:27.152710+0000","last_clean":"2024-04-25T00:20:27.152710+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:20:27.152710+0000","last_undegraded":"2024-04-25T00:20:27.152710+0000","last_fullsized":"2024-04-25T00:20:27.152710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041613+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:20:27.041613+0000","last_peered":"2024-04-25T00:20:27.041613+0000","last_clean":"2024-04-25T00:20:27.041613+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:20:27.041613+0000","last_undegraded":"2024-04-25T00:20:27.041613+0000","last_fullsized":"2024-04-25T00:20:27.041613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042946+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:20:27.042946+0000","last_peered":"2024-04-25T00:20:27.042946+0000","last_clean":"2024-04-25T00:20:27.042946+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:20:27.042946+0000","last_undegraded":"2024-04-25T00:20:27.042946+0000","last_fullsized":"2024-04-25T00:20:27.042946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041906+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:20:27.041906+0000","last_peered":"2024-04-25T00:20:27.041906+0000","last_clean":"2024-04-25T00:20:27.041906+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:20:27.041906+0000","last_undegraded":"2024-04-25T00:20:27.041906+0000","last_fullsized":"2024-04-25T00:20:27.041906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:04.931907+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:21:04.931907+0000","last_peered":"2024-04-25T00:21:04.931907+0000","last_clean":"2024-04-25T00:21:04.931907+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:21:04.931907+0000","last_undegraded":"2024-04-25T00:21:04.931907+0000","last_fullsized":"2024-04-25T00:21:04.931907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044141+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:20:27.044141+0000","last_peered":"2024-04-25T00:20:27.044141+0000","last_clean":"2024-04-25T00:20:27.044141+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:20:27.044141+0000","last_undegraded":"2024-04-25T00:20:27.044141+0000","last_fullsized":"2024-04-25T00:20:27.044141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151240+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:20:27.151240+0000","last_peered":"2024-04-25T00:20:27.151240+0000","last_clean":"2024-04-25T00:20:27.151240+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:20:27.151240+0000","last_undegraded":"2024-04-25T00:20:27.151240+0000","last_fullsized":"2024-04-25T00:20:27.151240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152987+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:20:27.152987+0000","last_peered":"2024-04-25T00:20:27.152987+0000","last_clean":"2024-04-25T00:20:27.152987+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:20:27.152987+0000","last_undegraded":"2024-04-25T00:20:27.152987+0000","last_fullsized":"2024-04-25T00:20:27.152987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044296+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:20:27.044296+0000","last_peered":"2024-04-25T00:20:27.044296+0000","last_clean":"2024-04-25T00:20:27.044296+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:20:27.044296+0000","last_undegraded":"2024-04-25T00:20:27.044296+0000","last_fullsized":"2024-04-25T00:20:27.044296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151171+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:20:27.151171+0000","last_peered":"2024-04-25T00:20:27.151171+0000","last_clean":"2024-04-25T00:20:27.151171+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:20:27.151171+0000","last_undegraded":"2024-04-25T00:20:27.151171+0000","last_fullsized":"2024-04-25T00:20:27.151171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048770+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:20:27.048770+0000","last_peered":"2024-04-25T00:20:27.048770+0000","last_clean":"2024-04-25T00:20:27.048770+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:20:27.048770+0000","last_undegraded":"2024-04-25T00:20:27.048770+0000","last_fullsized":"2024-04-25T00:20:27.048770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044188+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:20:27.044188+0000","last_peered":"2024-04-25T00:20:27.044188+0000","last_clean":"2024-04-25T00:20:27.044188+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:20:27.044188+0000","last_undegraded":"2024-04-25T00:20:27.044188+0000","last_fullsized":"2024-04-25T00:20:27.044188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043927+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:20:27.043927+0000","last_peered":"2024-04-25T00:20:27.043927+0000","last_clean":"2024-04-25T00:20:27.043927+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:20:27.043927+0000","last_undegraded":"2024-04-25T00:20:27.043927+0000","last_fullsized":"2024-04-25T00:20:27.043927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040850+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:20:27.040850+0000","last_peered":"2024-04-25T00:20:27.040850+0000","last_clean":"2024-04-25T00:20:27.040850+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:20:27.040850+0000","last_undegraded":"2024-04-25T00:20:27.040850+0000","last_fullsized":"2024-04-25T00:20:27.040850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040127+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:20:27.040127+0000","last_peered":"2024-04-25T00:20:27.040127+0000","last_clean":"2024-04-25T00:20:27.040127+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:20:27.040127+0000","last_undegraded":"2024-04-25T00:20:27.040127+0000","last_fullsized":"2024-04-25T00:20:27.040127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151631+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:20:27.151631+0000","last_peered":"2024-04-25T00:20:27.151631+0000","last_clean":"2024-04-25T00:20:27.151631+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:20:27.151631+0000","last_undegraded":"2024-04-25T00:20:27.151631+0000","last_fullsized":"2024-04-25T00:20:27.151631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048931+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:20:27.048931+0000","last_peered":"2024-04-25T00:20:27.048931+0000","last_clean":"2024-04-25T00:20:27.048931+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:20:27.048931+0000","last_undegraded":"2024-04-25T00:20:27.048931+0000","last_fullsized":"2024-04-25T00:20:27.048931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.045607+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:20:27.045607+0000","last_peered":"2024-04-25T00:20:27.045607+0000","last_clean":"2024-04-25T00:20:27.045607+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:20:27.045607+0000","last_undegraded":"2024-04-25T00:20:27.045607+0000","last_fullsized":"2024-04-25T00:20:27.045607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040794+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:20:27.040794+0000","last_peered":"2024-04-25T00:20:27.040794+0000","last_clean":"2024-04-25T00:20:27.040794+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:20:27.040794+0000","last_undegraded":"2024-04-25T00:20:27.040794+0000","last_fullsized":"2024-04-25T00:20:27.040794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136323+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:20:27.136323+0000","last_peered":"2024-04-25T00:20:27.136323+0000","last_clean":"2024-04-25T00:20:27.136323+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:20:27.136323+0000","last_undegraded":"2024-04-25T00:20:27.136323+0000","last_fullsized":"2024-04-25T00:20:27.136323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151694+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:20:27.151694+0000","last_peered":"2024-04-25T00:20:27.151694+0000","last_clean":"2024-04-25T00:20:27.151694+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:20:27.151694+0000","last_undegraded":"2024-04-25T00:20:27.151694+0000","last_fullsized":"2024-04-25T00:20:27.151694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136015+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:20:27.136015+0000","last_peered":"2024-04-25T00:20:27.136015+0000","last_clean":"2024-04-25T00:20:27.136015+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:20:27.136015+0000","last_undegraded":"2024-04-25T00:20:27.136015+0000","last_fullsized":"2024-04-25T00:20:27.136015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042377+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:20:27.042377+0000","last_peered":"2024-04-25T00:20:27.042377+0000","last_clean":"2024-04-25T00:20:27.042377+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:20:27.042377+0000","last_undegraded":"2024-04-25T00:20:27.042377+0000","last_fullsized":"2024-04-25T00:20:27.042377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048541+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:20:27.048541+0000","last_peered":"2024-04-25T00:20:27.048541+0000","last_clean":"2024-04-25T00:20:27.048541+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:20:27.048541+0000","last_undegraded":"2024-04-25T00:20:27.048541+0000","last_fullsized":"2024-04-25T00:20:27.048541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046816+0000","last_change":"2024-04-25T00:20:27.046816+0000","last_active":"2024-04-25T00:20:27.046816+0000","last_peered":"2024-04-25T00:20:27.046816+0000","last_clean":"2024-04-25T00:20:27.046816+0000","last_became_active":"2024-04-25T00:20:27.045845+0000","last_became_peered":"2024-04-25T00:20:27.045845+0000","last_unstale":"2024-04-25T00:20:27.046816+0000","last_undegraded":"2024-04-25T00:20:27.046816+0000","last_fullsized":"2024-04-25T00:20:27.046816+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:45.465308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284395,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67846,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":710,"num_read_kb":425,"num_write":238,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":502034,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":1149211,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":505388,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":501670,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1097886,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":496796,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":558468,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":498252,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":559079,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496504,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":501261,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":1093749,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:21:12.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:21:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:13.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:12 smithi045 ceph-mon[29194]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:13.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:12 smithi045 ceph-mon[29194]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:12 smithi122 ceph-mon[32959]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:12 smithi122 ceph-mon[32959]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:12 smithi071 ceph-mon[32667]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:12 smithi071 ceph-mon[32667]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:14.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 -- 172.21.15.45:0/1906600438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901082b0 msgr2=0x7fe790108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 --2- 172.21.15.45:0/1906600438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901082b0 0x7fe790108690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009990 tx=0x7fe78c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 -- 172.21.15.45:0/1906600438 shutdown_connections 2024-04-25T00:21:14.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 --2- 172.21.15.45:0/1906600438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79006a8e0 0x7fe79010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 --2- 172.21.15.45:0/1906600438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901082b0 0x7fe790108690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 --2- 172.21.15.45:0/1906600438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe790069f40 0x7fe79006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 -- 172.21.15.45:0/1906600438 >> 172.21.15.45:0/1906600438 conn(0x7fe7900780e0 msgr2=0x7fe7900784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 -- 172.21.15.45:0/1906600438 shutdown_connections 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.230+0000 7fe7972be700 1 -- 172.21.15.45:0/1906600438 wait complete. 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 Processor -- start 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 -- start start 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790069f40 0x7fe7901a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7901082b0 0x7fe79019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe790112510 con 0x7fe79006a8e0 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe790112390 con 0x7fe7901082b0 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe7972be700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe790112690 con 0x7fe790069f40 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42754/0 (socket says 172.21.15.45:42754) 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 -- 172.21.15.45:0/1543073395 learned_addr learned my addr 172.21.15.45:0/1543073395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe79585b700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7901082b0 0x7fe79019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe79505a700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790069f40 0x7fe7901a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 -- 172.21.15.45:0/1543073395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790069f40 msgr2=0x7fe7901a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790069f40 0x7fe7901a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 -- 172.21.15.45:0/1543073395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7901082b0 msgr2=0x7fe79019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7901082b0 0x7fe79019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.231+0000 7fe794859700 1 -- 172.21.15.45:0/1543073395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79019b150 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe794859700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe78000d950 tx=0x7fe78000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780016070 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe780009e20 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79019b440 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78000f620 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.232+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a7700 con 0x7fe79006a8e0 2024-04-25T00:21:14.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.233+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790068f20 con 0x7fe79006a8e0 2024-04-25T00:21:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.234+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe780019070 con 0x7fe79006a8e0 2024-04-25T00:21:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.235+0000 7fe7867fc700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 0x7fe77c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.235+0000 7fe79505a700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 0x7fe77c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.235+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe7800cf0e0 con 0x7fe79006a8e0 2024-04-25T00:21:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.235+0000 7fe79505a700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 0x7fe77c0a5170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.241+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe78009bba0 con 0x7fe79006a8e0 2024-04-25T00:21:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.443+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe79010db50 con 0x7fe77c0a2cb0 2024-04-25T00:21:14.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.453+0000 7fe7867fc700 1 -- 172.21.15.45:0/1543073395 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340691 (secure 0 0 0) 0x7fe79010db50 con 0x7fe77c0a2cb0 2024-04-25T00:21:14.458 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 msgr2=0x7fe77c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 0x7fe77c0a5170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c00b540 comp rx=0 tx=0).stop 2024-04-25T00:21:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 msgr2=0x7fe7901a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe78000d950 tx=0x7fe78000dc60 comp rx=0 tx=0).stop 2024-04-25T00:21:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 shutdown_connections 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe790069f40 0x7fe7901a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe77c0a2cb0 0x7fe77c0a5170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7901082b0 0x7fe79019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 --2- 172.21.15.45:0/1543073395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe79006a8e0 0x7fe7901a0d70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 >> 172.21.15.45:0/1543073395 conn(0x7fe7900780e0 msgr2=0x7fe79010d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 shutdown_connections 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:14.456+0000 7fe7972be700 1 -- 172.21.15.45:0/1543073395 wait complete. 2024-04-25T00:21:14.461 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:21:15.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:14 smithi045 ceph-mon[29194]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T00:21:13.287869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658448,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":851,"num_read_kb":544,"num_write":421,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386480,"kb_used_data":12296,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555344,"statfs":{"total":1151940427776,"available":1151544672256,"internally_reserved":0,"allocated":12591104,"data_stored":7960298,"data_compressed":355488,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003704"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044572+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:20:27.044572+0000","last_peered":"2024-04-25T00:20:27.044572+0000","last_clean":"2024-04-25T00:20:27.044572+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:20:27.044572+0000","last_undegraded":"2024-04-25T00:20:27.044572+0000","last_fullsized":"2024-04-25T00:20:27.044572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151575+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:20:27.151575+0000","last_peered":"2024-04-25T00:20:27.151575+0000","last_clean":"2024-04-25T00:20:27.151575+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:20:27.151575+0000","last_undegraded":"2024-04-25T00:20:27.151575+0000","last_fullsized":"2024-04-25T00:20:27.151575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151455+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:20:27.151455+0000","last_peered":"2024-04-25T00:20:27.151455+0000","last_clean":"2024-04-25T00:20:27.151455+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:20:27.151455+0000","last_undegraded":"2024-04-25T00:20:27.151455+0000","last_fullsized":"2024-04-25T00:20:27.151455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153002+0000","last_change":"2024-04-25T00:19:33.880302+0000","last_active":"2024-04-25T00:20:27.153002+0000","last_peered":"2024-04-25T00:20:27.153002+0000","last_clean":"2024-04-25T00:20:27.153002+0000","last_became_active":"2024-04-25T00:19:33.879849+0000","last_became_peered":"2024-04-25T00:19:33.879849+0000","last_unstale":"2024-04-25T00:20:27.153002+0000","last_undegraded":"2024-04-25T00:20:27.153002+0000","last_fullsized":"2024-04-25T00:20:27.153002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:35:32.449680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041494+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:20:27.041494+0000","last_peered":"2024-04-25T00:20:27.041494+0000","last_clean":"2024-04-25T00:20:27.041494+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:20:27.041494+0000","last_undegraded":"2024-04-25T00:20:27.041494+0000","last_fullsized":"2024-04-25T00:20:27.041494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046385+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:20:27.046385+0000","last_peered":"2024-04-25T00:20:27.046385+0000","last_clean":"2024-04-25T00:20:27.046385+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:20:27.046385+0000","last_undegraded":"2024-04-25T00:20:27.046385+0000","last_fullsized":"2024-04-25T00:20:27.046385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042227+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:20:27.042227+0000","last_peered":"2024-04-25T00:20:27.042227+0000","last_clean":"2024-04-25T00:20:27.042227+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:20:27.042227+0000","last_undegraded":"2024-04-25T00:20:27.042227+0000","last_fullsized":"2024-04-25T00:20:27.042227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135822+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:20:27.135822+0000","last_peered":"2024-04-25T00:20:27.135822+0000","last_clean":"2024-04-25T00:20:27.135822+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:20:27.135822+0000","last_undegraded":"2024-04-25T00:20:27.135822+0000","last_fullsized":"2024-04-25T00:20:27.135822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043011+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:20:27.043011+0000","last_peered":"2024-04-25T00:20:27.043011+0000","last_clean":"2024-04-25T00:20:27.043011+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:20:27.043011+0000","last_undegraded":"2024-04-25T00:20:27.043011+0000","last_fullsized":"2024-04-25T00:20:27.043011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.047266+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:20:27.047266+0000","last_peered":"2024-04-25T00:20:27.047266+0000","last_clean":"2024-04-25T00:20:27.047266+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:20:27.047266+0000","last_undegraded":"2024-04-25T00:20:27.047266+0000","last_fullsized":"2024-04-25T00:20:27.047266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044029+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:20:27.044029+0000","last_peered":"2024-04-25T00:20:27.044029+0000","last_clean":"2024-04-25T00:20:27.044029+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:20:27.044029+0000","last_undegraded":"2024-04-25T00:20:27.044029+0000","last_fullsized":"2024-04-25T00:20:27.044029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152857+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:20:27.152857+0000","last_peered":"2024-04-25T00:20:27.152857+0000","last_clean":"2024-04-25T00:20:27.152857+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:20:27.152857+0000","last_undegraded":"2024-04-25T00:20:27.152857+0000","last_fullsized":"2024-04-25T00:20:27.152857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048944+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:20:27.048944+0000","last_peered":"2024-04-25T00:20:27.048944+0000","last_clean":"2024-04-25T00:20:27.048944+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:20:27.048944+0000","last_undegraded":"2024-04-25T00:20:27.048944+0000","last_fullsized":"2024-04-25T00:20:27.048944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151234+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:20:27.151234+0000","last_peered":"2024-04-25T00:20:27.151234+0000","last_clean":"2024-04-25T00:20:27.151234+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:20:27.151234+0000","last_undegraded":"2024-04-25T00:20:27.151234+0000","last_fullsized":"2024-04-25T00:20:27.151234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151398+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:20:27.151398+0000","last_peered":"2024-04-25T00:20:27.151398+0000","last_clean":"2024-04-25T00:20:27.151398+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:20:27.151398+0000","last_undegraded":"2024-04-25T00:20:27.151398+0000","last_fullsized":"2024-04-25T00:20:27.151398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041714+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:20:27.041714+0000","last_peered":"2024-04-25T00:20:27.041714+0000","last_clean":"2024-04-25T00:20:27.041714+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:20:27.041714+0000","last_undegraded":"2024-04-25T00:20:27.041714+0000","last_fullsized":"2024-04-25T00:20:27.041714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044288+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:20:27.044288+0000","last_peered":"2024-04-25T00:20:27.044288+0000","last_clean":"2024-04-25T00:20:27.044288+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:20:27.044288+0000","last_undegraded":"2024-04-25T00:20:27.044288+0000","last_fullsized":"2024-04-25T00:20:27.044288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039592+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:20:27.039592+0000","last_peered":"2024-04-25T00:20:27.039592+0000","last_clean":"2024-04-25T00:20:27.039592+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:20:27.039592+0000","last_undegraded":"2024-04-25T00:20:27.039592+0000","last_fullsized":"2024-04-25T00:20:27.039592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041006+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:20:27.041006+0000","last_peered":"2024-04-25T00:20:27.041006+0000","last_clean":"2024-04-25T00:20:27.041006+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:20:27.041006+0000","last_undegraded":"2024-04-25T00:20:27.041006+0000","last_fullsized":"2024-04-25T00:20:27.041006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153842+0000","last_change":"2024-04-25T00:20:27.153842+0000","last_active":"2024-04-25T00:20:27.153842+0000","last_peered":"2024-04-25T00:20:27.153842+0000","last_clean":"2024-04-25T00:20:27.153842+0000","last_became_active":"2024-04-25T00:20:27.152421+0000","last_became_peered":"2024-04-25T00:20:27.152421+0000","last_unstale":"2024-04-25T00:20:27.153842+0000","last_undegraded":"2024-04-25T00:20:27.153842+0000","last_fullsized":"2024-04-25T00:20:27.153842+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:30.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:07.111686+0000","last_change":"2024-04-25T00:20:27.056346+0000","last_active":"2024-04-25T00:21:07.111686+0000","last_peered":"2024-04-25T00:21:07.111686+0000","last_clean":"2024-04-25T00:21:07.111686+0000","last_became_active":"2024-04-25T00:20:27.046470+0000","last_became_peered":"2024-04-25T00:20:27.046470+0000","last_unstale":"2024-04-25T00:21:07.111686+0000","last_undegraded":"2024-04-25T00:21:07.111686+0000","last_fullsized":"2024-04-25T00:21:07.111686+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:31.760140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135989+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:20:27.135989+0000","last_peered":"2024-04-25T00:20:27.135989+0000","last_clean":"2024-04-25T00:20:27.135989+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:20:27.135989+0000","last_undegraded":"2024-04-25T00:20:27.135989+0000","last_fullsized":"2024-04-25T00:20:27.135989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151867+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:20:27.151867+0000","last_peered":"2024-04-25T00:20:27.151867+0000","last_clean":"2024-04-25T00:20:27.151867+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:20:27.151867+0000","last_undegraded":"2024-04-25T00:20:27.151867+0000","last_fullsized":"2024-04-25T00:20:27.151867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136147+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:20:27.136147+0000","last_peered":"2024-04-25T00:20:27.136147+0000","last_clean":"2024-04-25T00:20:27.136147+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:20:27.136147+0000","last_undegraded":"2024-04-25T00:20:27.136147+0000","last_fullsized":"2024-04-25T00:20:27.136147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151599+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:20:27.151599+0000","last_peered":"2024-04-25T00:20:27.151599+0000","last_clean":"2024-04-25T00:20:27.151599+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:20:27.151599+0000","last_undegraded":"2024-04-25T00:20:27.151599+0000","last_fullsized":"2024-04-25T00:20:27.151599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152164+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:20:27.152164+0000","last_peered":"2024-04-25T00:20:27.152164+0000","last_clean":"2024-04-25T00:20:27.152164+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:20:27.152164+0000","last_undegraded":"2024-04-25T00:20:27.152164+0000","last_fullsized":"2024-04-25T00:20:27.152164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.173867+0000","last_change":"2024-04-25T00:20:27.173867+0000","last_active":"2024-04-25T00:20:27.173867+0000","last_peered":"2024-04-25T00:20:27.173867+0000","last_clean":"2024-04-25T00:20:27.173867+0000","last_became_active":"2024-04-25T00:20:27.154058+0000","last_became_peered":"2024-04-25T00:20:27.154058+0000","last_unstale":"2024-04-25T00:20:27.173867+0000","last_undegraded":"2024-04-25T00:20:27.173867+0000","last_fullsized":"2024-04-25T00:20:27.173867+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:55.519210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043781+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:20:27.043781+0000","last_peered":"2024-04-25T00:20:27.043781+0000","last_clean":"2024-04-25T00:20:27.043781+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:20:27.043781+0000","last_undegraded":"2024-04-25T00:20:27.043781+0000","last_fullsized":"2024-04-25T00:20:27.043781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151962+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:20:27.151962+0000","last_peered":"2024-04-25T00:20:27.151962+0000","last_clean":"2024-04-25T00:20:27.151962+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:20:27.151962+0000","last_undegraded":"2024-04-25T00:20:27.151962+0000","last_fullsized":"2024-04-25T00:20:27.151962+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041832+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:20:27.041832+0000","last_peered":"2024-04-25T00:20:27.041832+0000","last_clean":"2024-04-25T00:20:27.041832+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:20:27.041832+0000","last_undegraded":"2024-04-25T00:20:27.041832+0000","last_fullsized":"2024-04-25T00:20:27.041832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152864+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:20:27.152864+0000","last_peered":"2024-04-25T00:20:27.152864+0000","last_clean":"2024-04-25T00:20:27.152864+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:20:27.152864+0000","last_undegraded":"2024-04-25T00:20:27.152864+0000","last_fullsized":"2024-04-25T00:20:27.152864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040722+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:20:27.040722+0000","last_peered":"2024-04-25T00:20:27.040722+0000","last_clean":"2024-04-25T00:20:27.040722+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:20:27.040722+0000","last_undegraded":"2024-04-25T00:20:27.040722+0000","last_fullsized":"2024-04-25T00:20:27.040722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:09.939517+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:21:09.939517+0000","last_peered":"2024-04-25T00:21:09.939517+0000","last_clean":"2024-04-25T00:21:09.939517+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:21:09.939517+0000","last_undegraded":"2024-04-25T00:21:09.939517+0000","last_fullsized":"2024-04-25T00:21:09.939517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041617+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:20:27.041617+0000","last_peered":"2024-04-25T00:20:27.041617+0000","last_clean":"2024-04-25T00:20:27.041617+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:20:27.041617+0000","last_undegraded":"2024-04-25T00:20:27.041617+0000","last_fullsized":"2024-04-25T00:20:27.041617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041203+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:20:27.041203+0000","last_peered":"2024-04-25T00:20:27.041203+0000","last_clean":"2024-04-25T00:20:27.041203+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:20:27.041203+0000","last_undegraded":"2024-04-25T00:20:27.041203+0000","last_fullsized":"2024-04-25T00:20:27.041203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.054883+0000","last_change":"2024-04-25T00:20:27.054883+0000","last_active":"2024-04-25T00:20:27.054883+0000","last_peered":"2024-04-25T00:20:27.054883+0000","last_clean":"2024-04-25T00:20:27.054883+0000","last_became_active":"2024-04-25T00:20:27.049595+0000","last_became_peered":"2024-04-25T00:20:27.049595+0000","last_unstale":"2024-04-25T00:20:27.054883+0000","last_undegraded":"2024-04-25T00:20:27.054883+0000","last_fullsized":"2024-04-25T00:20:27.054883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:05.358193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152019+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:20:27.152019+0000","last_peered":"2024-04-25T00:20:27.152019+0000","last_clean":"2024-04-25T00:20:27.152019+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:20:27.152019+0000","last_undegraded":"2024-04-25T00:20:27.152019+0000","last_fullsized":"2024-04-25T00:20:27.152019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044148+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:20:27.044148+0000","last_peered":"2024-04-25T00:20:27.044148+0000","last_clean":"2024-04-25T00:20:27.044148+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:20:27.044148+0000","last_undegraded":"2024-04-25T00:20:27.044148+0000","last_fullsized":"2024-04-25T00:20:27.044148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048330+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:20:27.048330+0000","last_peered":"2024-04-25T00:20:27.048330+0000","last_clean":"2024-04-25T00:20:27.048330+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:20:27.048330+0000","last_undegraded":"2024-04-25T00:20:27.048330+0000","last_fullsized":"2024-04-25T00:20:27.048330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135705+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:20:27.135705+0000","last_peered":"2024-04-25T00:20:27.135705+0000","last_clean":"2024-04-25T00:20:27.135705+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:20:27.135705+0000","last_undegraded":"2024-04-25T00:20:27.135705+0000","last_fullsized":"2024-04-25T00:20:27.135705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151520+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:20:27.151520+0000","last_peered":"2024-04-25T00:20:27.151520+0000","last_clean":"2024-04-25T00:20:27.151520+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:20:27.151520+0000","last_undegraded":"2024-04-25T00:20:27.151520+0000","last_fullsized":"2024-04-25T00:20:27.151520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151351+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:20:27.151351+0000","last_peered":"2024-04-25T00:20:27.151351+0000","last_clean":"2024-04-25T00:20:27.151351+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:20:27.151351+0000","last_undegraded":"2024-04-25T00:20:27.151351+0000","last_fullsized":"2024-04-25T00:20:27.151351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039075+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:20:27.039075+0000","last_peered":"2024-04-25T00:20:27.039075+0000","last_clean":"2024-04-25T00:20:27.039075+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:20:27.039075+0000","last_undegraded":"2024-04-25T00:20:27.039075+0000","last_fullsized":"2024-04-25T00:20:27.039075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136131+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:20:27.136131+0000","last_peered":"2024-04-25T00:20:27.136131+0000","last_clean":"2024-04-25T00:20:27.136131+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:20:27.136131+0000","last_undegraded":"2024-04-25T00:20:27.136131+0000","last_fullsized":"2024-04-25T00:20:27.136131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.937051+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:21:08.937051+0000","last_peered":"2024-04-25T00:21:08.937051+0000","last_clean":"2024-04-25T00:21:08.937051+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:21:08.937051+0000","last_undegraded":"2024-04-25T00:21:08.937051+0000","last_fullsized":"2024-04-25T00:21:08.937051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043211+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:20:27.043211+0000","last_peered":"2024-04-25T00:20:27.043211+0000","last_clean":"2024-04-25T00:20:27.043211+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:20:27.043211+0000","last_undegraded":"2024-04-25T00:20:27.043211+0000","last_fullsized":"2024-04-25T00:20:27.043211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.038805+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:20:27.038805+0000","last_peered":"2024-04-25T00:20:27.038805+0000","last_clean":"2024-04-25T00:20:27.038805+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:20:27.038805+0000","last_undegraded":"2024-04-25T00:20:27.038805+0000","last_fullsized":"2024-04-25T00:20:27.038805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.176907+0000","last_change":"2024-04-25T00:20:27.176907+0000","last_active":"2024-04-25T00:20:27.176907+0000","last_peered":"2024-04-25T00:20:27.176907+0000","last_clean":"2024-04-25T00:20:27.176907+0000","last_became_active":"2024-04-25T00:20:27.153755+0000","last_became_peered":"2024-04-25T00:20:27.153755+0000","last_unstale":"2024-04-25T00:20:27.176907+0000","last_undegraded":"2024-04-25T00:20:27.176907+0000","last_fullsized":"2024-04-25T00:20:27.176907+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:07.142383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151718+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:20:27.151718+0000","last_peered":"2024-04-25T00:20:27.151718+0000","last_clean":"2024-04-25T00:20:27.151718+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:20:27.151718+0000","last_undegraded":"2024-04-25T00:20:27.151718+0000","last_fullsized":"2024-04-25T00:20:27.151718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152321+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:20:27.152321+0000","last_peered":"2024-04-25T00:20:27.152321+0000","last_clean":"2024-04-25T00:20:27.152321+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:20:27.152321+0000","last_undegraded":"2024-04-25T00:20:27.152321+0000","last_fullsized":"2024-04-25T00:20:27.152321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152741+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:20:27.152741+0000","last_peered":"2024-04-25T00:20:27.152741+0000","last_clean":"2024-04-25T00:20:27.152741+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:20:27.152741+0000","last_undegraded":"2024-04-25T00:20:27.152741+0000","last_fullsized":"2024-04-25T00:20:27.152741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043874+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:20:27.043874+0000","last_peered":"2024-04-25T00:20:27.043874+0000","last_clean":"2024-04-25T00:20:27.043874+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:20:27.043874+0000","last_undegraded":"2024-04-25T00:20:27.043874+0000","last_fullsized":"2024-04-25T00:20:27.043874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.038902+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:20:27.038902+0000","last_peered":"2024-04-25T00:20:27.038902+0000","last_clean":"2024-04-25T00:20:27.038902+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:20:27.038902+0000","last_undegraded":"2024-04-25T00:20:27.038902+0000","last_fullsized":"2024-04-25T00:20:27.038902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":202,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.937052+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:21:08.937052+0000","last_peered":"2024-04-25T00:21:08.937052+0000","last_clean":"2024-04-25T00:21:08.937052+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:21:08.937052+0000","last_undegraded":"2024-04-25T00:21:08.937052+0000","last_fullsized":"2024-04-25T00:21:08.937052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60642,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":130,"num_write":35,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042381+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:20:27.042381+0000","last_peered":"2024-04-25T00:20:27.042381+0000","last_clean":"2024-04-25T00:20:27.042381+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:20:27.042381+0000","last_undegraded":"2024-04-25T00:20:27.042381+0000","last_fullsized":"2024-04-25T00:20:27.042381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153230+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:20:27.153230+0000","last_peered":"2024-04-25T00:20:27.153230+0000","last_clean":"2024-04-25T00:20:27.153230+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:20:27.153230+0000","last_undegraded":"2024-04-25T00:20:27.153230+0000","last_fullsized":"2024-04-25T00:20:27.153230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041552+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:20:27.041552+0000","last_peered":"2024-04-25T00:20:27.041552+0000","last_clean":"2024-04-25T00:20:27.041552+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:20:27.041552+0000","last_undegraded":"2024-04-25T00:20:27.041552+0000","last_fullsized":"2024-04-25T00:20:27.041552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048551+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:20:27.048551+0000","last_peered":"2024-04-25T00:20:27.048551+0000","last_clean":"2024-04-25T00:20:27.048551+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:20:27.048551+0000","last_undegraded":"2024-04-25T00:20:27.048551+0000","last_fullsized":"2024-04-25T00:20:27.048551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044108+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:20:27.044108+0000","last_peered":"2024-04-25T00:20:27.044108+0000","last_clean":"2024-04-25T00:20:27.044108+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:20:27.044108+0000","last_undegraded":"2024-04-25T00:20:27.044108+0000","last_fullsized":"2024-04-25T00:20:27.044108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151820+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:20:27.151820+0000","last_peered":"2024-04-25T00:20:27.151820+0000","last_clean":"2024-04-25T00:20:27.151820+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:20:27.151820+0000","last_undegraded":"2024-04-25T00:20:27.151820+0000","last_fullsized":"2024-04-25T00:20:27.151820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043367+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:20:27.043367+0000","last_peered":"2024-04-25T00:20:27.043367+0000","last_clean":"2024-04-25T00:20:27.043367+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:20:27.043367+0000","last_undegraded":"2024-04-25T00:20:27.043367+0000","last_fullsized":"2024-04-25T00:20:27.043367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153934+0000","last_change":"2024-04-25T00:20:27.153934+0000","last_active":"2024-04-25T00:20:27.153934+0000","last_peered":"2024-04-25T00:20:27.153934+0000","last_clean":"2024-04-25T00:20:27.153934+0000","last_became_active":"2024-04-25T00:20:27.152647+0000","last_became_peered":"2024-04-25T00:20:27.152647+0000","last_unstale":"2024-04-25T00:20:27.153934+0000","last_undegraded":"2024-04-25T00:20:27.153934+0000","last_fullsized":"2024-04-25T00:20:27.153934+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.389722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.943761+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:21:08.943761+0000","last_peered":"2024-04-25T00:21:08.943761+0000","last_clean":"2024-04-25T00:21:08.943761+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:21:08.943761+0000","last_undegraded":"2024-04-25T00:21:08.943761+0000","last_fullsized":"2024-04-25T00:21:08.943761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042198+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:20:27.042198+0000","last_peered":"2024-04-25T00:20:27.042198+0000","last_clean":"2024-04-25T00:20:27.042198+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:20:27.042198+0000","last_undegraded":"2024-04-25T00:20:27.042198+0000","last_fullsized":"2024-04-25T00:20:27.042198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048902+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:20:27.048902+0000","last_peered":"2024-04-25T00:20:27.048902+0000","last_clean":"2024-04-25T00:20:27.048902+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:20:27.048902+0000","last_undegraded":"2024-04-25T00:20:27.048902+0000","last_fullsized":"2024-04-25T00:20:27.048902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041326+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:20:27.041326+0000","last_peered":"2024-04-25T00:20:27.041326+0000","last_clean":"2024-04-25T00:20:27.041326+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:20:27.041326+0000","last_undegraded":"2024-04-25T00:20:27.041326+0000","last_fullsized":"2024-04-25T00:20:27.041326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151539+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:20:27.151539+0000","last_peered":"2024-04-25T00:20:27.151539+0000","last_clean":"2024-04-25T00:20:27.151539+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:20:27.151539+0000","last_undegraded":"2024-04-25T00:20:27.151539+0000","last_fullsized":"2024-04-25T00:20:27.151539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151405+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:20:27.151405+0000","last_peered":"2024-04-25T00:20:27.151405+0000","last_clean":"2024-04-25T00:20:27.151405+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:20:27.151405+0000","last_undegraded":"2024-04-25T00:20:27.151405+0000","last_fullsized":"2024-04-25T00:20:27.151405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043123+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:20:27.043123+0000","last_peered":"2024-04-25T00:20:27.043123+0000","last_clean":"2024-04-25T00:20:27.043123+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:20:27.043123+0000","last_undegraded":"2024-04-25T00:20:27.043123+0000","last_fullsized":"2024-04-25T00:20:27.043123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044585+0000","last_change":"2024-04-25T00:19:33.880006+0000","last_active":"2024-04-25T00:20:27.044585+0000","last_peered":"2024-04-25T00:20:27.044585+0000","last_clean":"2024-04-25T00:20:27.044585+0000","last_became_active":"2024-04-25T00:19:33.879054+0000","last_became_peered":"2024-04-25T00:19:33.879054+0000","last_unstale":"2024-04-25T00:20:27.044585+0000","last_undegraded":"2024-04-25T00:20:27.044585+0000","last_fullsized":"2024-04-25T00:20:27.044585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:43.232276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041495+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:20:27.041495+0000","last_peered":"2024-04-25T00:20:27.041495+0000","last_clean":"2024-04-25T00:20:27.041495+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:20:27.041495+0000","last_undegraded":"2024-04-25T00:20:27.041495+0000","last_fullsized":"2024-04-25T00:20:27.041495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039212+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:20:27.039212+0000","last_peered":"2024-04-25T00:20:27.039212+0000","last_clean":"2024-04-25T00:20:27.039212+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:20:27.039212+0000","last_undegraded":"2024-04-25T00:20:27.039212+0000","last_fullsized":"2024-04-25T00:20:27.039212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046537+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:20:27.046537+0000","last_peered":"2024-04-25T00:20:27.046537+0000","last_clean":"2024-04-25T00:20:27.046537+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:20:27.046537+0000","last_undegraded":"2024-04-25T00:20:27.046537+0000","last_fullsized":"2024-04-25T00:20:27.046537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048652+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:20:27.048652+0000","last_peered":"2024-04-25T00:20:27.048652+0000","last_clean":"2024-04-25T00:20:27.048652+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:20:27.048652+0000","last_undegraded":"2024-04-25T00:20:27.048652+0000","last_fullsized":"2024-04-25T00:20:27.048652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:09.115013+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:21:09.115013+0000","last_peered":"2024-04-25T00:21:09.115013+0000","last_clean":"2024-04-25T00:21:09.115013+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:21:09.115013+0000","last_undegraded":"2024-04-25T00:21:09.115013+0000","last_fullsized":"2024-04-25T00:21:09.115013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046853+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:20:27.046853+0000","last_peered":"2024-04-25T00:20:27.046853+0000","last_clean":"2024-04-25T00:20:27.046853+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:20:27.046853+0000","last_undegraded":"2024-04-25T00:20:27.046853+0000","last_fullsized":"2024-04-25T00:20:27.046853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048657+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:20:27.048657+0000","last_peered":"2024-04-25T00:20:27.048657+0000","last_clean":"2024-04-25T00:20:27.048657+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:20:27.048657+0000","last_undegraded":"2024-04-25T00:20:27.048657+0000","last_fullsized":"2024-04-25T00:20:27.048657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151515+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:20:27.151515+0000","last_peered":"2024-04-25T00:20:27.151515+0000","last_clean":"2024-04-25T00:20:27.151515+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:20:27.151515+0000","last_undegraded":"2024-04-25T00:20:27.151515+0000","last_fullsized":"2024-04-25T00:20:27.151515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:08.943047+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:21:08.943047+0000","last_peered":"2024-04-25T00:21:08.943047+0000","last_clean":"2024-04-25T00:21:08.943047+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:21:08.943047+0000","last_undegraded":"2024-04-25T00:21:08.943047+0000","last_fullsized":"2024-04-25T00:21:08.943047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043698+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:20:27.043698+0000","last_peered":"2024-04-25T00:20:27.043698+0000","last_clean":"2024-04-25T00:20:27.043698+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:20:27.043698+0000","last_undegraded":"2024-04-25T00:20:27.043698+0000","last_fullsized":"2024-04-25T00:20:27.043698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135744+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:20:27.135744+0000","last_peered":"2024-04-25T00:20:27.135744+0000","last_clean":"2024-04-25T00:20:27.135744+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:20:27.135744+0000","last_undegraded":"2024-04-25T00:20:27.135744+0000","last_fullsized":"2024-04-25T00:20:27.135744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152165+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:20:27.152165+0000","last_peered":"2024-04-25T00:20:27.152165+0000","last_clean":"2024-04-25T00:20:27.152165+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:20:27.152165+0000","last_undegraded":"2024-04-25T00:20:27.152165+0000","last_fullsized":"2024-04-25T00:20:27.152165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042393+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:20:27.042393+0000","last_peered":"2024-04-25T00:20:27.042393+0000","last_clean":"2024-04-25T00:20:27.042393+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:20:27.042393+0000","last_undegraded":"2024-04-25T00:20:27.042393+0000","last_fullsized":"2024-04-25T00:20:27.042393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.049784+0000","last_change":"2024-04-25T00:20:27.049784+0000","last_active":"2024-04-25T00:20:27.049784+0000","last_peered":"2024-04-25T00:20:27.049784+0000","last_clean":"2024-04-25T00:20:27.049784+0000","last_became_active":"2024-04-25T00:20:27.049413+0000","last_became_peered":"2024-04-25T00:20:27.049413+0000","last_unstale":"2024-04-25T00:20:27.049784+0000","last_undegraded":"2024-04-25T00:20:27.049784+0000","last_fullsized":"2024-04-25T00:20:27.049784+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:26.533987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048161+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:20:27.048161+0000","last_peered":"2024-04-25T00:20:27.048161+0000","last_clean":"2024-04-25T00:20:27.048161+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:20:27.048161+0000","last_undegraded":"2024-04-25T00:20:27.048161+0000","last_fullsized":"2024-04-25T00:20:27.048161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042081+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:20:27.042081+0000","last_peered":"2024-04-25T00:20:27.042081+0000","last_clean":"2024-04-25T00:20:27.042081+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:20:27.042081+0000","last_undegraded":"2024-04-25T00:20:27.042081+0000","last_fullsized":"2024-04-25T00:20:27.042081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044368+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:20:27.044368+0000","last_peered":"2024-04-25T00:20:27.044368+0000","last_clean":"2024-04-25T00:20:27.044368+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:20:27.044368+0000","last_undegraded":"2024-04-25T00:20:27.044368+0000","last_fullsized":"2024-04-25T00:20:27.044368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042424+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:20:27.042424+0000","last_peered":"2024-04-25T00:20:27.042424+0000","last_clean":"2024-04-25T00:20:27.042424+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:20:27.042424+0000","last_undegraded":"2024-04-25T00:20:27.042424+0000","last_fullsized":"2024-04-25T00:20:27.042424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048828+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:20:27.048828+0000","last_peered":"2024-04-25T00:20:27.048828+0000","last_clean":"2024-04-25T00:20:27.048828+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:20:27.048828+0000","last_undegraded":"2024-04-25T00:20:27.048828+0000","last_fullsized":"2024-04-25T00:20:27.048828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135926+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:20:27.135926+0000","last_peered":"2024-04-25T00:20:27.135926+0000","last_clean":"2024-04-25T00:20:27.135926+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:20:27.135926+0000","last_undegraded":"2024-04-25T00:20:27.135926+0000","last_fullsized":"2024-04-25T00:20:27.135926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039561+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:20:27.039561+0000","last_peered":"2024-04-25T00:20:27.039561+0000","last_clean":"2024-04-25T00:20:27.039561+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:20:27.039561+0000","last_undegraded":"2024-04-25T00:20:27.039561+0000","last_fullsized":"2024-04-25T00:20:27.039561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136253+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:20:27.136253+0000","last_peered":"2024-04-25T00:20:27.136253+0000","last_clean":"2024-04-25T00:20:27.136253+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:20:27.136253+0000","last_undegraded":"2024-04-25T00:20:27.136253+0000","last_fullsized":"2024-04-25T00:20:27.136253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.039980+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:20:27.039980+0000","last_peered":"2024-04-25T00:20:27.039980+0000","last_clean":"2024-04-25T00:20:27.039980+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:20:27.039980+0000","last_undegraded":"2024-04-25T00:20:27.039980+0000","last_fullsized":"2024-04-25T00:20:27.039980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152851+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:20:27.152851+0000","last_peered":"2024-04-25T00:20:27.152851+0000","last_clean":"2024-04-25T00:20:27.152851+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:20:27.152851+0000","last_undegraded":"2024-04-25T00:20:27.152851+0000","last_fullsized":"2024-04-25T00:20:27.152851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151694+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:20:27.151694+0000","last_peered":"2024-04-25T00:20:27.151694+0000","last_clean":"2024-04-25T00:20:27.151694+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:20:27.151694+0000","last_undegraded":"2024-04-25T00:20:27.151694+0000","last_fullsized":"2024-04-25T00:20:27.151694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152702+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:20:27.152702+0000","last_peered":"2024-04-25T00:20:27.152702+0000","last_clean":"2024-04-25T00:20:27.152702+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:20:27.152702+0000","last_undegraded":"2024-04-25T00:20:27.152702+0000","last_fullsized":"2024-04-25T00:20:27.152702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041730+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:20:27.041730+0000","last_peered":"2024-04-25T00:20:27.041730+0000","last_clean":"2024-04-25T00:20:27.041730+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:20:27.041730+0000","last_undegraded":"2024-04-25T00:20:27.041730+0000","last_fullsized":"2024-04-25T00:20:27.041730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041358+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:20:27.041358+0000","last_peered":"2024-04-25T00:20:27.041358+0000","last_clean":"2024-04-25T00:20:27.041358+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:20:27.041358+0000","last_undegraded":"2024-04-25T00:20:27.041358+0000","last_fullsized":"2024-04-25T00:20:27.041358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.153703+0000","last_change":"2024-04-25T00:20:27.153703+0000","last_active":"2024-04-25T00:20:27.153703+0000","last_peered":"2024-04-25T00:20:27.153703+0000","last_clean":"2024-04-25T00:20:27.153703+0000","last_became_active":"2024-04-25T00:20:27.152715+0000","last_became_peered":"2024-04-25T00:20:27.152715+0000","last_unstale":"2024-04-25T00:20:27.153703+0000","last_undegraded":"2024-04-25T00:20:27.153703+0000","last_fullsized":"2024-04-25T00:20:27.153703+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:09.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.135777+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:20:27.135777+0000","last_peered":"2024-04-25T00:20:27.135777+0000","last_clean":"2024-04-25T00:20:27.135777+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:20:27.135777+0000","last_undegraded":"2024-04-25T00:20:27.135777+0000","last_fullsized":"2024-04-25T00:20:27.135777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043801+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:20:27.043801+0000","last_peered":"2024-04-25T00:20:27.043801+0000","last_clean":"2024-04-25T00:20:27.043801+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:20:27.043801+0000","last_undegraded":"2024-04-25T00:20:27.043801+0000","last_fullsized":"2024-04-25T00:20:27.043801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152609+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:20:27.152609+0000","last_peered":"2024-04-25T00:20:27.152609+0000","last_clean":"2024-04-25T00:20:27.152609+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:20:27.152609+0000","last_undegraded":"2024-04-25T00:20:27.152609+0000","last_fullsized":"2024-04-25T00:20:27.152609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040231+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:20:27.040231+0000","last_peered":"2024-04-25T00:20:27.040231+0000","last_clean":"2024-04-25T00:20:27.040231+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:20:27.040231+0000","last_undegraded":"2024-04-25T00:20:27.040231+0000","last_fullsized":"2024-04-25T00:20:27.040231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136038+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:20:27.136038+0000","last_peered":"2024-04-25T00:20:27.136038+0000","last_clean":"2024-04-25T00:20:27.136038+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:20:27.136038+0000","last_undegraded":"2024-04-25T00:20:27.136038+0000","last_fullsized":"2024-04-25T00:20:27.136038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152710+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:20:27.152710+0000","last_peered":"2024-04-25T00:20:27.152710+0000","last_clean":"2024-04-25T00:20:27.152710+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:20:27.152710+0000","last_undegraded":"2024-04-25T00:20:27.152710+0000","last_fullsized":"2024-04-25T00:20:27.152710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041613+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:20:27.041613+0000","last_peered":"2024-04-25T00:20:27.041613+0000","last_clean":"2024-04-25T00:20:27.041613+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:20:27.041613+0000","last_undegraded":"2024-04-25T00:20:27.041613+0000","last_fullsized":"2024-04-25T00:20:27.041613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042946+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:20:27.042946+0000","last_peered":"2024-04-25T00:20:27.042946+0000","last_clean":"2024-04-25T00:20:27.042946+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:20:27.042946+0000","last_undegraded":"2024-04-25T00:20:27.042946+0000","last_fullsized":"2024-04-25T00:20:27.042946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.041906+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:20:27.041906+0000","last_peered":"2024-04-25T00:20:27.041906+0000","last_clean":"2024-04-25T00:20:27.041906+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:20:27.041906+0000","last_undegraded":"2024-04-25T00:20:27.041906+0000","last_fullsized":"2024-04-25T00:20:27.041906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:21:09.939398+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:21:09.939398+0000","last_peered":"2024-04-25T00:21:09.939398+0000","last_clean":"2024-04-25T00:21:09.939398+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:21:09.939398+0000","last_undegraded":"2024-04-25T00:21:09.939398+0000","last_fullsized":"2024-04-25T00:21:09.939398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044141+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:20:27.044141+0000","last_peered":"2024-04-25T00:20:27.044141+0000","last_clean":"2024-04-25T00:20:27.044141+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:20:27.044141+0000","last_undegraded":"2024-04-25T00:20:27.044141+0000","last_fullsized":"2024-04-25T00:20:27.044141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151240+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:20:27.151240+0000","last_peered":"2024-04-25T00:20:27.151240+0000","last_clean":"2024-04-25T00:20:27.151240+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:20:27.151240+0000","last_undegraded":"2024-04-25T00:20:27.151240+0000","last_fullsized":"2024-04-25T00:20:27.151240+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.152987+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:20:27.152987+0000","last_peered":"2024-04-25T00:20:27.152987+0000","last_clean":"2024-04-25T00:20:27.152987+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:20:27.152987+0000","last_undegraded":"2024-04-25T00:20:27.152987+0000","last_fullsized":"2024-04-25T00:20:27.152987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044296+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:20:27.044296+0000","last_peered":"2024-04-25T00:20:27.044296+0000","last_clean":"2024-04-25T00:20:27.044296+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:20:27.044296+0000","last_undegraded":"2024-04-25T00:20:27.044296+0000","last_fullsized":"2024-04-25T00:20:27.044296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151171+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:20:27.151171+0000","last_peered":"2024-04-25T00:20:27.151171+0000","last_clean":"2024-04-25T00:20:27.151171+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:20:27.151171+0000","last_undegraded":"2024-04-25T00:20:27.151171+0000","last_fullsized":"2024-04-25T00:20:27.151171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048770+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:20:27.048770+0000","last_peered":"2024-04-25T00:20:27.048770+0000","last_clean":"2024-04-25T00:20:27.048770+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:20:27.048770+0000","last_undegraded":"2024-04-25T00:20:27.048770+0000","last_fullsized":"2024-04-25T00:20:27.048770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.044188+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:20:27.044188+0000","last_peered":"2024-04-25T00:20:27.044188+0000","last_clean":"2024-04-25T00:20:27.044188+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:20:27.044188+0000","last_undegraded":"2024-04-25T00:20:27.044188+0000","last_fullsized":"2024-04-25T00:20:27.044188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.043927+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:20:27.043927+0000","last_peered":"2024-04-25T00:20:27.043927+0000","last_clean":"2024-04-25T00:20:27.043927+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:20:27.043927+0000","last_undegraded":"2024-04-25T00:20:27.043927+0000","last_fullsized":"2024-04-25T00:20:27.043927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040850+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:20:27.040850+0000","last_peered":"2024-04-25T00:20:27.040850+0000","last_clean":"2024-04-25T00:20:27.040850+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:20:27.040850+0000","last_undegraded":"2024-04-25T00:20:27.040850+0000","last_fullsized":"2024-04-25T00:20:27.040850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040127+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:20:27.040127+0000","last_peered":"2024-04-25T00:20:27.040127+0000","last_clean":"2024-04-25T00:20:27.040127+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:20:27.040127+0000","last_undegraded":"2024-04-25T00:20:27.040127+0000","last_fullsized":"2024-04-25T00:20:27.040127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151631+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:20:27.151631+0000","last_peered":"2024-04-25T00:20:27.151631+0000","last_clean":"2024-04-25T00:20:27.151631+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:20:27.151631+0000","last_undegraded":"2024-04-25T00:20:27.151631+0000","last_fullsized":"2024-04-25T00:20:27.151631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048931+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:20:27.048931+0000","last_peered":"2024-04-25T00:20:27.048931+0000","last_clean":"2024-04-25T00:20:27.048931+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:20:27.048931+0000","last_undegraded":"2024-04-25T00:20:27.048931+0000","last_fullsized":"2024-04-25T00:20:27.048931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.045607+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:20:27.045607+0000","last_peered":"2024-04-25T00:20:27.045607+0000","last_clean":"2024-04-25T00:20:27.045607+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:20:27.045607+0000","last_undegraded":"2024-04-25T00:20:27.045607+0000","last_fullsized":"2024-04-25T00:20:27.045607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.040794+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:20:27.040794+0000","last_peered":"2024-04-25T00:20:27.040794+0000","last_clean":"2024-04-25T00:20:27.040794+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:20:27.040794+0000","last_undegraded":"2024-04-25T00:20:27.040794+0000","last_fullsized":"2024-04-25T00:20:27.040794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136323+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:20:27.136323+0000","last_peered":"2024-04-25T00:20:27.136323+0000","last_clean":"2024-04-25T00:20:27.136323+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:20:27.136323+0000","last_undegraded":"2024-04-25T00:20:27.136323+0000","last_fullsized":"2024-04-25T00:20:27.136323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.151694+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:20:27.151694+0000","last_peered":"2024-04-25T00:20:27.151694+0000","last_clean":"2024-04-25T00:20:27.151694+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:20:27.151694+0000","last_undegraded":"2024-04-25T00:20:27.151694+0000","last_fullsized":"2024-04-25T00:20:27.151694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.136015+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:20:27.136015+0000","last_peered":"2024-04-25T00:20:27.136015+0000","last_clean":"2024-04-25T00:20:27.136015+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:20:27.136015+0000","last_undegraded":"2024-04-25T00:20:27.136015+0000","last_fullsized":"2024-04-25T00:20:27.136015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.042377+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:20:27.042377+0000","last_peered":"2024-04-25T00:20:27.042377+0000","last_clean":"2024-04-25T00:20:27.042377+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:20:27.042377+0000","last_undegraded":"2024-04-25T00:20:27.042377+0000","last_fullsized":"2024-04-25T00:20:27.042377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.048541+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:20:27.048541+0000","last_peered":"2024-04-25T00:20:27.048541+0000","last_clean":"2024-04-25T00:20:27.048541+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:20:27.048541+0000","last_undegraded":"2024-04-25T00:20:27.048541+0000","last_fullsized":"2024-04-25T00:20:27.048541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T00:20:27.046816+0000","last_change":"2024-04-25T00:20:27.046816+0000","last_active":"2024-04-25T00:20:27.046816+0000","last_peered":"2024-04-25T00:20:27.046816+0000","last_clean":"2024-04-25T00:20:27.046816+0000","last_became_active":"2024-04-25T00:20:27.045845+0000","last_became_peered":"2024-04-25T00:20:27.045845+0000","last_unstale":"2024-04-25T00:20:27.046816+0000","last_undegraded":"2024-04-25T00:20:27.046816+0000","last_fullsized":"2024-04-25T00:20:27.046816+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:45.465308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284395,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67846,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":715,"num_read_kb":425,"num_write":238,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":502034,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":1149211,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":505388,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":501670,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1097886,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":496796,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":558468,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":498252,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":559079,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":496504,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":501261,"data_compressed":20583,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":1093749,"data_compressed":56747,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34525,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:21:15.133 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T00:21:15.133 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:21:15.133 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:21:15.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T00:21:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:14 smithi122 ceph-mon[32959]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:14 smithi071 ceph-mon[32667]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:15.385 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:16.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:15 smithi045 ceph-mon[29194]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:16.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:15 smithi122 ceph-mon[32959]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:15 smithi071 ceph-mon[32667]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- 172.21.15.45:0/1293304685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101420 msgr2=0x7fe578101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 --2- 172.21.15.45:0/1293304685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101420 0x7fe578101800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe5680099c0 tx=0x7fe568009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- 172.21.15.45:0/1293304685 shutdown_connections 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 --2- 172.21.15.45:0/1293304685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe57810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 --2- 172.21.15.45:0/1293304685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101420 0x7fe578101800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 --2- 172.21.15.45:0/1293304685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101dd0 0x7fe578105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- 172.21.15.45:0/1293304685 >> 172.21.15.45:0/1293304685 conn(0x7fe578077180 msgr2=0x7fe578077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- 172.21.15.45:0/1293304685 shutdown_connections 2024-04-25T00:21:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- 172.21.15.45:0/1293304685 wait complete. 2024-04-25T00:21:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 Processor -- start 2024-04-25T00:21:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.024+0000 7fe57e778700 1 -- start start 2024-04-25T00:21:17.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 0x7fe578102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe5781038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe578111470 con 0x7fe578101dd0 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781112f0 con 0x7fe578106470 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57e778700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781115f0 con 0x7fe578101420 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42772/0 (socket says 172.21.15.45:42772) 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 -- 172.21.15.45:0/1968241146 learned_addr learned my addr 172.21.15.45:0/1968241146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe577fff700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 0x7fe578102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57cd15700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe5781038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 -- 172.21.15.45:0/1968241146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 msgr2=0x7fe578102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 0x7fe578102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 -- 172.21.15.45:0/1968241146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 msgr2=0x7fe5781038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe5781038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe5777fe700 1 -- 172.21.15.45:0/1968241146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5781a6ef0 con 0x7fe578101dd0 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe577fff700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 0x7fe578102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.025+0000 7fe57cd15700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe5781038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:17.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe5777fe700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d950 tx=0x7fe56c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c017070 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56c00de30 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5781a71e0 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0054c0 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.026+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5781a76d0 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.027+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe578068f20 con 0x7fe578101dd0 2024-04-25T00:21:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.028+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe56c004750 con 0x7fe578101dd0 2024-04-25T00:21:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.028+0000 7fe5757fa700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 0x7fe5600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.029+0000 7fe577fff700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 0x7fe5600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.029+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe56c0ceea0 con 0x7fe578101dd0 2024-04-25T00:21:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.029+0000 7fe577fff700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 0x7fe5600a5170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe568000c00 tx=0x7fe568005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.033+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe56c09b960 con 0x7fe578101dd0 2024-04-25T00:21:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:16 smithi045 ceph-mon[29194]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:17.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.287+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe57802db30 con 0x7fe578101dd0 2024-04-25T00:21:17.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.288+0000 7fe5757fa700 1 -- 172.21.15.45:0/1968241146 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe56c09b260 con 0x7fe578101dd0 2024-04-25T00:21:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:17.289 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 msgr2=0x7fe5600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 0x7fe5600a5170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe568000c00 tx=0x7fe568005fd0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 msgr2=0x7fe578103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d950 tx=0x7fe56c00dd10 comp rx=0 tx=0).stop 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 shutdown_connections 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe578101420 0x7fe578102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe5600a2cb0 0x7fe5600a5170 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578106470 0x7fe5781038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 --2- 172.21.15.45:0/1968241146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe578101dd0 0x7fe578103370 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:17.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 >> 172.21.15.45:0/1968241146 conn(0x7fe578077180 msgr2=0x7fe57806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:17.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 shutdown_connections 2024-04-25T00:21:17.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:17.290+0000 7fe57e778700 1 -- 172.21.15.45:0/1968241146 wait complete. 2024-04-25T00:21:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:16 smithi122 ceph-mon[32959]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:16 smithi071 ceph-mon[32667]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:17.918 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:21:17.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 -- 172.21.15.45:0/3155242293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc01006a0 msgr2=0x7f1bc0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/3155242293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc01006a0 0x7f1bc0100ac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009900 tx=0x7f1bb002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 -- 172.21.15.45:0/3155242293 shutdown_connections 2024-04-25T00:21:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/3155242293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/3155242293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc0101890 0x7f1bc0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/3155242293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc01006a0 0x7f1bc0100ac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 -- 172.21.15.45:0/3155242293 >> 172.21.15.45:0/3155242293 conn(0x7f1bc00fbc30 msgr2=0x7f1bc00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 -- 172.21.15.45:0/3155242293 shutdown_connections 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.025+0000 7f1bc80bd640 1 -- 172.21.15.45:0/3155242293 wait complete. 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 Processor -- start 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 -- start start 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc01006a0 0x7f1bc0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc010c3c0 con 0x7f1bc0101890 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc010c240 con 0x7f1bc01006a0 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc80bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc010c540 con 0x7f1bc0102250 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc5658640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc5658640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42782/0 (socket says 172.21.15.45:42782) 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc5658640 1 -- 172.21.15.45:0/1776368244 learned_addr learned my addr 172.21.15.45:0/1776368244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc5e59640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc01006a0 0x7f1bc0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.026+0000 7f1bc665a640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 -- 172.21.15.45:0/1776368244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 msgr2=0x7f1bc019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 -- 172.21.15.45:0/1776368244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc01006a0 msgr2=0x7f1bc0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc01006a0 0x7f1bc0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 -- 172.21.15.45:0/1776368244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc019f740 con 0x7f1bc0101890 2024-04-25T00:21:18.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc665a640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc5658640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c890 tx=0x7f1bbc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004220 con 0x7f1bc0101890 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0043c0 con 0x7f1bc0101890 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc019fa30 con 0x7f1bc0101890 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0049e0 con 0x7f1bc0101890 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.027+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc01a0000 con 0x7f1bc0101890 2024-04-25T00:21:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.028+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc01a0290 con 0x7f1bc0101890 2024-04-25T00:21:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.029+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1bbc0079b0 con 0x7f1bc0101890 2024-04-25T00:21:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.030+0000 7f1bb6ffd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 0x7f1bac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.030+0000 7f1bc5e59640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 0x7f1bac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.030+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f1bbc0cf570 con 0x7f1bc0101890 2024-04-25T00:21:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.031+0000 7f1bc5e59640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 0x7f1bac0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bb002aa00 tx=0x7f1bb00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.035+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1bbc014030 con 0x7f1bc0101890 2024-04-25T00:21:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:17 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1968241146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:21:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.222+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bc01a0460 con 0x7f1bc0101890 2024-04-25T00:21:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.224+0000 7f1bb6ffd640 1 -- 172.21.15.45:0/1776368244 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24616 (secure 0 0 0) 0x7f1bbc09c030 con 0x7f1bc0101890 2024-04-25T00:21:18.225 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:18.225 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:20:27.035516+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 msgr2=0x7f1bac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 0x7f1bac0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bb002aa00 tx=0x7f1bb00315b0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 msgr2=0x7f1bc0198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c890 tx=0x7f1bbc00cd50 comp rx=0 tx=0).stop 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 shutdown_connections 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f1bac0a2c20 0x7f1bac0a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bc0102250 0x7f1bc019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bc0101890 0x7f1bc0198060 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 --2- 172.21.15.45:0/1776368244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bc01006a0 0x7f1bc0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 >> 172.21.15.45:0/1776368244 conn(0x7f1bc00fbc30 msgr2=0x7f1bc00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 shutdown_connections 2024-04-25T00:21:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.226+0000 7f1bc80bd640 1 -- 172.21.15.45:0/1776368244 wait complete. 2024-04-25T00:21:18.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:18.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:18.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:17 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1968241146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:21:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1541486031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8102250 msgr2=0x7f5da8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1541486031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8102250 0x7f5da8108ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009ca0 tx=0x7f5da002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1541486031 shutdown_connections 2024-04-25T00:21:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1541486031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8102250 0x7f5da8108ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1541486031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8101890 0x7f5da8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1541486031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1541486031 >> 172.21.15.45:0/1541486031 conn(0x7f5da80fbc10 msgr2=0x7f5da80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1541486031 shutdown_connections 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.344+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1541486031 wait complete. 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 Processor -- start 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 -- start start 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8102250 0x7f5da819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c300 con 0x7f5da8101890 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c180 con 0x7f5da8102250 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5daf2a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5da810c480 con 0x7f5da81006a0 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42798/0 (socket says 172.21.15.45:42798) 2024-04-25T00:21:18.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 -- 172.21.15.45:0/1720648587 learned_addr learned my addr 172.21.15.45:0/1720648587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dad041640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 -- 172.21.15.45:0/1720648587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 msgr2=0x7f5da806a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da806a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 -- 172.21.15.45:0/1720648587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8102250 msgr2=0x7f5da819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dad842640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8102250 0x7f5da819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8102250 0x7f5da819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 -- 172.21.15.45:0/1720648587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da819f690 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dad041640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.345+0000 7f5dac840640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c890 tx=0x7f5d9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.346+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98004220 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.346+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d980043c0 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.346+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980049e0 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.346+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da819f920 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.346+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da819fe20 con 0x7f5da8101890 2024-04-25T00:21:18.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.347+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8107960 con 0x7f5da8101890 2024-04-25T00:21:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.348+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d980079b0 con 0x7f5da8101890 2024-04-25T00:21:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.348+0000 7f5d9e7fc640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 0x7f5d940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.348+0000 7f5dad041640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 0x7f5d940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.348+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5d980cf670 con 0x7f5da8101890 2024-04-25T00:21:18.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.349+0000 7f5dad041640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 0x7f5d940a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5da81016f0 tx=0x7f5da400a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.352+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d98014030 con 0x7f5da8101890 2024-04-25T00:21:18.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:17 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1968241146' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:21:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.536+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5da802d300 con 0x7f5da8101890 2024-04-25T00:21:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.537+0000 7f5d9e7fc640 1 -- 172.21.15.45:0/1720648587 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f5d9809c130 con 0x7f5da8101890 2024-04-25T00:21:18.537 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 msgr2=0x7f5d940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 0x7f5d940a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5da81016f0 tx=0x7f5da400a5e0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 msgr2=0x7f5da819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c890 tx=0x7f5d9800cd50 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 shutdown_connections 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5d940a2c20 0x7f5d940a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5da8102250 0x7f5da819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da8101890 0x7f5da819c760 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 --2- 172.21.15.45:0/1720648587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da81006a0 0x7f5da806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 >> 172.21.15.45:0/1720648587 conn(0x7f5da80fbc10 msgr2=0x7f5da8104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 shutdown_connections 2024-04-25T00:21:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.539+0000 7f5daf2a5640 1 -- 172.21.15.45:0/1720648587 wait complete. 2024-04-25T00:21:18.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 -- 172.21.15.45:0/2121690423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0104bb0 msgr2=0x7fbac0106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 --2- 172.21.15.45:0/2121690423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0104bb0 0x7fbac0106fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 -- 172.21.15.45:0/2121690423 shutdown_connections 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 --2- 172.21.15.45:0/2121690423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac01074e0 0x7fbac01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 --2- 172.21.15.45:0/2121690423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0104bb0 0x7fbac0106fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 --2- 172.21.15.45:0/2121690423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac0102280 0x7fbac0104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.655+0000 7fbac7942640 1 -- 172.21.15.45:0/2121690423 >> 172.21.15.45:0/2121690423 conn(0x7fbac00fbc10 msgr2=0x7fbac00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- 172.21.15.45:0/2121690423 shutdown_connections 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- 172.21.15.45:0/2121690423 wait complete. 2024-04-25T00:21:18.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 Processor -- start 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- start start 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 0x7fbac0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 0x7fbac019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d460 con 0x7fbac0102280 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d2e0 con 0x7fbac0104bb0 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac7942640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d5e0 con 0x7fbac01074e0 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac56de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac56de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42814/0 (socket says 172.21.15.45:42814) 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.656+0000 7fbac56de640 1 -- 172.21.15.45:0/2390102307 learned_addr learned my addr 172.21.15.45:0/2390102307 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac5edf640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 0x7fbac019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac4edd640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 0x7fbac0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 -- 172.21.15.45:0/2390102307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 msgr2=0x7fbac019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 0x7fbac019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 -- 172.21.15.45:0/2390102307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 msgr2=0x7fbac0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 0x7fbac0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 -- 172.21.15.45:0/2390102307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac019f710 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac5edf640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 0x7fbac019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac4edd640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 0x7fbac0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac56de640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9f0 tx=0x7fbabc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00cd80 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc005500 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac019fa00 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc010690 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.657+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac019ff00 con 0x7fbac0102280 2024-04-25T00:21:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.658+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00fd700 con 0x7fbac0102280 2024-04-25T00:21:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.659+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbabc0046c0 con 0x7fbac0102280 2024-04-25T00:21:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.660+0000 7fbab67fc640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 0x7fbaac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.660+0000 7fbac4edd640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 0x7fbaac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.660+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fbabc0cfad0 con 0x7fbac0102280 2024-04-25T00:21:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.660+0000 7fbac4edd640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 0x7fbaac0a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbab002a930 tx=0x7fbab0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.664+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbabc09c530 con 0x7fbac0102280 2024-04-25T00:21:18.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.849+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbac0062340 con 0x7fbac0102280 2024-04-25T00:21:18.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.849+0000 7fbab67fc640 1 -- 172.21.15.45:0/2390102307 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fbabc09be30 con 0x7fbac0102280 2024-04-25T00:21:18.850 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 msgr2=0x7fbaac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 0x7fbaac0a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbab002a930 tx=0x7fbab0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 msgr2=0x7fbac0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9f0 tx=0x7fbabc00edf0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 shutdown_connections 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbaac0a2c20 0x7fbaac0a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac01074e0 0x7fbac019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac0104bb0 0x7fbac0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 --2- 172.21.15.45:0/2390102307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbac0102280 0x7fbac0197af0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 >> 172.21.15.45:0/2390102307 conn(0x7fbac00fbc10 msgr2=0x7fbac0102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 shutdown_connections 2024-04-25T00:21:18.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.851+0000 7fbac7942640 1 -- 172.21.15.45:0/2390102307 wait complete. 2024-04-25T00:21:18.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 -- 172.21.15.45:0/2125863827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d8102250 msgr2=0x7f33d8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 --2- 172.21.15.45:0/2125863827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d8102250 0x7f33d8108ae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009ca0 tx=0x7f33d002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 -- 172.21.15.45:0/2125863827 shutdown_connections 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 --2- 172.21.15.45:0/2125863827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d8102250 0x7f33d8108ae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 --2- 172.21.15.45:0/2125863827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 0x7f33d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 --2- 172.21.15.45:0/2125863827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d81006a0 0x7f33d8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 -- 172.21.15.45:0/2125863827 >> 172.21.15.45:0/2125863827 conn(0x7f33d80fbc30 msgr2=0x7f33d80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 -- 172.21.15.45:0/2125863827 shutdown_connections 2024-04-25T00:21:18.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.968+0000 7f33ddff9640 1 -- 172.21.15.45:0/2125863827 wait complete. 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 Processor -- start 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 -- start start 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 0x7f33d8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d8102250 0x7f33d819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33d810d4a0 con 0x7f33d81006a0 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f33d810d320 con 0x7f33d8101890 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33ddff9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33d810d620 con 0x7f33d8102250 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42834/0 (socket says 172.21.15.45:42834) 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33d77fe640 1 -- 172.21.15.45:0/3074961943 learned_addr learned my addr 172.21.15.45:0/3074961943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.969+0000 7f33d7fff640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d8102250 0x7f33d819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d6ffd640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 0x7f33d8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 -- 172.21.15.45:0/3074961943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d8102250 msgr2=0x7f33d819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d8102250 0x7f33d819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 -- 172.21.15.45:0/3074961943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 msgr2=0x7f33d8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 0x7f33d8198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 -- 172.21.15.45:0/3074961943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d819f6e0 con 0x7f33d81006a0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d77fe640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d950 tx=0x7f33c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8017070 con 0x7f33d81006a0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c8009c30 con 0x7f33d81006a0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d819f9d0 con 0x7f33d81006a0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c800f600 con 0x7f33d81006a0 2024-04-25T00:21:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.970+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d819fed0 con 0x7f33d81006a0 2024-04-25T00:21:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.971+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d8107960 con 0x7f33d81006a0 2024-04-25T00:21:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.972+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f33c801a070 con 0x7f33d81006a0 2024-04-25T00:21:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.973+0000 7f33d4ff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 0x7f33c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.973+0000 7f33d6ffd640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 0x7f33c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:18.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.973+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f33c80cefe0 con 0x7f33d81006a0 2024-04-25T00:21:18.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.973+0000 7f33d6ffd640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 0x7f33c00a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f33d8198fd0 tx=0x7f33cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:18.977+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33c8009de0 con 0x7f33d81006a0 2024-04-25T00:21:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:18 smithi045 ceph-mon[29194]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1776368244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1720648587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2390102307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.161+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33d802d170 con 0x7f33d81006a0 2024-04-25T00:21:19.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.161+0000 7f33d4ff9640 1 -- 172.21.15.45:0/3074961943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f33d802d170 con 0x7f33d81006a0 2024-04-25T00:21:19.162 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 msgr2=0x7f33c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 0x7f33c00a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f33d8198fd0 tx=0x7f33cc005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 msgr2=0x7f33d8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f33c800d950 tx=0x7f33c800de10 comp rx=0 tx=0).stop 2024-04-25T00:21:19.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 shutdown_connections 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f33c00a2c20 0x7f33c00a50e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33d8102250 0x7f33d819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d8101890 0x7f33d8198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 --2- 172.21.15.45:0/3074961943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81006a0 0x7f33d8197ac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 >> 172.21.15.45:0/3074961943 conn(0x7f33d80fbc30 msgr2=0x7f33d8104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.163+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 shutdown_connections 2024-04-25T00:21:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.164+0000 7f33ddff9640 1 -- 172.21.15.45:0/3074961943 wait complete. 2024-04-25T00:21:19.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 -- 172.21.15.45:0/3945093043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294107480 msgr2=0x7f12941098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 --2- 172.21.15.45:0/3945093043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294107480 0x7f12941098d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1284009ca0 tx=0x7f128402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 -- 172.21.15.45:0/3945093043 shutdown_connections 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 --2- 172.21.15.45:0/3945093043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294107480 0x7f12941098d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 --2- 172.21.15.45:0/3945093043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294104b50 0x7f1294106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 --2- 172.21.15.45:0/3945093043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294102220 0x7f1294104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 -- 172.21.15.45:0/3945093043 >> 172.21.15.45:0/3945093043 conn(0x7f12940fbc30 msgr2=0x7f12940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 -- 172.21.15.45:0/3945093043 shutdown_connections 2024-04-25T00:21:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.280+0000 7f129b631640 1 -- 172.21.15.45:0/3945093043 wait complete. 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 Processor -- start 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 -- start start 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294104b50 0x7f1294197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294107480 0x7f129419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f129410d440 con 0x7f1294102220 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f129410d2c0 con 0x7f1294107480 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f129b631640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f129410d5c0 con 0x7f1294104b50 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42846/0 (socket says 172.21.15.45:42846) 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 -- 172.21.15.45:0/1242497763 learned_addr learned my addr 172.21.15.45:0/1242497763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f1298bcc640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294104b50 0x7f1294197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 -- 172.21.15.45:0/1242497763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294104b50 msgr2=0x7f1294197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294104b50 0x7f1294197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f1299bce640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294107480 0x7f129419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 -- 172.21.15.45:0/1242497763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294107480 msgr2=0x7f129419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294107480 0x7f129419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.281+0000 7f12993cd640 1 -- 172.21.15.45:0/1242497763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129419f6c0 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f12993cd640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d770 tx=0x7f128c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c017070 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129419f9b0 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f128c005500 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c004e00 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.282+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129419feb0 con 0x7f1294102220 2024-04-25T00:21:19.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.283+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12940fd720 con 0x7f1294102220 2024-04-25T00:21:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.284+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f128c004fa0 con 0x7f1294102220 2024-04-25T00:21:19.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.285+0000 7f12927fc640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 0x7f127c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.285+0000 7f1298bcc640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 0x7f127c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.285+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f128c0cf320 con 0x7f1294102220 2024-04-25T00:21:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.285+0000 7f1298bcc640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 0x7f127c0a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1294198fb0 tx=0x7f128800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.288+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f128c09be80 con 0x7f1294102220 2024-04-25T00:21:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:18 smithi122 ceph-mon[32959]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:18 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1776368244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:18 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1720648587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:18 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2390102307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:18 smithi071 ceph-mon[32667]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:18 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1776368244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:18 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1720648587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:18 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2390102307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:19.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.471+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f129402d320 con 0x7f1294102220 2024-04-25T00:21:19.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.471+0000 7f12927fc640 1 -- 172.21.15.45:0/1242497763 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f128c09b780 con 0x7f1294102220 2024-04-25T00:21:19.472 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 msgr2=0x7f127c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 0x7f127c0a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1294198fb0 tx=0x7f128800b410 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 msgr2=0x7f1294197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d770 tx=0x7f128c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 shutdown_connections 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f127c0a2c20 0x7f127c0a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1294107480 0x7f129419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1294104b50 0x7f1294197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 --2- 172.21.15.45:0/1242497763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294102220 0x7f1294197aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 >> 172.21.15.45:0/1242497763 conn(0x7f12940fbc30 msgr2=0x7f1294102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 shutdown_connections 2024-04-25T00:21:19.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.473+0000 7f129b631640 1 -- 172.21.15.45:0/1242497763 wait complete. 2024-04-25T00:21:19.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 -- 172.21.15.45:0/2825078188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41022b0 msgr2=0x7f34e4108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 --2- 172.21.15.45:0/2825078188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41022b0 0x7f34e4108b40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009ca0 tx=0x7f34dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 -- 172.21.15.45:0/2825078188 shutdown_connections 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 --2- 172.21.15.45:0/2825078188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41022b0 0x7f34e4108b40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 --2- 172.21.15.45:0/2825078188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41018f0 0x7f34e4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 --2- 172.21.15.45:0/2825078188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 0x7f34e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 -- 172.21.15.45:0/2825078188 >> 172.21.15.45:0/2825078188 conn(0x7f34e40fbc90 msgr2=0x7f34e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 -- 172.21.15.45:0/2825078188 shutdown_connections 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.587+0000 7f34ec61c640 1 -- 172.21.15.45:0/2825078188 wait complete. 2024-04-25T00:21:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 Processor -- start 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 -- start start 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 0x7f34e4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41022b0 0x7f34e419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410d420 con 0x7f34e41018f0 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410d2a0 con 0x7f34e4100700 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ec61c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34e410d5a0 con 0x7f34e41022b0 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34e9bb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34e9bb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42856/0 (socket says 172.21.15.45:42856) 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34e9bb7640 1 -- 172.21.15.45:0/436121042 learned_addr learned my addr 172.21.15.45:0/436121042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34eabb9640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41022b0 0x7f34e419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.588+0000 7f34ea3b8640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 0x7f34e4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 -- 172.21.15.45:0/436121042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41022b0 msgr2=0x7f34e419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41022b0 0x7f34e419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 -- 172.21.15.45:0/436121042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 msgr2=0x7f34e4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 0x7f34e4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 -- 172.21.15.45:0/436121042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e419f550 con 0x7f34e41018f0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34e9bb7640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c920 tx=0x7f34d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4007c70 con 0x7f34e41018f0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e419f840 con 0x7f34e41018f0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4007e10 con 0x7f34e41018f0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4018890 con 0x7f34e41018f0 2024-04-25T00:21:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.589+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e419fd90 con 0x7f34e41018f0 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.590+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e4107950 con 0x7f34e41018f0 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.591+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f34d4018a70 con 0x7f34e41018f0 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.592+0000 7f34db7fe640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 0x7f34d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.592+0000 7f34ea3b8640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 0x7f34d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.592+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f34d40cf5b0 con 0x7f34e41018f0 2024-04-25T00:21:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.592+0000 7f34ea3b8640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 0x7f34d00a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34e4101750 tx=0x7f34e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.596+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34d4014030 con 0x7f34e41018f0 2024-04-25T00:21:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.817+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34e402d170 con 0x7f34e41018f0 2024-04-25T00:21:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.818+0000 7f34db7fe640 1 -- 172.21.15.45:0/436121042 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7f34d409c070 con 0x7f34e41018f0 2024-04-25T00:21:19.819 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:19.819 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:21:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 msgr2=0x7f34d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 0x7f34d00a50e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34e4101750 tx=0x7f34e0005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 msgr2=0x7f34e4197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c920 tx=0x7f34d400cde0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 shutdown_connections 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f34d00a2c20 0x7f34d00a50e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34e41022b0 0x7f34e419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34e41018f0 0x7f34e4197e70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 --2- 172.21.15.45:0/436121042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100700 0x7f34e4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 >> 172.21.15.45:0/436121042 conn(0x7f34e40fbc90 msgr2=0x7f34e4104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.820+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 shutdown_connections 2024-04-25T00:21:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.821+0000 7f34ec61c640 1 -- 172.21.15.45:0/436121042 wait complete. 2024-04-25T00:21:19.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:19.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 -- 172.21.15.45:0/2441051356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 msgr2=0x7fc5dc108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/2441051356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc108ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009ca0 tx=0x7fc5d402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 -- 172.21.15.45:0/2441051356 shutdown_connections 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/2441051356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc108ad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/2441051356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc101880 0x7fc5dc101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/2441051356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc100690 0x7fc5dc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 -- 172.21.15.45:0/2441051356 >> 172.21.15.45:0/2441051356 conn(0x7fc5dc0fbc20 msgr2=0x7fc5dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 -- 172.21.15.45:0/2441051356 shutdown_connections 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 -- 172.21.15.45:0/2441051356 wait complete. 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.935+0000 7fc5e2af7640 1 Processor -- start 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 -- start start 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 0x7fc5dc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 0x7fc5dc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10c2f0 con 0x7fc5dc102240 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10c170 con 0x7fc5dc100690 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e2af7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10c470 con 0x7fc5dc101880 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42874/0 (socket says 172.21.15.45:42874) 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 -- 172.21.15.45:0/23456799 learned_addr learned my addr 172.21.15.45:0/23456799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e0893640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 0x7fc5dc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5dbfff640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 0x7fc5dc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 -- 172.21.15.45:0/23456799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 msgr2=0x7fc5dc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 0x7fc5dc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 -- 172.21.15.45:0/23456799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 msgr2=0x7fc5dc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 0x7fc5dc197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 -- 172.21.15.45:0/23456799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc19f680 con 0x7fc5dc102240 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e0893640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 0x7fc5dc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5dbfff640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 0x7fc5dc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5e1094640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005e30 tx=0x7fc5d4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.936+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4003c30 con 0x7fc5dc102240 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.937+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc19f910 con 0x7fc5dc102240 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.937+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d4003dd0 con 0x7fc5dc102240 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.937+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d402fde0 con 0x7fc5dc102240 2024-04-25T00:21:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.937+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc19feb0 con 0x7fc5dc102240 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.938+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc1a0110 con 0x7fc5dc102240 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.939+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5d4004050 con 0x7fc5dc102240 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.940+0000 7fc5d9ffb640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 0x7fc5cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.940+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc5d40f0a50 con 0x7fc5dc102240 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.940+0000 7fc5e0893640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 0x7fc5cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:19.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.940+0000 7fc5e0893640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 0x7fc5cc0a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc1016e0 tx=0x7fc5d0005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:19.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:19.943+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5d40bd620 con 0x7fc5dc102240 2024-04-25T00:21:20.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.130+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5dc1a02e0 con 0x7fc5dc102240 2024-04-25T00:21:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.132+0000 7fc5d9ffb640 1 -- 172.21.15.45:0/23456799 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24616 (secure 0 0 0) 0x7fc5d40bcf20 con 0x7fc5dc102240 2024-04-25T00:21:20.133 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:20.133 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:20:27.035516+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 msgr2=0x7fc5cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 0x7fc5cc0a50e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc1016e0 tx=0x7fc5d0005e70 comp rx=0 tx=0).stop 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 msgr2=0x7fc5dc19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005e30 tx=0x7fc5d4031ee0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 shutdown_connections 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5cc0a2c20 0x7fc5cc0a50e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5dc102240 0x7fc5dc19f010 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc101880 0x7fc5dc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 --2- 172.21.15.45:0/23456799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5dc100690 0x7fc5dc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 >> 172.21.15.45:0/23456799 conn(0x7fc5dc0fbc20 msgr2=0x7fc5dc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 shutdown_connections 2024-04-25T00:21:20.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.134+0000 7fc5e2af7640 1 -- 172.21.15.45:0/23456799 wait complete. 2024-04-25T00:21:20.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:19 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3074961943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:19 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1242497763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:19 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/436121042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:20.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:20.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:20.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/1250493028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc41017a0 msgr2=0x7fbdc4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/1250493028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc41017a0 0x7fbdc4101bc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/1250493028 shutdown_connections 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/1250493028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/1250493028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4102990 0x7fbdc4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/1250493028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc41017a0 0x7fbdc4101bc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/1250493028 >> 172.21.15.45:0/1250493028 conn(0x7fbdc40fcd10 msgr2=0x7fbdc40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/1250493028 shutdown_connections 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/1250493028 wait complete. 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 Processor -- start 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.252+0000 7fbdc8e4d640 1 -- start start 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc41017a0 0x7fbdc4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410d3f0 con 0x7fbdc4102990 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410d270 con 0x7fbdc41017a0 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc8e4d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410d570 con 0x7fbdc4103350 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42892/0 (socket says 172.21.15.45:42892) 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 -- 172.21.15.45:0/3894186857 learned_addr learned my addr 172.21.15.45:0/3894186857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc259c640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc41017a0 0x7fbdc4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc2d9d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 -- 172.21.15.45:0/3894186857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 msgr2=0x7fbdc419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 -- 172.21.15.45:0/3894186857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc41017a0 msgr2=0x7fbdc4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc41017a0 0x7fbdc4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 -- 172.21.15.45:0/3894186857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc419f6c0 con 0x7fbdc4102990 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc2d9d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdc1d9b640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e9f0 tx=0x7fbdb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.253+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800cd80 con 0x7fbdc4102990 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.254+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8010910 con 0x7fbdc4102990 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.254+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc419f9b0 con 0x7fbdc4102990 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.254+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80188f0 con 0x7fbdc4102990 2024-04-25T00:21:20.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.254+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc419ff00 con 0x7fbdc4102990 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.254+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc4108b30 con 0x7fbdc4102990 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.256+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbdb8010ac0 con 0x7fbdc4102990 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.256+0000 7fbdb37fe640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 0x7fbdac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.256+0000 7fbdc259c640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 0x7fbdac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.256+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fbdb80cf970 con 0x7fbdc4102990 2024-04-25T00:21:20.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.257+0000 7fbdc259c640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 0x7fbdac0a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4005ab0 tx=0x7fbdb4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.261+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbdb809c430 con 0x7fbdc4102990 2024-04-25T00:21:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:19 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3074961943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:19 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1242497763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:19 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/436121042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:20.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3074961943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1242497763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:20.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:19 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/436121042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.445+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbdc402d3c0 con 0x7fbdc4102990 2024-04-25T00:21:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.446+0000 7fbdb37fe640 1 -- 172.21.15.45:0/3894186857 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fbdb809bd30 con 0x7fbdc4102990 2024-04-25T00:21:20.447 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 msgr2=0x7fbdac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 0x7fbdac0a50e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4005ab0 tx=0x7fbdb4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 msgr2=0x7fbdc4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e9f0 tx=0x7fbdb800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 shutdown_connections 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbdac0a2c20 0x7fbdac0a50e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc4103350 0x7fbdc419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdc4102990 0x7fbdc4197fe0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 --2- 172.21.15.45:0/3894186857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc41017a0 0x7fbdc4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 >> 172.21.15.45:0/3894186857 conn(0x7fbdc40fcd10 msgr2=0x7fbdc4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 shutdown_connections 2024-04-25T00:21:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.448+0000 7fbdc8e4d640 1 -- 172.21.15.45:0/3894186857 wait complete. 2024-04-25T00:21:20.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:20.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 -- 172.21.15.45:0/2573302153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9080ffa80 msgr2=0x7fd9080ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 --2- 172.21.15.45:0/2573302153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9080ffa80 0x7fd9080ffea0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd904009cd0 tx=0x7fd90402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 -- 172.21.15.45:0/2573302153 shutdown_connections 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 --2- 172.21.15.45:0/2573302153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 0x7fd908109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 --2- 172.21.15.45:0/2573302153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd908102af0 0x7fd908102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 --2- 172.21.15.45:0/2573302153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9080ffa80 0x7fd9080ffea0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.564+0000 7fd91071d640 1 -- 172.21.15.45:0/2573302153 >> 172.21.15.45:0/2573302153 conn(0x7fd9080fb370 msgr2=0x7fd9080fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- 172.21.15.45:0/2573302153 shutdown_connections 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- 172.21.15.45:0/2573302153 wait complete. 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 Processor -- start 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- start start 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9080ffa80 0x7fd908102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 0x7fd908100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810d430 con 0x7fd908102af0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810d2b0 con 0x7fd9081034b0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd91071d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810d5b0 con 0x7fd9080ffa80 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd90dcb8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd90dcb8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42910/0 (socket says 172.21.15.45:42910) 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.565+0000 7fd90dcb8640 1 -- 172.21.15.45:0/191316417 learned_addr learned my addr 172.21.15.45:0/191316417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90ecba640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 0x7fd908100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90e4b9640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9080ffa80 0x7fd908102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 -- 172.21.15.45:0/191316417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9080ffa80 msgr2=0x7fd908102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9080ffa80 0x7fd908102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 -- 172.21.15.45:0/191316417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 msgr2=0x7fd908100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 0x7fd908100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 -- 172.21.15.45:0/191316417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908101870 con 0x7fd908102af0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd90dcb8640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800d770 tx=0x7fd8f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8017070 con 0x7fd908102af0 2024-04-25T00:21:20.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f8005500 con 0x7fd908102af0 2024-04-25T00:21:20.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8004e00 con 0x7fd908102af0 2024-04-25T00:21:20.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9081a1bd0 con 0x7fd908102af0 2024-04-25T00:21:20.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.566+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9081a20d0 con 0x7fd908102af0 2024-04-25T00:21:20.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.567+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd908108ca0 con 0x7fd908102af0 2024-04-25T00:21:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.568+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd8f8004fa0 con 0x7fd908102af0 2024-04-25T00:21:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.569+0000 7fd8ff7fe640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 0x7fd8f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.569+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd8f80cf2c0 con 0x7fd908102af0 2024-04-25T00:21:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.569+0000 7fd90e4b9640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 0x7fd8f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.570+0000 7fd90e4b9640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 0x7fd8f40a5160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd904009ca0 tx=0x7fd904009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.573+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd8f809b750 con 0x7fd908102af0 2024-04-25T00:21:20.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.755+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd90802d3c0 con 0x7fd908102af0 2024-04-25T00:21:20.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.756+0000 7fd8ff7fe640 1 -- 172.21.15.45:0/191316417 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fd8f809fa50 con 0x7fd908102af0 2024-04-25T00:21:20.756 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:20.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 msgr2=0x7fd8f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 0x7fd8f40a5160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd904009ca0 tx=0x7fd904009360 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 msgr2=0x7fd908100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800d770 tx=0x7fd8f800dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 shutdown_connections 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8f40a2ca0 0x7fd8f40a5160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9081034b0 0x7fd908100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd908102af0 0x7fd908100aa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 --2- 172.21.15.45:0/191316417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9080ffa80 0x7fd908102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 >> 172.21.15.45:0/191316417 conn(0x7fd9080fb370 msgr2=0x7fd908105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 shutdown_connections 2024-04-25T00:21:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.758+0000 7fd91071d640 1 -- 172.21.15.45:0/191316417 wait complete. 2024-04-25T00:21:20.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- 172.21.15.45:0/1979282342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 msgr2=0x7f4588100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- 172.21.15.45:0/1979282342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588100ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4584009900 tx=0x7f458402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- 172.21.15.45:0/1979282342 shutdown_connections 2024-04-25T00:21:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- 172.21.15.45:0/1979282342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 0x7f4588108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- 172.21.15.45:0/1979282342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 0x7f4588101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- 172.21.15.45:0/1979282342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588100ab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- 172.21.15.45:0/1979282342 >> 172.21.15.45:0/1979282342 conn(0x7f45880fbc00 msgr2=0x7f45880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- 172.21.15.45:0/1979282342 shutdown_connections 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- 172.21.15.45:0/1979282342 wait complete. 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 Processor -- start 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- start start 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 0x7f4588198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 0x7f458819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f458810c2e0 con 0x7f4588100690 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f458810c160 con 0x7f4588102240 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.874+0000 7f459039e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f458810c460 con 0x7f4588101880 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42932/0 (socket says 172.21.15.45:42932) 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 -- 172.21.15.45:0/2642777784 learned_addr learned my addr 172.21.15.45:0/2642777784 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e93b640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 0x7f458819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458d939640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 0x7f4588198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 -- 172.21.15.45:0/2642777784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 msgr2=0x7f4588198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 0x7f4588198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 -- 172.21.15.45:0/2642777784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 msgr2=0x7f458819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 0x7f458819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 -- 172.21.15.45:0/2642777784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458819f6e0 con 0x7f4588100690 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f458e13a640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4584031610 tx=0x7f45840043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458403f030 con 0x7f4588100690 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4584003ce0 con 0x7f4588100690 2024-04-25T00:21:20.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458819f970 con 0x7f4588100690 2024-04-25T00:21:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584035070 con 0x7f4588100690 2024-04-25T00:21:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.875+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458819fe60 con 0x7f4588100690 2024-04-25T00:21:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.876+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45881a0030 con 0x7f4588100690 2024-04-25T00:21:20.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.877+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4584042030 con 0x7f4588100690 2024-04-25T00:21:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.878+0000 7f457f7fe640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 0x7f45740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.878+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f45840efde0 con 0x7f4588100690 2024-04-25T00:21:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.878+0000 7f458d939640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 0x7f45740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.879+0000 7f458d939640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 0x7f45740a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4578005e10 tx=0x7f4578005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:20.882+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45840bc8a0 con 0x7f4588100690 2024-04-25T00:21:21.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.067+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45881a0200 con 0x7f4588100690 2024-04-25T00:21:21.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.067+0000 7f457f7fe640 1 -- 172.21.15.45:0/2642777784 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f45840bc1a0 con 0x7f4588100690 2024-04-25T00:21:21.068 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 msgr2=0x7f45740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 0x7f45740a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4578005e10 tx=0x7f4578005da0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 msgr2=0x7f4588197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4584031610 tx=0x7f45840043b0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 shutdown_connections 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f45740a2c20 0x7f45740a50e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4588102240 0x7f458819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588101880 0x7f4588198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 --2- 172.21.15.45:0/2642777784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4588100690 0x7f4588197b30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 >> 172.21.15.45:0/2642777784 conn(0x7f45880fbc00 msgr2=0x7f45880fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 shutdown_connections 2024-04-25T00:21:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.069+0000 7f459039e640 1 -- 172.21.15.45:0/2642777784 wait complete. 2024-04-25T00:21:21.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:20 smithi045 ceph-mon[29194]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:21.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:20 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/23456799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:21.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:20 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3894186857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:20 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/191316417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 -- 172.21.15.45:0/3102413448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47901006a0 msgr2=0x7f4790100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 --2- 172.21.15.45:0/3102413448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47901006a0 0x7f4790100ac0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4788009900 tx=0x7f478802a410 comp rx=0 tx=0).stop 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 -- 172.21.15.45:0/3102413448 shutdown_connections 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 --2- 172.21.15.45:0/3102413448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4790102250 0x7f4790108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 --2- 172.21.15.45:0/3102413448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 0x7f4790101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 --2- 172.21.15.45:0/3102413448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47901006a0 0x7f4790100ac0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.185+0000 7f479868c640 1 -- 172.21.15.45:0/3102413448 >> 172.21.15.45:0/3102413448 conn(0x7f47900fbc30 msgr2=0x7f47900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- 172.21.15.45:0/3102413448 shutdown_connections 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- 172.21.15.45:0/3102413448 wait complete. 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 Processor -- start 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- start start 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47901006a0 0x7f4790197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 0x7f4790198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f479010c2f0 con 0x7f4790102250 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f479010c170 con 0x7f4790101890 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f479868c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f479010c470 con 0x7f47901006a0 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.186+0000 7f4796c29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42944/0 (socket says 172.21.15.45:42944) 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 -- 172.21.15.45:0/631685250 learned_addr learned my addr 172.21.15.45:0/631685250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:21.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796428640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47901006a0 0x7f4790197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4795c27640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 0x7f4790198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 -- 172.21.15.45:0/631685250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47901006a0 msgr2=0x7f4790197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47901006a0 0x7f4790197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 -- 172.21.15.45:0/631685250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 msgr2=0x7f4790198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 0x7f4790198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 -- 172.21.15.45:0/631685250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479019f6e0 con 0x7f4790102250 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f4796c29640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f478000eaa0 tx=0x7f478000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478000cdb0 con 0x7f4790102250 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4780005020 con 0x7f4790102250 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47800116b0 con 0x7f4790102250 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479019f9d0 con 0x7f4790102250 2024-04-25T00:21:21.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.187+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479019ff20 con 0x7f4790102250 2024-04-25T00:21:21.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.188+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47901a0210 con 0x7f4790102250 2024-04-25T00:21:21.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.189+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4780005560 con 0x7f4790102250 2024-04-25T00:21:21.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.190+0000 7f478f7fe640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 0x7f47780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.190+0000 7f4796428640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 0x7f47780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.190+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f47800cf680 con 0x7f4790102250 2024-04-25T00:21:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.191+0000 7f4796428640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 0x7f47780a5160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4788005ab0 tx=0x7f4788000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.195+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4780016030 con 0x7f4790102250 2024-04-25T00:21:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:20 smithi122 ceph-mon[32959]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:20 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/23456799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:20 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3894186857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:20 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/191316417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:20 smithi071 ceph-mon[32667]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:21:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:20 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/23456799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:20 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3894186857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:20 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/191316417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.379+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47901a03e0 con 0x7f4790102250 2024-04-25T00:21:21.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.379+0000 7f478f7fe640 1 -- 172.21.15.45:0/631685250 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f478009baf0 con 0x7f4790102250 2024-04-25T00:21:21.380 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 msgr2=0x7f47780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 0x7f47780a5160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4788005ab0 tx=0x7f4788000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 msgr2=0x7f479019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f478000eaa0 tx=0x7f478000ef60 comp rx=0 tx=0).stop 2024-04-25T00:21:21.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 shutdown_connections 2024-04-25T00:21:21.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47780a2ca0 0x7f47780a5160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4790102250 0x7f479019f070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4790101890 0x7f4790198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 --2- 172.21.15.45:0/631685250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47901006a0 0x7f4790197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 >> 172.21.15.45:0/631685250 conn(0x7f47900fbc30 msgr2=0x7f4790069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 shutdown_connections 2024-04-25T00:21:21.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.381+0000 7f479868c640 1 -- 172.21.15.45:0/631685250 wait complete. 2024-04-25T00:21:21.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:21.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 -- 172.21.15.45:0/4007451712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f4101890 msgr2=0x7f82f4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/4007451712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f4101890 0x7f82f4101d10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009900 tx=0x7f82e402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 -- 172.21.15.45:0/4007451712 shutdown_connections 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/4007451712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f4102250 0x7f82f4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/4007451712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f4101890 0x7f82f4101d10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/4007451712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f41006a0 0x7f82f4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 -- 172.21.15.45:0/4007451712 >> 172.21.15.45:0/4007451712 conn(0x7f82f40fbc30 msgr2=0x7f82f40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 -- 172.21.15.45:0/4007451712 shutdown_connections 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.496+0000 7f82fb2cd640 1 -- 172.21.15.45:0/4007451712 wait complete. 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 Processor -- start 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 -- start start 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f4101890 0x7f82f4109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f4102250 0x7f82f4109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410d440 con 0x7f82f41006a0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410d2c0 con 0x7f82f4101890 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82fb2cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410d5c0 con 0x7f82f4102250 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42966/0 (socket says 172.21.15.45:42966) 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 -- 172.21.15.45:0/3206548662 learned_addr learned my addr 172.21.15.45:0/3206548662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f986a640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f4102250 0x7f82f4109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 -- 172.21.15.45:0/3206548662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f4102250 msgr2=0x7f82f4109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f4102250 0x7f82f4109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 -- 172.21.15.45:0/3206548662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f4101890 msgr2=0x7f82f4109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f4101890 0x7f82f4109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 -- 172.21.15.45:0/3206548662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f410a430 con 0x7f82f41006a0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.497+0000 7f82f9069640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e9c0 tx=0x7f82f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f000cd80 con 0x7f82f41006a0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f00035f0 con 0x7f82f41006a0 2024-04-25T00:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41a1c10 con 0x7f82f41006a0 2024-04-25T00:21:21.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0010690 con 0x7f82f41006a0 2024-04-25T00:21:21.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a2110 con 0x7f82f41006a0 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.498+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4107a60 con 0x7f82f41006a0 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.500+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f82f00037a0 con 0x7f82f41006a0 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.500+0000 7f82ea7fc640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 0x7f82e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.500+0000 7f82f8868640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 0x7f82e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.500+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f82f00cfa50 con 0x7f82f41006a0 2024-04-25T00:21:21.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.501+0000 7f82f8868640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 0x7f82e00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f82e40096c0 tx=0x7f82e4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.505+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82f009c510 con 0x7f82f41006a0 2024-04-25T00:21:21.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.730+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f402d170 con 0x7f82f41006a0 2024-04-25T00:21:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.731+0000 7f82ea7fc640 1 -- 172.21.15.45:0/3206548662 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7f82f009be10 con 0x7f82f41006a0 2024-04-25T00:21:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:21:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 msgr2=0x7f82e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 0x7f82e00a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f82e40096c0 tx=0x7f82e4031400 comp rx=0 tx=0).stop 2024-04-25T00:21:21.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 msgr2=0x7f82f410b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e9c0 tx=0x7f82f000ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 shutdown_connections 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f82e00a2c20 0x7f82e00a50e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82f4102250 0x7f82f4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f4101890 0x7f82f4109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 --2- 172.21.15.45:0/3206548662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82f41006a0 0x7f82f410b020 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 >> 172.21.15.45:0/3206548662 conn(0x7f82f40fbc30 msgr2=0x7f82f4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 shutdown_connections 2024-04-25T00:21:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.733+0000 7f82fb2cd640 1 -- 172.21.15.45:0/3206548662 wait complete. 2024-04-25T00:21:21.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.846+0000 7f8d66a38640 1 -- 172.21.15.45:0/830159783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60101890 msgr2=0x7f8d60101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.846+0000 7f8d66a38640 1 --2- 172.21.15.45:0/830159783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60101890 0x7f8d60101d10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d5402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 -- 172.21.15.45:0/830159783 shutdown_connections 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 --2- 172.21.15.45:0/830159783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60102250 0x7f8d60108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 --2- 172.21.15.45:0/830159783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60101890 0x7f8d60101d10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 --2- 172.21.15.45:0/830159783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 0x7f8d60100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 -- 172.21.15.45:0/830159783 >> 172.21.15.45:0/830159783 conn(0x7f8d600fbc30 msgr2=0x7f8d600fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 -- 172.21.15.45:0/830159783 shutdown_connections 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 -- 172.21.15.45:0/830159783 wait complete. 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 Processor -- start 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 -- start start 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 0x7f8d6006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.847+0000 7f8d66a38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60101890 0x7f8d6019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010c300 con 0x7f8d60102250 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010c180 con 0x7f8d60101890 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6010c480 con 0x7f8d601006a0 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42988/0 (socket says 172.21.15.45:42988) 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 -- 172.21.15.45:0/2233936838 learned_addr learned my addr 172.21.15.45:0/2233936838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d5f7fe640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60101890 0x7f8d6019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d5ffff640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 0x7f8d6006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 -- 172.21.15.45:0/2233936838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 msgr2=0x7f8d6006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 0x7f8d6006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 -- 172.21.15.45:0/2233936838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60101890 msgr2=0x7f8d6019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60101890 0x7f8d6019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:21.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 -- 172.21.15.45:0/2233936838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6019f6d0 con 0x7f8d60102250 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d64fd5640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009fd0 tx=0x7f8d5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58017070 con 0x7f8d60102250 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6019f9c0 con 0x7f8d60102250 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d58003e50 con 0x7f8d60102250 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580106c0 con 0x7f8d60102250 2024-04-25T00:21:21.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.848+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6019fe90 con 0x7f8d60102250 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.849+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d600689b0 con 0x7f8d60102250 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.851+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8d58003970 con 0x7f8d60102250 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.851+0000 7f8d5d7fa640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 0x7f8d480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.851+0000 7f8d5ffff640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 0x7f8d480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.851+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d580cf2c0 con 0x7f8d60102250 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.852+0000 7f8d5ffff640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 0x7f8d480a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005db0 tx=0x7f8d50005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:21.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:21.856+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d5809bd80 con 0x7f8d60102250 2024-04-25T00:21:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.045+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d60062340 con 0x7f8d60102250 2024-04-25T00:21:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.046+0000 7f8d5d7fa640 1 -- 172.21.15.45:0/2233936838 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24616 (secure 0 0 0) 0x7f8d5809b680 con 0x7f8d60102250 2024-04-25T00:21:22.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:22.047 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:20:27.035516+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 msgr2=0x7f8d480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 0x7f8d480a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005db0 tx=0x7f8d50005d40 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 msgr2=0x7f8d6019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009fd0 tx=0x7f8d5800b260 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 shutdown_connections 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8d480a2c20 0x7f8d480a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d60102250 0x7f8d6019ee10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d60101890 0x7f8d6019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 --2- 172.21.15.45:0/2233936838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d601006a0 0x7f8d6006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 >> 172.21.15.45:0/2233936838 conn(0x7f8d600fbc30 msgr2=0x7f8d60104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 shutdown_connections 2024-04-25T00:21:22.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.048+0000 7f8d66a38640 1 -- 172.21.15.45:0/2233936838 wait complete. 2024-04-25T00:21:22.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2642777784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/631685250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:21 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3206548662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:22.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- 172.21.15.45:0/1286171432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 msgr2=0x7f5ca4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/1286171432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4101d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- 172.21.15.45:0/1286171432 shutdown_connections 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/1286171432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca4102250 0x7f5ca4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/1286171432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4101d10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/1286171432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca41006a0 0x7f5ca4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- 172.21.15.45:0/1286171432 >> 172.21.15.45:0/1286171432 conn(0x7f5ca40fbc30 msgr2=0x7f5ca40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- 172.21.15.45:0/1286171432 shutdown_connections 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- 172.21.15.45:0/1286171432 wait complete. 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 Processor -- start 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 -- start start 2024-04-25T00:21:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca41006a0 0x7f5ca4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.162+0000 7f5cab6d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca4102250 0x7f5ca419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5cab6d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d440 con 0x7f5ca4101890 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5cab6d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d2c0 con 0x7f5ca4102250 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5cab6d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d5c0 con 0x7f5ca41006a0 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43018/0 (socket says 172.21.15.45:43018) 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 -- 172.21.15.45:0/3261602359 learned_addr learned my addr 172.21.15.45:0/3261602359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca9c71640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca4102250 0x7f5ca419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca9470640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca41006a0 0x7f5ca4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 -- 172.21.15.45:0/3261602359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca41006a0 msgr2=0x7f5ca4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca41006a0 0x7f5ca4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 -- 172.21.15.45:0/3261602359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca4102250 msgr2=0x7f5ca419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca4102250 0x7f5ca419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 -- 172.21.15.45:0/3261602359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca419f6b0 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5ca8c6f640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009230 tx=0x7f5c94004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403f030 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c94003ce0 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca419f940 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94035070 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.163+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca419fe30 con 0x7f5ca4101890 2024-04-25T00:21:22.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.164+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca41079c0 con 0x7f5ca4101890 2024-04-25T00:21:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.166+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5c94042030 con 0x7f5ca4101890 2024-04-25T00:21:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.166+0000 7f5c9a7fc640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 0x7f5c900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.166+0000 7f5ca9470640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 0x7f5c900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.166+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5c940efee0 con 0x7f5ca4101890 2024-04-25T00:21:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.167+0000 7f5ca9470640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 0x7f5c900a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005fd0 tx=0x7f5ca0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.170+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c940bc9a0 con 0x7f5ca4101890 2024-04-25T00:21:22.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2642777784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/631685250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:21 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3206548662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:22.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2642777784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/631685250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:22.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:21 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3206548662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.391+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ca402d170 con 0x7f5ca4101890 2024-04-25T00:21:22.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.392+0000 7f5c9a7fc640 1 -- 172.21.15.45:0/3261602359 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7f5c940bc2a0 con 0x7f5ca4101890 2024-04-25T00:21:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:22.393 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:21:22.393 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 msgr2=0x7f5c900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 0x7f5c900a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005fd0 tx=0x7f5ca0005850 comp rx=0 tx=0).stop 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 msgr2=0x7f5ca4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009230 tx=0x7f5c94004360 comp rx=0 tx=0).stop 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 shutdown_connections 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5c900a2c20 0x7f5c900a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ca4102250 0x7f5ca419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca4101890 0x7f5ca4197fd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 --2- 172.21.15.45:0/3261602359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ca41006a0 0x7f5ca4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 >> 172.21.15.45:0/3261602359 conn(0x7f5ca40fbc30 msgr2=0x7f5ca4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 shutdown_connections 2024-04-25T00:21:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.394+0000 7f5cab6d4640 1 -- 172.21.15.45:0/3261602359 wait complete. 2024-04-25T00:21:22.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:19:33.221862+0000', 'modified': '2024-04-25T00:20:01.835169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24493, 'mds_2': 24478, 'mds_3': 24467, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/2848612989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 2848612989}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 2848612989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6841/3316648689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 3316648689}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 3316648689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/1511686553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 1511686553}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 1511686553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6833/3066271880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 3066271880}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 3066271880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/741071549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 741071549}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 741071549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.122:6835/1025393130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 1025393130}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 1025393130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.71:6837/2635562743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 2635562743}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 2635562743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.122:6837/3810708134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 3810708134}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 3810708134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.122:6839/2298542043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 2298542043}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 2298542043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/102345855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 102345855}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 102345855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24467, 24460, 24478, 24500, 24493]} 2024-04-25T00:21:22.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T00:21:22.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T00:21:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.509+0000 7f539587d640 1 -- 172.21.15.45:0/3278892936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390103340 msgr2=0x7f5390109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.509+0000 7f539587d640 1 --2- 172.21.15.45:0/3278892936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390103340 0x7f5390109bd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5388009900 tx=0x7f538802a410 comp rx=0 tx=0).stop 2024-04-25T00:21:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 -- 172.21.15.45:0/3278892936 shutdown_connections 2024-04-25T00:21:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 --2- 172.21.15.45:0/3278892936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390103340 0x7f5390109bd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 --2- 172.21.15.45:0/3278892936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390102980 0x7f5390102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 --2- 172.21.15.45:0/3278892936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f5390101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 -- 172.21.15.45:0/3278892936 >> 172.21.15.45:0/3278892936 conn(0x7f53900fcd20 msgr2=0x7f53900ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 -- 172.21.15.45:0/3278892936 shutdown_connections 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 -- 172.21.15.45:0/3278892936 wait complete. 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 Processor -- start 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 -- start start 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f53900754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.510+0000 7f539587d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390103340 0x7f5390074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f539010d410 con 0x7f5390102980 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f539010d290 con 0x7f5390103340 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f539010d590 con 0x7f5390101790 2024-04-25T00:21:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43034/0 (socket says 172.21.15.45:43034) 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 -- 172.21.15.45:0/2849447035 learned_addr learned my addr 172.21.15.45:0/2849447035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390103340 0x7f5390074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538effd640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f53900754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 -- 172.21.15.45:0/2849447035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 msgr2=0x7f53900754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f53900754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 -- 172.21.15.45:0/2849447035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390103340 msgr2=0x7f5390074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390103340 0x7f5390074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 -- 172.21.15.45:0/2849447035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5390074900 con 0x7f5390102980 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538effd640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f53900754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f537800d950 tx=0x7f537800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539487b640 1 -- 172.21.15.45:0/2849447035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378017030 con 0x7f5390102980 2024-04-25T00:21:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539487b640 1 -- 172.21.15.45:0/2849447035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5378004020 con 0x7f5390102980 2024-04-25T00:21:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a1b70 con 0x7f5390102980 2024-04-25T00:21:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539487b640 1 -- 172.21.15.45:0/2849447035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780047a0 con 0x7f5390102980 2024-04-25T00:21:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.511+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a20c0 con 0x7f5390102980 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539487b640 1 -- 172.21.15.45:0/2849447035 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53780041d0 con 0x7f5390102980 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 Processor -- start 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 -- start start 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539487b640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 0x7f537c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b00b047f00 con 0x55b00b0e4e30 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b00aff0750 con 0x55b00b0e4440 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f539858e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b00b043ec0 con 0x55b00b0e9fa0 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f538effd640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 0x7f537c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43040/0 (socket says 172.21.15.45:43040) 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f538f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.514+0000 7f538e7fc640 1 -- 172.21.15.45:0/2803548685 learned_addr learned my addr 172.21.15.45:0/2803548685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f539487b640 1 -- 172.21.15.45:0/2849447035 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f53780cef80 con 0x7f5390102980 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 -- 172.21.15.45:0/2803548685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 msgr2=0x55b00b0ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 -- 172.21.15.45:0/2803548685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 msgr2=0x55b00b0e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 -- 172.21.15.45:0/2803548685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b00ad5c2d0 con 0x55b00b0e4e30 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538effd640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 0x7f537c0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53901027e0 tx=0x7f5380009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5378097d10 tx=0x7f53780051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42577c6771f3f56b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f5385ffb640 1 -- 172.21.15.45:0/2803548685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537809aca0 con 0x55b00b0e4e30 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f5385ffb640 1 -- 172.21.15.45:0/2803548685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537809ba40 con 0x55b00b0e4e30 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.515+0000 7f5385ffb640 1 -- 172.21.15.45:0/2803548685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780666b0 con 0x55b00b0e4e30 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- 172.21.15.45:0/2803548685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 msgr2=0x55b00b0e9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5378097d10 tx=0x7f53780051b0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- 172.21.15.45:0/2803548685 shutdown_connections 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0e9930 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 --2- 172.21.15.45:0/2803548685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- 172.21.15.45:0/2803548685 >> 172.21.15.45:0/2803548685 conn(0x55b00b0dfab0 msgr2=0x55b00b0dfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- 172.21.15.45:0/2803548685 shutdown_connections 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- 172.21.15.45:0/2803548685 wait complete. 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 -- start start 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.516+0000 7f539858e740 1 Processor -- start 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e4e30 0x55b00b0eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0ece90 con 0x55b00b0e9fa0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b00b003cf0 con 0x55b00b0e4440 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0ed010 con 0x55b00b0e4e30 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 10 client.0 _ll_register_callbacks cb 0x55b00b0036e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43048/0 (socket says 172.21.15.45:43048) 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 -- 172.21.15.45:0/2998886675 learned_addr learned my addr 172.21.15.45:0/2998886675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e4e30 0x55b00b0eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538effd640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 -- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e4e30 msgr2=0x55b00b0eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e4e30 0x55b00b0eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 -- 172.21.15.45:0/2998886675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 msgr2=0x55b00b0ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 -- 172.21.15.45:0/2998886675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00ae266b0 con 0x55b00b0e9fa0 2024-04-25T00:21:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5388004750 tx=0x7f53880318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2145385009d773b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f536effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388044070 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5388004450 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.517+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537809ba40 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5388051050 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b00b0f8dd0 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537809ba40 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5388004be0 con 0x55b00b0e9fa0 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 10 client.15234 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 20 client.15234 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b0fb4c0 0x55b00b0fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 4 client.15234 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b00b055440 con 0x55b00b0fb4c0 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.518+0000 7f538effd640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b0fb4c0 0x55b00b0fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.519+0000 7f538effd640 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b0fb4c0 0x55b00b0fc930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=372c181303116a62 server_cookie=989fefbaf5ea6527 in_seq=0 out_seq=0 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.519+0000 7f536effd640 10 client.15234 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f536effd640 1 -- 172.21.15.45:0/2998886675 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b00b055440 con 0x55b00b0fb4c0 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f536effd640 10 client.15234 handle_command_reply: tid=0 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 client.15234 shutdown 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 msgr2=0x55b00b0f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5388004750 tx=0x7f53880318e0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 shutdown_connections 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b0fb4c0 0x55b00b0fc930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e9fa0 0x55b00b0f5d80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e4e30 0x55b00b0eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 --2- 172.21.15.45:0/2998886675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 >> 172.21.15.45:0/2998886675 conn(0x55b00b0dfab0 msgr2=0x55b00b0deae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f536effd640 0 client.15234 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 shutdown_connections 2024-04-25T00:21:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 1 -- 172.21.15.45:0/2998886675 wait complete. 2024-04-25T00:21:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.520+0000 7f539858e740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T00:21:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 Processor -- start 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 -- start start 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fac70 con 0x55b00b0e4e30 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fb340 con 0x55b00b0e4440 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f539858e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fba10 con 0x55b00b0e9fa0 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f538e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43064/0 (socket says 172.21.15.45:43064) 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.538+0000 7f538e7fc640 1 -- 172.21.15.45:0/2196770395 learned_addr learned my addr 172.21.15.45:0/2196770395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538effd640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538f7fe640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 -- 172.21.15.45:0/2196770395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 msgr2=0x55b00b0ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 -- 172.21.15.45:0/2196770395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 msgr2=0x55b00b0da690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0da690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 -- 172.21.15.45:0/2196770395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b00b0e0780 con 0x55b00b0e4e30 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538effd640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0da690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f538e7fc640 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53780cc670 tx=0x7f53780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49474cbb2ae130d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f53857fa640 1 -- 172.21.15.45:0/2196770395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537809aca0 con 0x55b00b0e4e30 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f53857fa640 1 -- 172.21.15.45:0/2196770395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537809ba40 con 0x55b00b0e4e30 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.539+0000 7f53857fa640 1 -- 172.21.15.45:0/2196770395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53780666b0 con 0x55b00b0e4e30 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- 172.21.15.45:0/2196770395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 msgr2=0x55b00b0f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53780cc670 tx=0x7f53780057b0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- 172.21.15.45:0/2196770395 shutdown_connections 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b0ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4e30 0x55b00b0f5d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- 172.21.15.45:0/2196770395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4440 0x55b00b0da690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- 172.21.15.45:0/2196770395 >> 172.21.15.45:0/2196770395 conn(0x55b00b0dfab0 msgr2=0x55b00b0dfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- 172.21.15.45:0/2196770395 shutdown_connections 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- 172.21.15.45:0/2196770395 wait complete. 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- start start 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 Processor -- start 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 0x55b00b15d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b15f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fc7b0 con 0x55b00b0e4440 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fc0e0 con 0x55b00b0e4e30 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b00b0fce80 con 0x55b00b0e9fa0 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f539858e740 10 client.0 _ll_register_callbacks cb 0x7f5388032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.540+0000 7f538effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43070/0 (socket says 172.21.15.45:43070) 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 -- 172.21.15.45:0/3417236542 learned_addr learned my addr 172.21.15.45:0/3417236542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538e7fc640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 0x55b00b15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538f7fe640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b15f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 -- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 msgr2=0x55b00b15f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b15f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 -- 172.21.15.45:0/3417236542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 msgr2=0x55b00b15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 0x55b00b15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 -- 172.21.15.45:0/3417236542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00b1601f0 con 0x55b00b0e4440 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538e7fc640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 0x55b00b15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f538effd640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5380015040 tx=0x7f5380004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7380713f9d15f993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f536effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538002b030 con 0x55b00b0e4440 2024-04-25T00:21:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5380027d60 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537809ba40 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5380027310 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f539858e740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b00b161560 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.541+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537809ba40 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f538000f040 con 0x55b00b0e4440 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 10 client.15240 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 20 client.15240 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b1661b0 0x55b00b164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 4 client.15240 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "status"}) -- 0x55b00b04f790 con 0x55b00b1661b0 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f538e7fc640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b1661b0 0x55b00b164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f536effd640 10 client.15240 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.542+0000 7f538e7fc640 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b1661b0 0x55b00b164ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b778a8a1472e2c92 server_cookie=5926ad0a0fbb84ef in_seq=0 out_seq=0 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f536effd640 1 -- 172.21.15.45:0/3417236542 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b00b04f790 con 0x55b00b1661b0 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f536effd640 10 client.15240 handle_command_reply: tid=0 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 client.15240 shutdown 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 msgr2=0x55b00b0f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5380015040 tx=0x7f5380004ab0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 shutdown_connections 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b00b1661b0 0x55b00b164ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b00b0e9fa0 0x55b00b15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b00b0e4e30 0x55b00b15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 --2- 172.21.15.45:0/3417236542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b00b0e4440 0x55b00b0f3d30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f536effd640 0 client.15240 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 >> 172.21.15.45:0/3417236542 conn(0x55b00b0dfab0 msgr2=0x55b00b0da690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 shutdown_connections 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 1 -- 172.21.15.45:0/3417236542 wait complete. 2024-04-25T00:21:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539858e740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 4, 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24460, 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 81.563208588999998, 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 91, 2024-04-25T00:21:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 132.104545379 2024-04-25T00:21:22.552 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 msgr2=0x7f537c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 0x7f537c0a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53901027e0 tx=0x7f5380009520 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 msgr2=0x7f5390073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f537800d950 tx=0x7f537800de10 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.543+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 shutdown_connections 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f537c0a2c20 0x7f537c0a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5390103340 0x7f5390074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5390102980 0x7f5390073b30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 --2- 172.21.15.45:0/2849447035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5390101790 0x7f53900754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 >> 172.21.15.45:0/2849447035 conn(0x7f53900fcd20 msgr2=0x7f5390105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 shutdown_connections 2024-04-25T00:21:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.544+0000 7f539587d640 1 -- 172.21.15.45:0/2849447035 wait complete. 2024-04-25T00:21:22.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:21:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 -- 172.21.15.45:0/824220518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59801017a0 msgr2=0x7f5980101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 --2- 172.21.15.45:0/824220518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59801017a0 0x7f5980101bc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f597002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 -- 172.21.15.45:0/824220518 shutdown_connections 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 --2- 172.21.15.45:0/824220518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 0x7f5980109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 --2- 172.21.15.45:0/824220518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5980102990 0x7f5980102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 --2- 172.21.15.45:0/824220518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59801017a0 0x7f5980101bc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 -- 172.21.15.45:0/824220518 >> 172.21.15.45:0/824220518 conn(0x7f59800fcd10 msgr2=0x7f59800ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 -- 172.21.15.45:0/824220518 shutdown_connections 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.657+0000 7f5984cb3640 1 -- 172.21.15.45:0/824220518 wait complete. 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 Processor -- start 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 -- start start 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 0x7f5980197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 0x7f598019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d410 con 0x7f5980102990 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d290 con 0x7f5980103350 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5984cb3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f598010d590 con 0x7f59801017a0 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43088/0 (socket says 172.21.15.45:43088) 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f5977fff640 1 -- 172.21.15.45:0/1847952328 learned_addr learned my addr 172.21.15.45:0/1847952328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.658+0000 7f597ed9d640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 0x7f598019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f597e59c640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 0x7f5980197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 -- 172.21.15.45:0/1847952328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 msgr2=0x7f5980197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 0x7f5980197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 -- 172.21.15.45:0/1847952328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 msgr2=0x7f598019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 0x7f598019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 -- 172.21.15.45:0/1847952328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598019f730 con 0x7f5980102990 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f597e59c640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 0x7f5980197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5977fff640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f596800d770 tx=0x7f596800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f59777fe640 1 -- 172.21.15.45:0/1847952328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968017070 con 0x7f5980102990 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f59777fe640 1 -- 172.21.15.45:0/1847952328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5968005500 con 0x7f5980102990 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598019fa20 con 0x7f5980102990 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f59777fe640 1 -- 172.21.15.45:0/1847952328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968004e00 con 0x7f5980102990 2024-04-25T00:21:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.659+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598019ff20 con 0x7f5980102990 2024-04-25T00:21:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.661+0000 7f59777fe640 1 -- 172.21.15.45:0/1847952328 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5968004fa0 con 0x7f5980102990 2024-04-25T00:21:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 Processor -- start 2024-04-25T00:21:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 -- start start 2024-04-25T00:21:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a91440 0x55de44a91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de449f4f00 con 0x55de44a96fa0 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de4499d750 con 0x55de44a91e30 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59879c4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de449f0ec0 con 0x55de44a91440 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43102/0 (socket says 172.21.15.45:43102) 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f5977fff640 1 -- 172.21.15.45:0/2198888857 learned_addr learned my addr 172.21.15.45:0/2198888857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f59777fe640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 0x7f596c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f597ed9d640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a91440 0x55de44a91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f597e59c640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.662+0000 7f597ed9d640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 0x7f596c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f59777fe640 1 -- 172.21.15.45:0/1847952328 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f59680ceff0 con 0x7f5980102990 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 -- 172.21.15.45:0/2198888857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a91440 msgr2=0x55de44a91860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a91440 0x55de44a91860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 -- 172.21.15.45:0/2198888857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 msgr2=0x55de44a96930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a96930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 -- 172.21.15.45:0/2198888857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de447092d0 con 0x55de44a96fa0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f597e59c640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5977fff640 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59680cc540 tx=0x7f5968009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a40ffe8d9d3568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f597ed9d640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 0x7f596c0a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55de44a92690 tx=0x7f5978004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5974ff9640 1 -- 172.21.15.45:0/2198888857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968020030 con 0x55de44a96fa0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5974ff9640 1 -- 172.21.15.45:0/2198888857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59680d26b0 con 0x55de44a96fa0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.663+0000 7f5974ff9640 1 -- 172.21.15.45:0/2198888857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968099720 con 0x55de44a96fa0 2024-04-25T00:21:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- 172.21.15.45:0/2198888857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 msgr2=0x55de44a99430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59680cc540 tx=0x7f5968009c30 comp rx=0 tx=0).stop 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- 172.21.15.45:0/2198888857 shutdown_connections 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a96fa0 0x55de44a99430 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- 172.21.15.45:0/2198888857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a91440 0x55de44a91860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- 172.21.15.45:0/2198888857 >> 172.21.15.45:0/2198888857 conn(0x55de44a8cab0 msgr2=0x55de44a8ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- 172.21.15.45:0/2198888857 shutdown_connections 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- 172.21.15.45:0/2198888857 wait complete. 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- start start 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 Processor -- start 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44aa2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de44a99ef0 con 0x55de44a91440 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de449b0cf0 con 0x55de44a91e30 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de44a9a070 con 0x55de44a96fa0 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.664+0000 7f59879c4740 10 client.0 _ll_register_callbacks cb 0x55de449b06e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43108/0 (socket says 172.21.15.45:43108) 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 -- 172.21.15.45:0/4145660303 learned_addr learned my addr 172.21.15.45:0/4145660303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f5977fff640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597ed9d640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44aa2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 -- 172.21.15.45:0/4145660303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 msgr2=0x55de44aa2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44aa2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 -- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 msgr2=0x55de44a9bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a9bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 -- 172.21.15.45:0/4145660303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de447d36b0 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f597e59c640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f59700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7771a101ac401ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f5965ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970035040 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5970005230 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de44a02440 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59700044e0 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f59879c4740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.665+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de44aa5d80 con 0x55de44a91440 2024-04-25T00:21:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55de44a02440 con 0x55de44a91440 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59700035f0 con 0x55de44a91440 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 20 client.15252 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44aa8460 0x55de44aa98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 4 client.15252 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de449fc790 con 0x55de44aa8460 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.666+0000 7f5977fff640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44aa8460 0x55de44aa98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f5977fff640 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44aa8460 0x55de44aa98d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66b9f5f75aa03003 server_cookie=e0050e2dd7bb1993 in_seq=0 out_seq=0 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f5965ffb640 10 client.15252 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f5965ffb640 1 -- 172.21.15.45:0/4145660303 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de449fc790 con 0x55de44aa8460 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f5965ffb640 10 client.15252 handle_command_reply: tid=0 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f59879c4740 1 client.15252 shutdown 2024-04-25T00:21:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 msgr2=0x55de44a9b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.667+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5970009900 tx=0x7f59700042f0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 shutdown_connections 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44aa8460 0x55de44aa98d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44aa2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91e30 0x55de44a9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 --2- 172.21.15.45:0/4145660303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91440 0x55de44a9b7b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f5965ffb640 0 client.15252 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 >> 172.21.15.45:0/4145660303 conn(0x55de44a8cab0 msgr2=0x55de44a8bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 shutdown_connections 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 1 -- 172.21.15.45:0/4145660303 wait complete. 2024-04-25T00:21:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.668+0000 7f59879c4740 20 client.15252 trim_cache size 0 max 16384 2024-04-25T00:21:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 Processor -- start 2024-04-25T00:21:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 -- start start 2024-04-25T00:21:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44a8b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44a9aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa7c10 con 0x55de44a91e30 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa82e0 con 0x55de44a91440 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f59879c4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa89b0 con 0x55de44a96fa0 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43122/0 (socket says 172.21.15.45:43122) 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 -- 172.21.15.45:0/3586806269 learned_addr learned my addr 172.21.15.45:0/3586806269 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 -- 172.21.15.45:0/3586806269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 msgr2=0x55de44a9aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44a9aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 -- 172.21.15.45:0/3586806269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 msgr2=0x55de44a8b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44a8b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 -- 172.21.15.45:0/3586806269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de44a92210 con 0x55de44a91e30 2024-04-25T00:21:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.686+0000 7f5977fff640 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f596800d740 tx=0x7f59680bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c115d00179c1f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f5967fff640 1 -- 172.21.15.45:0/3586806269 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968020040 con 0x55de44a91e30 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f5967fff640 1 -- 172.21.15.45:0/3586806269 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596809ba50 con 0x55de44a91e30 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f5967fff640 1 -- 172.21.15.45:0/3586806269 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59680cc850 con 0x55de44a91e30 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- 172.21.15.45:0/3586806269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 msgr2=0x55de44a8bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f596800d740 tx=0x7f59680bb040 comp rx=0 tx=0).stop 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- 172.21.15.45:0/3586806269 shutdown_connections 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44a9aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44a8bea0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 --2- 172.21.15.45:0/3586806269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44a8b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- 172.21.15.45:0/3586806269 >> 172.21.15.45:0/3586806269 conn(0x55de44a8cab0 msgr2=0x55de44a8b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- 172.21.15.45:0/3586806269 shutdown_connections 2024-04-25T00:21:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- 172.21.15.45:0/3586806269 wait complete. 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.687+0000 7f59879c4740 1 -- start start 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 Processor -- start 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44b0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44b11000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa9750 con 0x55de44a91e30 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa9080 con 0x55de44a91440 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de44aa9e20 con 0x55de44a96fa0 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 10 client.0 _ll_register_callbacks cb 0x7f5970031b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43138/0 (socket says 172.21.15.45:43138) 2024-04-25T00:21:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 -- 172.21.15.45:0/163791460 learned_addr learned my addr 172.21.15.45:0/163791460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f597e59c640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44b0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f597ed9d640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44b11000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 -- 172.21.15.45:0/163791460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 msgr2=0x55de44b11000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44b11000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 -- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 msgr2=0x55de44b0c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44b0c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 -- 172.21.15.45:0/163791460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de44b11830 con 0x55de44a91e30 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f597e59c640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44b0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5977fff640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5968005760 tx=0x7f59680cc540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3466c3d03be904d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5965ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968020070 con 0x55de44a91e30 2024-04-25T00:21:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f59680cc850 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.688+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f596809ba50 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59680b4200 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de44b12b30 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f596809ba50 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59680970a0 con 0x55de44a91e30 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 20 client.15258 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44b17060 0x55de44b13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 4 client.15258 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "status"}) -- 0x55de4497ab80 con 0x55de44b17060 2024-04-25T00:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.689+0000 7f597e59c640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44b17060 0x55de44b13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f597e59c640 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44b17060 0x55de44b13b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70e7b7e1b61547bd server_cookie=d5ad3bdde9b953d7 in_seq=0 out_seq=0 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f5965ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f5965ffb640 1 -- 172.21.15.45:0/163791460 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55de4497ab80 con 0x55de44b17060 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f5965ffb640 10 client.15258 handle_command_reply: tid=0 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 client.15258 shutdown 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 msgr2=0x55de44b0e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5968005760 tx=0x7f59680cc540 comp rx=0 tx=0).stop 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 shutdown_connections 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55de44b17060 0x55de44b13b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de44a96fa0 0x55de44b11000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f5965ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55de44a91e30 0x55de44b0e950 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 --2- 172.21.15.45:0/163791460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de44a91440 0x55de44b0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 >> 172.21.15.45:0/163791460 conn(0x55de44a8cab0 msgr2=0x55de44a8bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 shutdown_connections 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 1 -- 172.21.15.45:0/163791460 wait complete. 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f59879c4740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 3, 2024-04-25T00:21:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24467, 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 82.717096283999993, 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 91, 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 126.45835791499999 2024-04-25T00:21:22.700 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.690+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 msgr2=0x7f596c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 0x7f596c0a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55de44a92690 tx=0x7f5978004ab0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 msgr2=0x7f59801980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f596800d770 tx=0x7f596800dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 shutdown_connections 2024-04-25T00:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f596c0a2c20 0x7f596c0a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980103350 0x7f598019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5980102990 0x7f59801980a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 --2- 172.21.15.45:0/1847952328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59801017a0 0x7f5980197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 >> 172.21.15.45:0/1847952328 conn(0x7f59800fcd10 msgr2=0x7f5980105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 shutdown_connections 2024-04-25T00:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.691+0000 7f5984cb3640 1 -- 172.21.15.45:0/1847952328 wait complete. 2024-04-25T00:21:22.703 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:21:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 -- 172.21.15.45:0/2660004982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 msgr2=0x7f7c84108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 --2- 172.21.15.45:0/2660004982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c84108ae0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009900 tx=0x7f7c7c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 -- 172.21.15.45:0/2660004982 shutdown_connections 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 --2- 172.21.15.45:0/2660004982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c84108ae0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 --2- 172.21.15.45:0/2660004982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c84101890 0x7f7c84101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 --2- 172.21.15.45:0/2660004982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c841006a0 0x7f7c84100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 -- 172.21.15.45:0/2660004982 >> 172.21.15.45:0/2660004982 conn(0x7f7c840fbc30 msgr2=0x7f7c840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 -- 172.21.15.45:0/2660004982 shutdown_connections 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 -- 172.21.15.45:0/2660004982 wait complete. 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.805+0000 7f7c8b328640 1 Processor -- start 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 -- start start 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 0x7f7c8406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c84101890 0x7f7c8419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410c300 con 0x7f7c84102250 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410c180 con 0x7f7c841006a0 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c8b328640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8410c480 con 0x7f7c84101890 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43142/0 (socket says 172.21.15.45:43142) 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 -- 172.21.15.45:0/424530876 learned_addr learned my addr 172.21.15.45:0/424530876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c888c3640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c84101890 0x7f7c8419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c890c4640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 0x7f7c8406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 -- 172.21.15.45:0/424530876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c84101890 msgr2=0x7f7c8419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c84101890 0x7f7c8419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 -- 172.21.15.45:0/424530876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 msgr2=0x7f7c8406a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 0x7f7c8406a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 -- 172.21.15.45:0/424530876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8419f5c0 con 0x7f7c84102250 2024-04-25T00:21:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c890c4640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 0x7f7c8406a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c898c5640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0314e0 tx=0x7f7c7c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.806+0000 7f7c827fc640 1 -- 172.21.15.45:0/424530876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c03f030 con 0x7f7c84102250 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.807+0000 7f7c827fc640 1 -- 172.21.15.45:0/424530876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c003ce0 con 0x7f7c84102250 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.807+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8419f850 con 0x7f7c84102250 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.807+0000 7f7c827fc640 1 -- 172.21.15.45:0/424530876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c035070 con 0x7f7c84102250 2024-04-25T00:21:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.807+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8419fd40 con 0x7f7c84102250 2024-04-25T00:21:22.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.809+0000 7f7c827fc640 1 -- 172.21.15.45:0/424530876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c7c042030 con 0x7f7c84102250 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.809+0000 7f7c8e030740 1 Processor -- start 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.809+0000 7f7c8e030740 1 -- start start 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b00860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557617a63f00 con 0x557617b05fa0 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557617a0c750 con 0x557617b00440 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c8e030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557617a5fec0 con 0x557617b00e30 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c827fc640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 0x7f7c6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c888c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c898c5640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 0x7f7c6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:42012/0 (socket says 172.21.15.45:42012) 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 -- 172.21.15.45:0/1092061853 learned_addr learned my addr 172.21.15.45:0/1092061853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c888c3640 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b00860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c827fc640 1 -- 172.21.15.45:0/424530876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7c7c0eff90 con 0x7f7c84102250 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 -- 172.21.15.45:0/1092061853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 msgr2=0x557617b00860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b00860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 -- 172.21.15.45:0/1092061853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 msgr2=0x557617b08430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b08430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 -- 172.21.15.45:0/1092061853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576177782d0 con 0x557617b00e30 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c888c3640 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c898c5640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 0x7f7c6c0a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0bc340 tx=0x7f7c7c0c0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.810+0000 7f7c890c4640 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400aed0 tx=0x7f7c7400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc7e1f8c46bd84e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c6b7fe640 1 -- 172.21.15.45:0/1092061853 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74004d20 con 0x557617b00e30 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c6b7fe640 1 -- 172.21.15.45:0/1092061853 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c74011b30 con 0x557617b00e30 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c6b7fe640 1 -- 172.21.15.45:0/1092061853 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c740055e0 con 0x557617b00e30 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 -- 172.21.15.45:0/1092061853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 msgr2=0x557617b05930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400aed0 tx=0x7f7c7400f730 comp rx=0 tx=0).stop 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 -- 172.21.15.45:0/1092061853 shutdown_connections 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00e30 0x557617b05930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1092061853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b00860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 -- 172.21.15.45:0/1092061853 >> 172.21.15.45:0/1092061853 conn(0x557617afbab0 msgr2=0x557617afbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 -- 172.21.15.45:0/1092061853 shutdown_connections 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.811+0000 7f7c8e030740 1 -- 172.21.15.45:0/1092061853 wait complete. 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 -- start start 2024-04-25T00:21:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 Processor -- start 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b05fa0 0x557617b11e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557617b08f50 con 0x557617b00e30 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557617a1fcf0 con 0x557617b05fa0 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557617b090d0 con 0x557617b00440 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c8e030740 10 client.0 _ll_register_callbacks cb 0x557617a1f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43162/0 (socket says 172.21.15.45:43162) 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c890c4640 1 -- 172.21.15.45:0/3253316198 learned_addr learned my addr 172.21.15.45:0/3253316198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c888c3640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.812+0000 7f7c888c3640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b05fa0 0x557617b11e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 -- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 msgr2=0x557617b0a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b0a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 -- 172.21.15.45:0/3253316198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b05fa0 msgr2=0x557617b11e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b05fa0 0x557617b11e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 -- 172.21.15.45:0/3253316198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576178426b0 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c888c3640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b0a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c890c4640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400a300 tx=0x7f7c74011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f01286a8e1db15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74004c50 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7c74003970 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c74011b30 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c74003b70 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c8e030740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557617b14e00 con 0x557617b00e30 2024-04-25T00:21:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c74011b30 con 0x557617b00e30 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.813+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7c74011ce0 con 0x557617b00e30 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 10 client.15270 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 20 client.15270 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b17240 0x557617b186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 4 client.15270 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557617a71440 con 0x557617b17240 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c888c3640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b17240 0x557617b186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c68ff9640 10 client.15270 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.814+0000 7f7c888c3640 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b17240 0x557617b186b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1882ae75f534ea7 server_cookie=94ecc68eb60ef6a4 in_seq=0 out_seq=0 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c68ff9640 1 -- 172.21.15.45:0/3253316198 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557617a71440 con 0x557617b17240 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c68ff9640 10 client.15270 handle_command_reply: tid=0 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 client.15270 shutdown 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 msgr2=0x557617b0ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400a300 tx=0x7f7c74011880 comp rx=0 tx=0).stop 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 shutdown_connections 2024-04-25T00:21:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b17240 0x557617b186b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b05fa0 0x557617b11e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c68ff9640 0 client.15270 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0ade0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 --2- 172.21.15.45:0/3253316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b0a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 >> 172.21.15.45:0/3253316198 conn(0x557617afbab0 msgr2=0x557617afaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 shutdown_connections 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.815+0000 7f7c8e030740 1 -- 172.21.15.45:0/3253316198 wait complete. 2024-04-25T00:21:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.816+0000 7f7c8e030740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T00:21:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 Processor -- start 2024-04-25T00:21:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 -- start start 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b08e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00e30 0x557617b11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557617b169f0 con 0x557617b05fa0 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557617b170c0 con 0x557617b00e30 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c8e030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557617b17790 con 0x557617b00440 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43174/0 (socket says 172.21.15.45:43174) 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 -- 172.21.15.45:0/1413110131 learned_addr learned my addr 172.21.15.45:0/1413110131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 -- 172.21.15.45:0/1413110131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 msgr2=0x557617b08e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b08e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 -- 172.21.15.45:0/1413110131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00e30 msgr2=0x557617b11de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00e30 0x557617b11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.834+0000 7f7c888c3640 1 -- 172.21.15.45:0/1413110131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557617b054c0 con 0x557617b05fa0 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c888c3640 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x557617b19040 tx=0x7f7c78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fce1572668d95f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c6affd640 1 -- 172.21.15.45:0/1413110131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78013230 con 0x557617b05fa0 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c6affd640 1 -- 172.21.15.45:0/1413110131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c78011d70 con 0x557617b05fa0 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c6affd640 1 -- 172.21.15.45:0/1413110131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004a00 con 0x557617b05fa0 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 -- 172.21.15.45:0/1413110131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 msgr2=0x557617b0ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x557617b19040 tx=0x7f7c78004730 comp rx=0 tx=0).stop 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 -- 172.21.15.45:0/1413110131 shutdown_connections 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b05fa0 0x557617b0ad40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00e30 0x557617b11de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 --2- 172.21.15.45:0/1413110131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b00440 0x557617b08e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 -- 172.21.15.45:0/1413110131 >> 172.21.15.45:0/1413110131 conn(0x557617afbab0 msgr2=0x557617afbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.835+0000 7f7c8e030740 1 -- 172.21.15.45:0/1413110131 shutdown_connections 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 -- 172.21.15.45:0/1413110131 wait complete. 2024-04-25T00:21:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 -- start start 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 Processor -- start 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b05fa0 0x557617b10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557617b18530 con 0x557617b00e30 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557617b17e60 con 0x557617b00440 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557617b18c00 con 0x557617b05fa0 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c8e030740 10 client.0 _ll_register_callbacks cb 0x7f7c7400ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c890c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43184/0 (socket says 172.21.15.45:43184) 2024-04-25T00:21:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.836+0000 7f7c890c4640 1 -- 172.21.15.45:0/639281582 learned_addr learned my addr 172.21.15.45:0/639281582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c888c3640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b05fa0 0x557617b10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 -- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b05fa0 msgr2=0x557617b10810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b05fa0 0x557617b10810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c888c3640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 -- 172.21.15.45:0/639281582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 msgr2=0x557617b0d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b0d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 -- 172.21.15.45:0/639281582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557617b10e00 con 0x557617b00e30 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c890c4640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005e70 tx=0x7f7c74004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e2c0834411bc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400a770 con 0x557617b00e30 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c78011d70 con 0x557617b00e30 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7c74021040 con 0x557617b00e30 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c74003970 con 0x557617b00e30 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557617b12100 con 0x557617b00e30 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c78011d70 con 0x557617b00e30 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7c74037970 con 0x557617b00e30 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 10 client.15276 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.837+0000 7f7c8e030740 20 client.15276 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b81bd0 0x557617b808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 4 client.15276 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "status"}) -- 0x557617a6b790 con 0x557617b81bd0 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c888c3640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b81bd0 0x557617b808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c888c3640 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b81bd0 0x557617b808e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=634fc17a85114b06 server_cookie=655f88b8b0959f0f in_seq=0 out_seq=0 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c68ff9640 10 client.15276 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c68ff9640 1 -- 172.21.15.45:0/639281582 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557617a6b790 con 0x557617b81bd0 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c68ff9640 10 client.15276 handle_command_reply: tid=0 2024-04-25T00:21:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 client.15276 shutdown 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 msgr2=0x557617b0d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005e70 tx=0x7f7c74004970 comp rx=0 tx=0).stop 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 shutdown_connections 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x557617b81bd0 0x557617b808e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557617b05fa0 0x557617b10810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557617b00e30 0x557617b0d9c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.838+0000 7f7c68ff9640 0 client.15276 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 1 --2- 172.21.15.45:0/639281582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557617b00440 0x557617b0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 >> 172.21.15.45:0/639281582 conn(0x557617afbab0 msgr2=0x557617af6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 shutdown_connections 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 1 -- 172.21.15.45:0/639281582 wait complete. 2024-04-25T00:21:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8e030740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24478, 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 83.876009397999994, 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 91, 2024-04-25T00:21:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 129.45165821800001 2024-04-25T00:21:22.848 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 msgr2=0x7f7c6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 0x7f7c6c0a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0bc340 tx=0x7f7c7c0c0040 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 msgr2=0x7f7c8419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0314e0 tx=0x7f7c7c0043c0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 shutdown_connections 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7c6c0a2c20 0x7f7c6c0a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c84102250 0x7f7c8419ed00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c84101890 0x7f7c8419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 --2- 172.21.15.45:0/424530876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c841006a0 0x7f7c8406a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 >> 172.21.15.45:0/424530876 conn(0x7f7c840fbc30 msgr2=0x7f7c84104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 shutdown_connections 2024-04-25T00:21:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.839+0000 7f7c8b328640 1 -- 172.21.15.45:0/424530876 wait complete. 2024-04-25T00:21:22.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:21:22.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 -- 172.21.15.45:0/2030176867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c102250 msgr2=0x7fc56c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2030176867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c102250 0x7fc56c108ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ca0 tx=0x7fc56402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 -- 172.21.15.45:0/2030176867 shutdown_connections 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2030176867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c102250 0x7fc56c108ae0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2030176867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c101890 0x7fc56c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2030176867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c1006a0 0x7fc56c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 -- 172.21.15.45:0/2030176867 >> 172.21.15.45:0/2030176867 conn(0x7fc56c0fbc30 msgr2=0x7fc56c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 -- 172.21.15.45:0/2030176867 shutdown_connections 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 -- 172.21.15.45:0/2030176867 wait complete. 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.954+0000 7fc571d1b640 1 Processor -- start 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 -- start start 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 0x7fc56c06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c102250 0x7fc56c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c10c410 con 0x7fc56c101890 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c10c290 con 0x7fc56c102250 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc571d1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c10c590 con 0x7fc56c1006a0 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43200/0 (socket says 172.21.15.45:43200) 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 -- 172.21.15.45:0/2509992187 learned_addr learned my addr 172.21.15.45:0/2509992187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56bfff640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c102250 0x7fc56c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 0x7fc56c06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 -- 172.21.15.45:0/2509992187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 msgr2=0x7fc56c06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 0x7fc56c06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 -- 172.21.15.45:0/2509992187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c102250 msgr2=0x7fc56c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c102250 0x7fc56c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 -- 172.21.15.45:0/2509992187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c19f7e0 con 0x7fc56c101890 2024-04-25T00:21:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 0x7fc56c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.955+0000 7fc56affd640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc56000b4a0 tx=0x7fc56000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.956+0000 7fc568ff9640 1 -- 172.21.15.45:0/2509992187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560014070 con 0x7fc56c101890 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.956+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c19fad0 con 0x7fc56c101890 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.956+0000 7fc568ff9640 1 -- 172.21.15.45:0/2509992187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560010af0 con 0x7fc56c101890 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.956+0000 7fc568ff9640 1 -- 172.21.15.45:0/2509992187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560017d10 con 0x7fc56c101890 2024-04-25T00:21:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.956+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c19ffd0 con 0x7fc56c101890 2024-04-25T00:21:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 Processor -- start 2024-04-25T00:21:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc568ff9640 1 -- 172.21.15.45:0/2509992187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc560017070 con 0x7fc56c101890 2024-04-25T00:21:22.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 -- start start 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf5357860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aaf52baf00 con 0x55aaf535cfa0 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aaf5263750 con 0x55aaf5357440 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc574a2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aaf52b6ec0 con 0x55aaf5357e30 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43204/0 (socket says 172.21.15.45:43204) 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 -- 172.21.15.45:0/1186855829 learned_addr learned my addr 172.21.15.45:0/1186855829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56bfff640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf5357860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 -- 172.21.15.45:0/1186855829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 msgr2=0x55aaf535c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 -- 172.21.15.45:0/1186855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 msgr2=0x55aaf5357860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf5357860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc568ff9640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 0x7fc5540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56affd640 1 -- 172.21.15.45:0/1186855829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf4fcf2d0 con 0x55aaf535cfa0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.959+0000 7fc56bfff640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf5357860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc56bfff640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 0x7fc5540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc568ff9640 1 -- 172.21.15.45:0/2509992187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc5600ce8f0 con 0x7fc56c101890 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc56affd640 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cbfb0 tx=0x7fc5600d4f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e4550d938c9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc551ffb640 1 -- 172.21.15.45:0/1186855829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001e070 con 0x55aaf535cfa0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc551ffb640 1 -- 172.21.15.45:0/1186855829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5600a1960 con 0x55aaf535cfa0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.960+0000 7fc551ffb640 1 -- 172.21.15.45:0/1186855829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56009a720 con 0x55aaf535cfa0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- 172.21.15.45:0/1186855829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 msgr2=0x55aaf535f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc56bfff640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 0x7fc5540a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc564031c10 tx=0x7fc564005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cbfb0 tx=0x7fc5600d4f30 comp rx=0 tx=0).stop 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- 172.21.15.45:0/1186855829 shutdown_connections 2024-04-25T00:21:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535f430 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1186855829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf5357860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- 172.21.15.45:0/1186855829 >> 172.21.15.45:0/1186855829 conn(0x55aaf5352ab0 msgr2=0x55aaf5352eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- 172.21.15.45:0/1186855829 shutdown_connections 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- 172.21.15.45:0/1186855829 wait complete. 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- start start 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 Processor -- start 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 0x55aaf53618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5368e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aaf535ff50 con 0x55aaf5357e30 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aaf5276cf0 con 0x55aaf535cfa0 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.961+0000 7fc574a2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aaf53600d0 con 0x55aaf5357440 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc574a2b740 10 client.0 _ll_register_callbacks cb 0x55aaf52766e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43214/0 (socket says 172.21.15.45:43214) 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 -- 172.21.15.45:0/2203869037 learned_addr learned my addr 172.21.15.45:0/2203869037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 0x55aaf53618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56bfff640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5368e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 -- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 msgr2=0x55aaf53618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 0x55aaf53618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 -- 172.21.15.45:0/2203869037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 msgr2=0x55aaf5368e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5368e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 -- 172.21.15.45:0/2203869037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf50996b0 con 0x55aaf5357e30 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 0x55aaf53618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc56affd640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cc4e0 tx=0x7fc560017eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824344fb93c0f45c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc54affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001e050 con 0x55aaf5357e30 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc5600990c0 con 0x55aaf5357e30 2024-04-25T00:21:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aaf52c8440 con 0x55aaf5357e30 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc56009a720 con 0x55aaf5357e30 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc574a2b740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.962+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf536be00 con 0x55aaf5357e30 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aaf52c8440 con 0x55aaf5357e30 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc560097430 con 0x55aaf5357e30 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 10 client.15288 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 20 client.15288 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf536dfc0 0x55aaf536f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 4 client.15288 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf52c2790 con 0x55aaf536dfc0 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.963+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf536dfc0 0x55aaf536f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.964+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf536dfc0 0x55aaf536f430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8ca94443471061b server_cookie=16a810e60d3aed60 in_seq=0 out_seq=0 2024-04-25T00:21:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.964+0000 7fc54affd640 10 client.15288 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.964+0000 7fc54affd640 1 -- 172.21.15.45:0/2203869037 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaf52c2790 con 0x55aaf536dfc0 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.964+0000 7fc54affd640 10 client.15288 handle_command_reply: tid=0 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.964+0000 7fc574a2b740 1 client.15288 shutdown 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 msgr2=0x55aaf5361de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cc4e0 tx=0x7fc560017eb0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 shutdown_connections 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf536dfc0 0x55aaf536f430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5368e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357e30 0x55aaf5361de0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 --2- 172.21.15.45:0/2203869037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357440 0x55aaf53618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc54affd640 0 client.15288 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 >> 172.21.15.45:0/2203869037 conn(0x55aaf5352ab0 msgr2=0x55aaf5351ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 shutdown_connections 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 1 -- 172.21.15.45:0/2203869037 wait complete. 2024-04-25T00:21:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.965+0000 7fc574a2b740 20 client.15288 trim_cache size 0 max 16384 2024-04-25T00:21:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.983+0000 7fc574a2b740 1 Processor -- start 2024-04-25T00:21:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.983+0000 7fc574a2b740 1 -- start start 2024-04-25T00:21:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf53604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf5368de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536d770 con 0x55aaf535cfa0 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536de40 con 0x55aaf5357440 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc574a2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536e510 con 0x55aaf5357e30 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43216/0 (socket says 172.21.15.45:43216) 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 -- 172.21.15.45:0/524867925 learned_addr learned my addr 172.21.15.45:0/524867925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf53604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56affd640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf5368de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 -- 172.21.15.45:0/524867925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 msgr2=0x55aaf5368de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf5368de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 -- 172.21.15.45:0/524867925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 msgr2=0x55aaf53604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf53604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 -- 172.21.15.45:0/524867925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf535ccf0 con 0x55aaf535cfa0 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf53604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc56bfff640 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ca0 tx=0x7fc564040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbdc0b79ea4da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc5517fa640 1 -- 172.21.15.45:0/524867925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564004040 con 0x55aaf535cfa0 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc5517fa640 1 -- 172.21.15.45:0/524867925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564036040 con 0x55aaf535cfa0 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.984+0000 7fc5517fa640 1 -- 172.21.15.45:0/524867925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56402fc90 con 0x55aaf535cfa0 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- 172.21.15.45:0/524867925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 msgr2=0x55aaf535fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ca0 tx=0x7fc564040000 comp rx=0 tx=0).stop 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- 172.21.15.45:0/524867925 shutdown_connections 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf535cfa0 0x55aaf535fda0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf5368de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 --2- 172.21.15.45:0/524867925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf5357440 0x55aaf53604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- 172.21.15.45:0/524867925 >> 172.21.15.45:0/524867925 conn(0x55aaf5352ab0 msgr2=0x55aaf5352e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- 172.21.15.45:0/524867925 shutdown_connections 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- 172.21.15.45:0/524867925 wait complete. 2024-04-25T00:21:22.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.985+0000 7fc574a2b740 1 -- start start 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 Processor -- start 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5367420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536f2b0 con 0x55aaf5357440 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536ebe0 con 0x55aaf535cfa0 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aaf536f980 con 0x55aaf5357e30 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc574a2b740 10 client.0 _ll_register_callbacks cb 0x7fc5600d4300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43226/0 (socket says 172.21.15.45:43226) 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 -- 172.21.15.45:0/1721072696 learned_addr learned my addr 172.21.15.45:0/1721072696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56affd640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56bfff640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5367420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 -- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 msgr2=0x55aaf535b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 -- 172.21.15.45:0/1721072696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 msgr2=0x55aaf5367420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5367420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 -- 172.21.15.45:0/1721072696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf5367a10 con 0x55aaf5357440 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc56b7fe640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000c00 tx=0x7fc55c01be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd217236c953c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:22.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.986+0000 7fc54affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c01b450 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc55c01b5f0 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc564036040 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc55c01b810 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf5368de0 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564036040 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc55c0245b0 con 0x55aaf5357440 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 20 client.15294 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf53d85f0 0x55aaf53d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 4 client.15294 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "status"}) -- 0x55aaf5240b80 con 0x55aaf53d85f0 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.987+0000 7fc56affd640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf53d85f0 0x55aaf53d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc54affd640 10 client.15294 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc56affd640 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf53d85f0 0x55aaf53d4a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebcebe15f50bf860 server_cookie=679df05645a52d63 in_seq=0 out_seq=0 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc54affd640 1 -- 172.21.15.45:0/1721072696 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55aaf5240b80 con 0x55aaf53d85f0 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc54affd640 10 client.15294 handle_command_reply: tid=0 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 client.15294 shutdown 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 msgr2=0x55aaf535b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000c00 tx=0x7fc55c01be40 comp rx=0 tx=0).stop 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 shutdown_connections 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55aaf53d85f0 0x55aaf53d4a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aaf535cfa0 0x55aaf5367420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aaf5357e30 0x55aaf535b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc54affd640 0 client.15294 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 --2- 172.21.15.45:0/1721072696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aaf5357440 0x55aaf535b380 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 >> 172.21.15.45:0/1721072696 conn(0x55aaf5352ab0 msgr2=0x55aaf534d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 shutdown_connections 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 1 -- 172.21.15.45:0/1721072696 wait complete. 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.988+0000 7fc574a2b740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24493, 2024-04-25T00:21:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 85.036633807000001, 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 91, 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 124.03312239 2024-04-25T00:21:22.998 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 msgr2=0x7fc5540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 0x7fc5540a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc564031c10 tx=0x7fc564005fa0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 msgr2=0x7fc56c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc56000b4a0 tx=0x7fc56000b960 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 shutdown_connections 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fc5540a2c20 0x7fc5540a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc56c102250 0x7fc56c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c101890 0x7fc56c19c8e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 --2- 172.21.15.45:0/2509992187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56c1006a0 0x7fc56c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 >> 172.21.15.45:0/2509992187 conn(0x7fc56c0fbc30 msgr2=0x7fc56c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:22.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 shutdown_connections 2024-04-25T00:21:23.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:22.989+0000 7fc571d1b640 1 -- 172.21.15.45:0/2509992187 wait complete. 2024-04-25T00:21:23.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:22 smithi045 ceph-mon[29194]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:23.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:22 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2233936838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:22 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3261602359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:21:23.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 -- 172.21.15.45:0/4186846836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24102230 msgr2=0x7fca24108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 --2- 172.21.15.45:0/4186846836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24102230 0x7fca24108ac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009ca0 tx=0x7fca1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 -- 172.21.15.45:0/4186846836 shutdown_connections 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 --2- 172.21.15.45:0/4186846836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24102230 0x7fca24108ac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 --2- 172.21.15.45:0/4186846836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca24101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 --2- 172.21.15.45:0/4186846836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24100680 0x7fca24100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.103+0000 7fca2bdce640 1 -- 172.21.15.45:0/4186846836 >> 172.21.15.45:0/4186846836 conn(0x7fca240fbbf0 msgr2=0x7fca240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- 172.21.15.45:0/4186846836 shutdown_connections 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- 172.21.15.45:0/4186846836 wait complete. 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 Processor -- start 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- start start 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca2419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24102230 0x7fca241a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c2e0 con 0x7fca24100680 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c160 con 0x7fca24101870 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca2bdce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca2410c460 con 0x7fca24102230 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43248/0 (socket says 172.21.15.45:43248) 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.104+0000 7fca29b6a640 1 -- 172.21.15.45:0/1912108017 learned_addr learned my addr 172.21.15.45:0/1912108017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca2a36b640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24102230 0x7fca241a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29369640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca2419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 -- 172.21.15.45:0/1912108017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24102230 msgr2=0x7fca241a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24102230 0x7fca241a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 -- 172.21.15.45:0/1912108017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 msgr2=0x7fca2419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca2419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 -- 172.21.15.45:0/1912108017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca241a1860 con 0x7fca24100680 2024-04-25T00:21:23.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29369640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca2419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca29b6a640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c970 tx=0x7fca2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca1affd640 1 -- 172.21.15.45:0/1912108017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004220 con 0x7fca24100680 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca1affd640 1 -- 172.21.15.45:0/1912108017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca200043c0 con 0x7fca24100680 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca241a1b50 con 0x7fca24100680 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca1affd640 1 -- 172.21.15.45:0/1912108017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000f450 con 0x7fca24100680 2024-04-25T00:21:23.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.105+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca241a20f0 con 0x7fca24100680 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.107+0000 7fca1affd640 1 -- 172.21.15.45:0/1912108017 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca200079b0 con 0x7fca24100680 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 Processor -- start 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 -- start start 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b570d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b5715430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5648b5670f00 con 0x5648b570de30 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5648b5619750 con 0x5648b570d440 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648b566cec0 con 0x5648b5712fa0 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca1affd640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 0x7fca100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43262/0 (socket says 172.21.15.45:43262) 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca29b6a640 1 -- 172.21.15.45:0/876969711 learned_addr learned my addr 172.21.15.45:0/876969711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2a36b640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 0x7fca100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca29369640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b5715430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca2a36b640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b570d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.108+0000 7fca1affd640 1 -- 172.21.15.45:0/1912108017 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fca200cf340 con 0x7fca24100680 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 -- 172.21.15.45:0/876969711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 msgr2=0x5648b5715430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b5715430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 -- 172.21.15.45:0/876969711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 msgr2=0x5648b570d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b570d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 -- 172.21.15.45:0/876969711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b53852d0 con 0x5648b570de30 2024-04-25T00:21:23.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29369640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b5715430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca2a36b640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 0x7fca100a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009430 tx=0x7fca1c009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca29b6a640 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c940 tx=0x7fca20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d9f740fc8810c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca0ffff640 1 -- 172.21.15.45:0/876969711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001d070 con 0x5648b570de30 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca0ffff640 1 -- 172.21.15.45:0/876969711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20097040 con 0x5648b570de30 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.109+0000 7fca0ffff640 1 -- 172.21.15.45:0/876969711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20007ca0 con 0x5648b570de30 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- 172.21.15.45:0/876969711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 msgr2=0x5648b5712930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c940 tx=0x7fca20000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- 172.21.15.45:0/876969711 shutdown_connections 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b5715430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570de30 0x5648b5712930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/876969711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b570d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- 172.21.15.45:0/876969711 >> 172.21.15.45:0/876969711 conn(0x5648b5708ab0 msgr2=0x5648b5708eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- 172.21.15.45:0/876969711 shutdown_connections 2024-04-25T00:21:23.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- 172.21.15.45:0/876969711 wait complete. 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 -- start start 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.110+0000 7fca2ebb0740 1 Processor -- start 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570de30 0x5648b5717de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b571ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5648b5715f50 con 0x5648b570d440 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5648b562ccf0 con 0x5648b570de30 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648b57160d0 con 0x5648b5712fa0 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 10 client.0 _ll_register_callbacks cb 0x5648b562c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43270/0 (socket says 172.21.15.45:43270) 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 -- 172.21.15.45:0/2700275082 learned_addr learned my addr 172.21.15.45:0/2700275082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29369640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570de30 0x5648b5717de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2a36b640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b571ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 -- 172.21.15.45:0/2700275082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 msgr2=0x5648b571ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b571ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 -- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570de30 msgr2=0x5648b5717de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570de30 0x5648b5717de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 -- 172.21.15.45:0/2700275082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b544f6b0 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2a36b640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b571ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca29b6a640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fca2009b630 tx=0x7fca20086170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080a2ae18707010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001d040 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fca20007ca0 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.111+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20097040 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca2009b6a0 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648b5721e00 con 0x5648b570d440 2024-04-25T00:21:23.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20097040 con 0x5648b570d440 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca20050ac0 con 0x5648b570d440 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 20 client.15306 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b5723f20 0x5648b5725390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 4 client.15306 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b567e440 con 0x5648b5723f20 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.112+0000 7fca29369640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b5723f20 0x5648b5725390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.113+0000 7fca29369640 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b5723f20 0x5648b5725390 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8023c2eb573ea77 server_cookie=c16059a1b3bae407 in_seq=0 out_seq=0 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.113+0000 7fca0d7fa640 10 client.15306 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.113+0000 7fca0d7fa640 1 -- 172.21.15.45:0/2700275082 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648b567e440 con 0x5648b5723f20 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.113+0000 7fca0d7fa640 10 client.15306 handle_command_reply: tid=0 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 client.15306 shutdown 2024-04-25T00:21:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 msgr2=0x5648b57178a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fca2009b630 tx=0x7fca20086170 comp rx=0 tx=0).stop 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 shutdown_connections 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b5712fa0 0x5648b571ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570de30 0x5648b5717de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b5723f20 0x5648b5725390 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/2700275082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b57178a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca0d7fa640 0 client.15306 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 >> 172.21.15.45:0/2700275082 conn(0x5648b5708ab0 msgr2=0x5648b5707ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 shutdown_connections 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 1 -- 172.21.15.45:0/2700275082 wait complete. 2024-04-25T00:21:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.114+0000 7fca2ebb0740 20 client.15306 trim_cache size 0 max 16384 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 Processor -- start 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 -- start start 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b5708040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.132+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5648b57236d0 con 0x5648b5712fa0 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5648b5723da0 con 0x5648b570d440 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648b5724470 con 0x5648b570de30 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43282/0 (socket says 172.21.15.45:43282) 2024-04-25T00:21:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 -- 172.21.15.45:0/558656311 learned_addr learned my addr 172.21.15.45:0/558656311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca29b6a640 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b5708040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca29369640 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 -- 172.21.15.45:0/558656311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 msgr2=0x5648b571edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 -- 172.21.15.45:0/558656311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 msgr2=0x5648b5708040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b5708040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 -- 172.21.15.45:0/558656311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b570e210 con 0x5648b5712fa0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca2a36b640 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031f80 tx=0x7fca1c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d72c2285c8891b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca0f7fe640 1 -- 172.21.15.45:0/558656311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c04f070 con 0x5648b5712fa0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca0f7fe640 1 -- 172.21.15.45:0/558656311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c031810 con 0x5648b5712fa0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.133+0000 7fca0f7fe640 1 -- 172.21.15.45:0/558656311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0305c0 con 0x5648b5712fa0 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- 172.21.15.45:0/558656311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 msgr2=0x5648b5708e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fca1c031f80 tx=0x7fca1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- 172.21.15.45:0/558656311 shutdown_connections 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b5712fa0 0x5648b5708e90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/558656311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b570d440 0x5648b5708040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- 172.21.15.45:0/558656311 >> 172.21.15.45:0/558656311 conn(0x5648b5708ab0 msgr2=0x5648b5712890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- 172.21.15.45:0/558656311 shutdown_connections 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- 172.21.15.45:0/558656311 wait complete. 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 -- start start 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.134+0000 7fca2ebb0740 1 Processor -- start 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 0x5648b571e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5648b5725210 con 0x5648b570d440 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5648b5724b40 con 0x5648b5712fa0 2024-04-25T00:21:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648b57258e0 con 0x5648b570de30 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 10 client.0 _ll_register_callbacks cb 0x7fca200998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43296/0 (socket says 172.21.15.45:43296) 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 -- 172.21.15.45:0/673000896 learned_addr learned my addr 172.21.15.45:0/673000896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29369640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2a36b640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 0x5648b571e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 -- 172.21.15.45:0/673000896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 msgr2=0x5648b571b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 -- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 msgr2=0x5648b571e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 0x5648b571e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 -- 172.21.15.45:0/673000896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b571e7d0 con 0x5648b570d440 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29369640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2a36b640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 0x5648b571e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca29b6a640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fca20050e30 tx=0x7fca20004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ea1cd744a655f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001d070 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fca200afbc0 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.135+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca1c031810 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca2009b000 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648b578a690 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c031810 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca2009a560 con 0x5648b570d440 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 20 client.15312 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b578f0b0 0x5648b578b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 4 client.15312 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "status"}) -- 0x5648b5678790 con 0x5648b578f0b0 2024-04-25T00:21:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.136+0000 7fca29369640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b578f0b0 0x5648b578b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca0d7fa640 10 client.15312 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca29369640 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b578f0b0 0x5648b578b4f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb3397036339d93 server_cookie=7f0b1c3e68abf020 in_seq=0 out_seq=0 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca0d7fa640 1 -- 172.21.15.45:0/673000896 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5648b5678790 con 0x5648b578f0b0 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca0d7fa640 10 client.15312 handle_command_reply: tid=0 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 client.15312 shutdown 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 msgr2=0x5648b571af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fca20050e30 tx=0x7fca20004570 comp rx=0 tx=0).stop 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 shutdown_connections 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5648b5712fa0 0x5648b571e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648b570de30 0x5648b571b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5648b578f0b0 0x5648b578b4f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 --2- 172.21.15.45:0/673000896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648b570d440 0x5648b571af70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca0d7fa640 0 client.15312 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 >> 172.21.15.45:0/673000896 conn(0x5648b5708ab0 msgr2=0x5648b57127c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 shutdown_connections 2024-04-25T00:21:23.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 1 -- 172.21.15.45:0/673000896 wait complete. 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.137+0000 7fca2ebb0740 20 client.15312 trim_cache size 0 max 16384 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24500, 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 109.269454916, 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 91, 2024-04-25T00:21:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 115.590569925 2024-04-25T00:21:23.146 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:23.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 msgr2=0x7fca100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 0x7fca100a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009430 tx=0x7fca1c009800 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 msgr2=0x7fca24199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c970 tx=0x7fca2000ce30 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 shutdown_connections 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fca100a2ca0 0x7fca100a5160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca24102230 0x7fca241a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca24101870 0x7fca2419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 --2- 172.21.15.45:0/1912108017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca24100680 0x7fca24199c50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 >> 172.21.15.45:0/1912108017 conn(0x7fca240fbbf0 msgr2=0x7fca240fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 shutdown_connections 2024-04-25T00:21:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:23.138+0000 7fca2bdce640 1 -- 172.21.15.45:0/1912108017 wait complete. 2024-04-25T00:21:23.149 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T00:21:23.150 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:21:23.160 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:21:23.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T00:21:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:22 smithi122 ceph-mon[32959]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:22 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2233936838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:22 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3261602359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:22 smithi071 ceph-mon[32667]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:21:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:22 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2233936838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:22 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3261602359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:23.407 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:25.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 -- 172.21.15.45:0/2499088867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8101190 msgr2=0x7f04c81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 --2- 172.21.15.45:0/2499088867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8101190 0x7f04c81015b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0099c0 tx=0x7f04bc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:24 smithi045 ceph-mon[29194]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 -- 172.21.15.45:0/2499088867 shutdown_connections 2024-04-25T00:21:25.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 --2- 172.21.15.45:0/2499088867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8102d30 0x7f04c81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 --2- 172.21.15.45:0/2499088867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8101190 0x7f04c81015b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 --2- 172.21.15.45:0/2499088867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102390 0x7f04c81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 -- 172.21.15.45:0/2499088867 >> 172.21.15.45:0/2499088867 conn(0x7f04c80fc730 msgr2=0x7f04c80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 -- 172.21.15.45:0/2499088867 shutdown_connections 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.168+0000 7f04cc99a700 1 -- 172.21.15.45:0/2499088867 wait complete. 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 Processor -- start 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 -- start start 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8101190 0x7f04c819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102d30 0x7f04c81a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810cd50 con 0x7f04c8102390 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810cbd0 con 0x7f04c8101190 2024-04-25T00:21:25.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04cc99a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810ced0 con 0x7f04c8102d30 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04c5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04c5d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46996/0 (socket says 172.21.15.45:46996) 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.169+0000 7f04c5d9b700 1 -- 172.21.15.45:0/449845663 learned_addr learned my addr 172.21.15.45:0/449845663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c659c700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8101190 0x7f04c819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c6d9d700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102d30 0x7f04c81a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 -- 172.21.15.45:0/449845663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102d30 msgr2=0x7f04c81a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102d30 0x7f04c81a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 -- 172.21.15.45:0/449845663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8101190 msgr2=0x7f04c819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8101190 0x7f04c819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 -- 172.21.15.45:0/449845663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c81a2350 con 0x7f04c8102390 2024-04-25T00:21:25.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04c5d9b700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c9e0 tx=0x7f04b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8007ac0 con 0x7f04c8102390 2024-04-25T00:21:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c81a2640 con 0x7f04c8102390 2024-04-25T00:21:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8007c60 con 0x7f04c8102390 2024-04-25T00:21:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8018730 con 0x7f04c8102390 2024-04-25T00:21:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.170+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a2b80 con 0x7f04c8102390 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.171+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c8068f20 con 0x7f04c8102390 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.172+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f04b80188d0 con 0x7f04c8102390 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.173+0000 7f04b77fe700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 0x7f04b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.173+0000 7f04c659c700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 0x7f04b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.173+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f04b80cf520 con 0x7f04c8102390 2024-04-25T00:21:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.173+0000 7f04c659c700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 0x7f04b00a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.177+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f04b8014030 con 0x7f04c8102390 2024-04-25T00:21:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:24 smithi122 ceph-mon[32959]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:24 smithi071 ceph-mon[32667]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:25.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.435+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f04c80fe890 con 0x7f04b00a2cb0 2024-04-25T00:21:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.437+0000 7f04b77fe700 1 -- 172.21.15.45:0/449845663 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f04c80fe890 con 0x7f04b00a2cb0 2024-04-25T00:21:25.437 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-04-25T00:21:25.437 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-04-25T00:21:25.437 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 msgr2=0x7f04b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 0x7f04b00a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005fb0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 msgr2=0x7f04c819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c9e0 tx=0x7f04b800ccf0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 shutdown_connections 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c8102d30 0x7f04c81a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f04b00a2cb0 0x7f04b00a5170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04c8101190 0x7f04c819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 --2- 172.21.15.45:0/449845663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04c8102390 0x7f04c819ac30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 >> 172.21.15.45:0/449845663 conn(0x7f04c80fc730 msgr2=0x7f04c80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 shutdown_connections 2024-04-25T00:21:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:25.439+0000 7f04cc99a700 1 -- 172.21.15.45:0/449845663 wait complete. 2024-04-25T00:21:26.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T00:21:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:27.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:26 smithi045 ceph-mon[29194]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:27.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:26 smithi045 ceph-mon[29194]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:27.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:26 smithi045 ceph-mon[29194]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T00:21:27.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T00:21:27.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:26 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:21:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:26 smithi122 ceph-mon[32959]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:26 smithi122 ceph-mon[32959]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:26 smithi122 ceph-mon[32959]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T00:21:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T00:21:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:26 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:21:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:26 smithi071 ceph-mon[32667]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:26 smithi071 ceph-mon[32667]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:26 smithi071 ceph-mon[32667]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T00:21:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T00:21:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:26 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.002+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/3595014703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 msgr2=0x7f4f441015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/3595014703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f441015d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4f400099c0 tx=0x7f4f40009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/3595014703 shutdown_connections 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/3595014703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f44109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/3595014703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f441015d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/3595014703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/3595014703 >> 172.21.15.45:0/3595014703 conn(0x7f4f440fc750 msgr2=0x7f4f440feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/3595014703 shutdown_connections 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/3595014703 wait complete. 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 Processor -- start 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.003+0000 7f4f4bfbd700 1 -- start start 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f4419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410ce70 con 0x7f4f441011b0 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410ccf0 con 0x7f4f441023b0 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4bfbd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410cff0 con 0x7f4f44102d50 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47014/0 (socket says 172.21.15.45:47014) 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 -- 172.21.15.45:0/4108137096 learned_addr learned my addr 172.21.15.45:0/4108137096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49558700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4a55a700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f4419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 -- 172.21.15.45:0/4108137096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 msgr2=0x7f4f4419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f4419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 -- 172.21.15.45:0/4108137096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 msgr2=0x7f4f44198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 -- 172.21.15.45:0/4108137096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f441a0260 con 0x7f4f441011b0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f4a55a700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f4419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49558700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f49d59700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4f40005e80 tx=0x7f4f40003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.004+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40027070 con 0x7f4f441011b0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.005+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f441a04f0 con 0x7f4f441011b0 2024-04-25T00:21:28.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.005+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a0aa0 con 0x7f4f441011b0 2024-04-25T00:21:28.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.005+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f40022470 con 0x7f4f441011b0 2024-04-25T00:21:28.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.005+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000f6c0 con 0x7f4f441011b0 2024-04-25T00:21:28.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.005+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f28005360 con 0x7f4f441011b0 2024-04-25T00:21:28.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.007+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f4002a070 con 0x7f4f441011b0 2024-04-25T00:21:28.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.007+0000 7f4f3affd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 0x7f4f300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.008+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f4f400d0350 con 0x7f4f441011b0 2024-04-25T00:21:28.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.008+0000 7f4f49558700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 0x7f4f300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.008+0000 7f4f49558700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 0x7f4f300a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f340099d0 tx=0x7f4f34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.011+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f40013070 con 0x7f4f441011b0 2024-04-25T00:21:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T00:21:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:27 smithi045 ceph-mon[29194]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:21:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:27 smithi045 ceph-mon[29194]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:21:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:27 smithi045 ceph-mon[29194]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:21:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.234+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4f28000bf0 con 0x7f4f300a2ca0 2024-04-25T00:21:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.248+0000 7f4f3affd700 1 -- 172.21.15.45:0/4108137096 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f4f28000bf0 con 0x7f4f300a2ca0 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (2m) 28s ago 2m 18.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 36bc0840dc98 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi071 running (2m) 54s ago 2m 25.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9efd29d082b0 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi122 running (2m) 54s ago 2m 17.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 05ccd812718f 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (2m) 28s ago 2m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b9832431b5c 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi071 running (2m) 54s ago 2m 19.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1e27693faf6f 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi122 running (2m) 54s ago 2m 18.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef e4126ecd1580 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (2m) 28s ago 2m 18.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef eb39981a1970 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi071 running (2m) 54s ago 2m 19.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 907d4bec4283 2024-04-25T00:21:28.249 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi122 running (2m) 54s ago 2m 19.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef c547891e9449 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (2m) 28s ago 2m 13.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7475b49d5dfb 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi071 running (2m) 54s ago 2m 18.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1a2b2438e672 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi122 running (2m) 54s ago 2m 17.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 358d2787ef1f 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (9m) 28s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef 69afc426db49 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi071 *:8443,8765 running (7m) 54s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef bc1d8e561dbd 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi122 *:8443,8765 running (7m) 54s ago 7m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef e2985e6f3a91 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (9m) 28s ago 9m 49.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 6599841ffb1a 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi071 running (8m) 54s ago 8m 41.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef bfaa5703b9bf 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi122 running (8m) 54s ago 8m 40.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef d47d25f3654b 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (7m) 28s ago 7m 60.0M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 5d857de0ac37 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi071 running (7m) 54s ago 7m 57.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 2c3a1248b7d6 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi122 running (6m) 54s ago 6m 46.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 7c33b5f8504f 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (6m) 28s ago 6m 44.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef ddffbb85fc27 2024-04-25T00:21:28.250 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi071 running (5m) 54s ago 5m 54.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 07e76dab0800 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi122 running (5m) 54s ago 5m 57.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b64ea0c5c04c 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (5m) 28s ago 5m 53.3M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 07cddafa6249 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi071 running (4m) 54s ago 4m 54.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef a59733ba9b66 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi122 running (4m) 54s ago 4m 53.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 28c82bc731d9 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (3m) 28s ago 3m 53.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 4dc97ebc6ecf 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi071 running (3m) 54s ago 3m 43.6M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef fda36a70a651 2024-04-25T00:21:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi122 running (2m) 54s ago 2m 51.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 5c0f84667477 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 msgr2=0x7f4f300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 0x7f4f300a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f340099d0 tx=0x7f4f34008040 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 msgr2=0x7f4f44198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4f40005e80 tx=0x7f4f40003680 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 shutdown_connections 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44102d50 0x7f4f4419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f4f300a2ca0 0x7f4f300a5160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f441011b0 0x7f4f44198600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 --2- 172.21.15.45:0/4108137096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f441023b0 0x7f4f44198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 >> 172.21.15.45:0/4108137096 conn(0x7f4f440fc750 msgr2=0x7f4f440fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 shutdown_connections 2024-04-25T00:21:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:28.250+0000 7f4f4bfbd700 1 -- 172.21.15.45:0/4108137096 wait complete. 2024-04-25T00:21:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T00:21:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:27 smithi122 ceph-mon[32959]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:21:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:27 smithi122 ceph-mon[32959]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:21:28.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:27 smithi122 ceph-mon[32959]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:21:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T00:21:28.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:27 smithi071 ceph-mon[32667]: from='mgr.14150 172.21.15.45:0/2123716385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:21:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:27 smithi071 ceph-mon[32667]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:21:28.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:27 smithi071 ceph-mon[32667]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:21:29.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:28 smithi045 ceph-mon[29194]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T00:21:29.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T00:21:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:28 smithi122 ceph-mon[32959]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T00:21:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:28 smithi071 ceph-mon[32667]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T00:21:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:29 smithi045 ceph-mon[29194]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:30.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:29 smithi122 ceph-mon[32959]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:30.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:29 smithi071 ceph-mon[32667]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:31.074 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:30 smithi045 ceph-mon[29194]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-25T00:21:31.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.197+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3923728511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c4074c10 msgr2=0x7f53c4075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.197+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3923728511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c4074c10 0x7f53c4075090 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009990 tx=0x7f53b4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3923728511 shutdown_connections 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3923728511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c40761e0 0x7f53c4074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3923728511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c410a1b0 0x7f53c410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3923728511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c4074c10 0x7f53c4075090 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3923728511 >> 172.21.15.45:0/3923728511 conn(0x7f53c40fd850 msgr2=0x7f53c40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3923728511 shutdown_connections 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3923728511 wait complete. 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 Processor -- start 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- start start 2024-04-25T00:21:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 0x7f53c419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 0x7f53c419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4110060 con 0x7f53c410a1b0 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53c410fee0 con 0x7f53c40761e0 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.198+0000 7f53cbfbd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41101e0 con 0x7f53c4074c10 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53c9558700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 0x7f53c419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47042/0 (socket says 172.21.15.45:47042) 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 -- 172.21.15.45:0/3586958742 learned_addr learned my addr 172.21.15.45:0/3586958742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53c9d59700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 0x7f53c419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 -- 172.21.15.45:0/3586958742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 msgr2=0x7f53c419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 0x7f53c419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 -- 172.21.15.45:0/3586958742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 msgr2=0x7f53c419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:31.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 0x7f53c419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 -- 172.21.15.45:0/3586958742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c41a2220 con 0x7f53c410a1b0 2024-04-25T00:21:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53c9d59700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 0x7f53c419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53c9558700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 0x7f53c419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53ca55a700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00ef60 tx=0x7f53bc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:31.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0092e0 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53bc004bb0 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.199+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c41a24b0 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.200+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc004280 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.200+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c41a29a0 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.201+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c4068f20 con 0x7f53c410a1b0 2024-04-25T00:21:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.203+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53bc01b030 con 0x7f53c410a1b0 2024-04-25T00:21:31.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.203+0000 7f53baffd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:31.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.204+0000 7f53c9d59700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:31.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.204+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f53bc0cf850 con 0x7f53c410a1b0 2024-04-25T00:21:31.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.204+0000 7f53c9d59700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 0x7f53b00a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005fd0 tx=0x7f53c0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:31.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.207+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53bc022020 con 0x7f53c410a1b0 2024-04-25T00:21:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:30 smithi122 ceph-mon[32959]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-25T00:21:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:30 smithi071 ceph-mon[32667]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 8 op/s 2024-04-25T00:21:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.425+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f53c4062730 con 0x7f53b00a2c20 2024-04-25T00:21:31.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.431+0000 7f53baffd700 1 -- 172.21.15.45:0/3586958742 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f53c4062730 con 0x7f53b00a2c20 2024-04-25T00:21:31.431 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T00:21:31.431 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 57s ago 2m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-25T00:21:31.431 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 57s ago 7m smithi045=x;smithi071=y;smithi122=z;count:3 2024-04-25T00:21:31.432 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 57s ago 8m smithi045:172.21.15.45=a;smithi071:172.21.15.71=b;smithi122:172.21.15.122=c;count:3 2024-04-25T00:21:31.432 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 57s ago - 2024-04-25T00:21:31.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 msgr2=0x7f53b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:31.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 0x7f53b00a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005fd0 tx=0x7f53c0005e30 comp rx=0 tx=0).stop 2024-04-25T00:21:31.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 msgr2=0x7f53c41a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00ef60 tx=0x7f53bc00ef90 comp rx=0 tx=0).stop 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 shutdown_connections 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c4074c10 0x7f53c419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f53b00a2c20 0x7f53b00a50e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53c410a1b0 0x7f53c41a1b80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 --2- 172.21.15.45:0/3586958742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c40761e0 0x7f53c419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 >> 172.21.15.45:0/3586958742 conn(0x7f53c40fd850 msgr2=0x7f53c4106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 shutdown_connections 2024-04-25T00:21:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:31.433+0000 7f53cbfbd700 1 -- 172.21.15.45:0/3586958742 wait complete. 2024-04-25T00:21:32.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T00:21:32.366 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:32 smithi045 ceph-mon[29194]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-25T00:21:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:32 smithi045 ceph-mon[29194]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:32 smithi122 ceph-mon[32959]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-25T00:21:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:32 smithi122 ceph-mon[32959]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:32 smithi071 ceph-mon[32667]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-25T00:21:33.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:32 smithi071 ceph-mon[32667]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 -- 172.21.15.45:0/3553343411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103e30 msgr2=0x7fd82810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 --2- 172.21.15.45:0/3553343411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103e30 0x7fd82810a6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd8180099c0 tx=0x7fd818009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 -- 172.21.15.45:0/3553343411 shutdown_connections 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 --2- 172.21.15.45:0/3553343411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103e30 0x7fd82810a6e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 --2- 172.21.15.45:0/3553343411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828102290 0x7fd8281026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 --2- 172.21.15.45:0/3553343411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828103490 0x7fd8281038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 -- 172.21.15.45:0/3553343411 >> 172.21.15.45:0/3553343411 conn(0x7fd8280fd830 msgr2=0x7fd8280ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 -- 172.21.15.45:0/3553343411 shutdown_connections 2024-04-25T00:21:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.137+0000 7fd82f61b700 1 -- 172.21.15.45:0/3553343411 wait complete. 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 Processor -- start 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 -- start start 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828102290 0x7fd828198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828103e30 0x7fd82819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810ded0 con 0x7fd828103490 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810dd50 con 0x7fd828103e30 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82f61b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810e050 con 0x7fd828102290 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82cbb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82cbb6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47700/0 (socket says 172.21.15.45:47700) 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82cbb6700 1 -- 172.21.15.45:0/4114218078 learned_addr learned my addr 172.21.15.45:0/4114218078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82dbb8700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828103e30 0x7fd82819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.138+0000 7fd82d3b7700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828102290 0x7fd828198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 -- 172.21.15.45:0/4114218078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828102290 msgr2=0x7fd828198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828102290 0x7fd828198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 -- 172.21.15.45:0/4114218078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828103e30 msgr2=0x7fd82819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828103e30 0x7fd82819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 -- 172.21.15.45:0/4114218078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281a00f0 con 0x7fd828103490 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82cbb6700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b560 tx=0x7fd81c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c015020 con 0x7fd828103490 2024-04-25T00:21:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd81c011d10 con 0x7fd828103490 2024-04-25T00:21:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8281a03e0 con 0x7fd828103490 2024-04-25T00:21:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c010e50 con 0x7fd828103490 2024-04-25T00:21:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.139+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281a09f0 con 0x7fd828103490 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.140+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd828068f20 con 0x7fd828103490 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.141+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd81c018020 con 0x7fd828103490 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.142+0000 7fd8267fc700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 0x7fd8100a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.142+0000 7fd82d3b7700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 0x7fd8100a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.142+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fd81c0ceaf0 con 0x7fd828103490 2024-04-25T00:21:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.143+0000 7fd82d3b7700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 0x7fd8100a5300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd8281032f0 tx=0x7fd820009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.147+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd81c01f020 con 0x7fd828103490 2024-04-25T00:21:34.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.361+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd828062670 con 0x7fd8100a2e40 2024-04-25T00:21:34.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.363+0000 7fd8267fc700 1 -- 172.21.15.45:0/4114218078 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd828062670 con 0x7fd8100a2e40 2024-04-25T00:21:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-04-25T00:21:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-04-25T00:21:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:smithi071 172.21.15.71 2024-04-25T00:21:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:smithi122 172.21.15.122 2024-04-25T00:21:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-04-25T00:21:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 msgr2=0x7fd8100a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:34.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 0x7fd8100a5300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd8281032f0 tx=0x7fd820009040 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 msgr2=0x7fd8281989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00b560 tx=0x7fd81c00b870 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 shutdown_connections 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd828102290 0x7fd828198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd8100a2e40 0x7fd8100a5300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd828103e30 0x7fd82819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 --2- 172.21.15.45:0/4114218078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd828103490 0x7fd8281989d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 >> 172.21.15.45:0/4114218078 conn(0x7fd8280fd830 msgr2=0x7fd828106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 shutdown_connections 2024-04-25T00:21:34.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:34.365+0000 7fd82f61b700 1 -- 172.21.15.45:0/4114218078 wait complete. 2024-04-25T00:21:35.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T00:21:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:34 smithi122 ceph-mon[32959]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s 2024-04-25T00:21:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:34 smithi071 ceph-mon[32667]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s 2024-04-25T00:21:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:34 smithi045 ceph-mon[29194]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s 2024-04-25T00:21:36.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:35 smithi045 ceph-mon[29194]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:35 smithi122 ceph-mon[32959]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:36.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:35 smithi071 ceph-mon[32667]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 -- 172.21.15.45:0/2274399245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 msgr2=0x7f887c110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 --2- 172.21.15.45:0/2274399245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c110c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f887400b0a0 tx=0x7f887400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 -- 172.21.15.45:0/2274399245 shutdown_connections 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 --2- 172.21.15.45:0/2274399245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c076260 0x7f887c0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 --2- 172.21.15.45:0/2274399245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c110c80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 --2- 172.21.15.45:0/2274399245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c074c90 0x7f887c075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 -- 172.21.15.45:0/2274399245 >> 172.21.15.45:0/2274399245 conn(0x7f887c0fd850 msgr2=0x7f887c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 -- 172.21.15.45:0/2274399245 shutdown_connections 2024-04-25T00:21:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.130+0000 7f8883066700 1 -- 172.21.15.45:0/2274399245 wait complete. 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 Processor -- start 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 -- start start 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c074c90 0x7f887c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 0x7f887c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1144b0 con 0x7f887c10e890 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f887c114330 con 0x7f887c076260 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8883066700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f887c114630 con 0x7f887c074c90 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47720/0 (socket says 172.21.15.45:47720) 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 -- 172.21.15.45:0/3880835109 learned_addr learned my addr 172.21.15.45:0/3880835109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f887bfff700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 0x7f887c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8880e02700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c074c90 0x7f887c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 -- 172.21.15.45:0/3880835109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c074c90 msgr2=0x7f887c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c074c90 0x7f887c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 -- 172.21.15.45:0/3880835109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 msgr2=0x7f887c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 0x7f887c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f8881603700 1 -- 172.21.15.45:0/3880835109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c1a66b0 con 0x7f887c10e890 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.131+0000 7f887bfff700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 0x7f887c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8881603700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8874004d00 tx=0x7f8874004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874021070 con 0x7f887c10e890 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c1a6940 con 0x7f887c10e890 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8874007b80 con 0x7f887c10e890 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740053a0 con 0x7f887c10e890 2024-04-25T00:21:37.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.132+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c1a6e40 con 0x7f887c10e890 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.133+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887c068f20 con 0x7f887c10e890 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.134+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8874024070 con 0x7f887c10e890 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.135+0000 7f8879ffb700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 0x7f886c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.135+0000 7f8880e02700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 0x7f886c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.135+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f887400f020 con 0x7f887c10e890 2024-04-25T00:21:37.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.135+0000 7f8880e02700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 0x7f886c0a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f887c076640 tx=0x7f8870009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:37.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.139+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f887402b020 con 0x7f887c10e890 2024-04-25T00:21:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:36 smithi045 ceph-mon[29194]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 488 B/s wr, 11 op/s 2024-04-25T00:21:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:36 smithi122 ceph-mon[32959]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 488 B/s wr, 11 op/s 2024-04-25T00:21:37.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.354+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 --> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f887c062670 con 0x7f886c0a2c20 2024-04-25T00:21:37.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.356+0000 7f8879ffb700 1 -- 172.21.15.45:0/3880835109 <== mgr.14150 v2:172.21.15.45:6800/3305016128 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f887c062670 con 0x7f886c0a2c20 2024-04-25T00:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T00:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:21:37.357 INFO:teuthology.orchestra.run.smithi045.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T00:21:37.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 msgr2=0x7f886c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 0x7f886c0a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f887c076640 tx=0x7f8870009260 comp rx=0 tx=0).stop 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 msgr2=0x7f887c1a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8874004d00 tx=0x7f8874004de0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 shutdown_connections 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f887c074c90 0x7f887c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f886c0a2c20 0x7f886c0a50e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f887c10e890 0x7f887c1a6010 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 --2- 172.21.15.45:0/3880835109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f887c076260 0x7f887c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 >> 172.21.15.45:0/3880835109 conn(0x7f887c0fd850 msgr2=0x7f887c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 shutdown_connections 2024-04-25T00:21:37.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:37.358+0000 7f8883066700 1 -- 172.21.15.45:0/3880835109 wait complete. 2024-04-25T00:21:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:36 smithi071 ceph-mon[32667]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 488 B/s wr, 11 op/s 2024-04-25T00:21:37.974 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:21:37.983 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:21:37.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T00:21:38.235 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:38 smithi122 ceph-mon[32959]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T00:21:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:38 smithi122 ceph-mon[32959]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:39.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:38 smithi045 ceph-mon[29194]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T00:21:39.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:38 smithi045 ceph-mon[29194]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:38 smithi071 ceph-mon[32667]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T00:21:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:38 smithi071 ceph-mon[32667]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:21:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 -- 172.21.15.45:0/1013846103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c102290 msgr2=0x7fac3c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 --2- 172.21.15.45:0/1013846103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c102290 0x7fac3c1026b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0099c0 tx=0x7fac2c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 -- 172.21.15.45:0/1013846103 shutdown_connections 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 --2- 172.21.15.45:0/1013846103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 0x7fac3c10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 --2- 172.21.15.45:0/1013846103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c102290 0x7fac3c1026b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 --2- 172.21.15.45:0/1013846103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c103490 0x7fac3c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 -- 172.21.15.45:0/1013846103 >> 172.21.15.45:0/1013846103 conn(0x7fac3c0fd830 msgr2=0x7fac3c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.916+0000 7fac41f97700 1 -- 172.21.15.45:0/1013846103 shutdown_connections 2024-04-25T00:21:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.917+0000 7fac41f97700 1 -- 172.21.15.45:0/1013846103 wait complete. 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.917+0000 7fac41f97700 1 Processor -- start 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 -- start start 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c102290 0x7fac3c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 0x7fac3c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10de50 con 0x7fac3c103490 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10dcd0 con 0x7fac3c102290 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac41f97700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10dfd0 con 0x7fac3c103e30 2024-04-25T00:21:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47736/0 (socket says 172.21.15.45:47736) 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 -- 172.21.15.45:0/1626591361 learned_addr learned my addr 172.21.15.45:0/1626591361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3bfff700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 0x7fac3c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3b7fe700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c102290 0x7fac3c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 -- 172.21.15.45:0/1626591361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 msgr2=0x7fac3c19fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 0x7fac3c19fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 -- 172.21.15.45:0/1626591361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c102290 msgr2=0x7fac3c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c102290 0x7fac3c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.918+0000 7fac3affd700 1 -- 172.21.15.45:0/1626591361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1a0120 con 0x7fac3c103490 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac3affd700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e9d0 tx=0x7fac3000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3000cbd0 con 0x7fac3c103490 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c1a0410 con 0x7fac3c103490 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac3000ee70 con 0x7fac3c103490 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30018660 con 0x7fac3c103490 2024-04-25T00:21:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.919+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a0950 con 0x7fac3c103490 2024-04-25T00:21:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.920+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c068f20 con 0x7fac3c103490 2024-04-25T00:21:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.921+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac30010430 con 0x7fac3c103490 2024-04-25T00:21:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.922+0000 7fac38ff9700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 0x7fac240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.922+0000 7fac3b7fe700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 0x7fac240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.922+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fac300cf5c0 con 0x7fac3c103490 2024-04-25T00:21:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.922+0000 7fac3b7fe700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 0x7fac240a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b580 tx=0x7fac2c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:39.926+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fac3009c1a0 con 0x7fac3c103490 2024-04-25T00:21:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.181+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fac3c062e40 con 0x7fac3c103490 2024-04-25T00:21:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.182+0000 7fac38ff9700 1 -- 172.21.15.45:0/1626591361 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3092 (secure 0 0 0) 0x7fac3000cd70 con 0x7fac3c103490 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:e27 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 27 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-25T00:19:33.221862+0000 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-25T00:20:01.835169+0000 2024-04-25T00:21:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 5 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2,3,4 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24500,1=24493,2=24478,3=24467,4=24460} 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-04-25T00:21:40.184 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3,4] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24500 members: 24467,24460,24478,24500,24493 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24500} state up:active seq 3 addr [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{0:24434} state up:standby-replay seq 1 addr [v2:172.21.15.71:6834/741071549,v1:172.21.15.71:6835/741071549] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{1:24493} state up:active seq 11 addr [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{1:24409} state up:standby-replay seq 1 addr [v2:172.21.15.122:6832/3066271880,v1:172.21.15.122:6833/3066271880] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{2:24478} state up:active seq 13 addr [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{2:24401} state up:standby-replay seq 1 addr [v2:172.21.15.71:6832/1511686553,v1:172.21.15.71:6833/1511686553] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{3:24467} state up:active seq 12 addr [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.185 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.45:6840/3316648689,v1:172.21.15.45:6841/3316648689] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{4:24460} state up:active seq 14 addr [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.45:6838/2848612989,v1:172.21.15.45:6839/2848612989] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.45:6834/3465528843,v1:172.21.15.45:6835/3465528843] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.45:6836/2745663812,v1:172.21.15.45:6837/2745663812] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 msgr2=0x7fac240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 0x7fac240a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b580 tx=0x7fac2c005fb0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 msgr2=0x7fac3c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e9d0 tx=0x7fac3000ece0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 shutdown_connections 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac3c103e30 0x7fac3c19fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fac240a2c20 0x7fac240a50e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3c102290 0x7fac3c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 --2- 172.21.15.45:0/1626591361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3c103490 0x7fac3c198a00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 >> 172.21.15.45:0/1626591361 conn(0x7fac3c0fd830 msgr2=0x7fac3c106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 shutdown_connections 2024-04-25T00:21:40.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:40.184+0000 7fac41f97700 1 -- 172.21.15.45:0/1626591361 wait complete. 2024-04-25T00:21:40.188 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:40.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T00:21:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:40 smithi045 ceph-mon[29194]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 359 B/s wr, 9 op/s 2024-04-25T00:21:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:40 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1626591361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:21:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:40 smithi122 ceph-mon[32959]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 359 B/s wr, 9 op/s 2024-04-25T00:21:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:40 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1626591361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:21:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:40 smithi071 ceph-mon[32667]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 359 B/s wr, 9 op/s 2024-04-25T00:21:41.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:40 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1626591361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:21:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- 172.21.15.45:0/670892762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878102290 msgr2=0x7ff8781026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 --2- 172.21.15.45:0/670892762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878102290 0x7ff8781026b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff8680099c0 tx=0x7ff868009cd0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- 172.21.15.45:0/670892762 shutdown_connections 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 --2- 172.21.15.45:0/670892762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 0x7ff87810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 --2- 172.21.15.45:0/670892762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878102290 0x7ff8781026b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 --2- 172.21.15.45:0/670892762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878103490 0x7ff8781038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- 172.21.15.45:0/670892762 >> 172.21.15.45:0/670892762 conn(0x7ff8780fd830 msgr2=0x7ff8780ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- 172.21.15.45:0/670892762 shutdown_connections 2024-04-25T00:21:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- 172.21.15.45:0/670892762 wait complete. 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 Processor -- start 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.958+0000 7ff87e333700 1 -- start start 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 0x7ff8781984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 0x7ff87819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810de80 con 0x7ff878103490 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810dd00 con 0x7ff878102290 2024-04-25T00:21:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810e000 con 0x7ff878103e30 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47756/0 (socket says 172.21.15.45:47756) 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 -- 172.21.15.45:0/2170786496 learned_addr learned my addr 172.21.15.45:0/2170786496 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff877fff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 0x7ff8781984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87c8d0700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 0x7ff87819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 -- 172.21.15.45:0/2170786496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 msgr2=0x7ff87819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 0x7ff87819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 -- 172.21.15.45:0/2170786496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 msgr2=0x7ff8781984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 0x7ff8781984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 -- 172.21.15.45:0/2170786496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781a0120 con 0x7ff878103490 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff877fff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 0x7ff8781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff86ffff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff86000e9d0 tx=0x7ff86000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86000cbd0 con 0x7ff878103490 2024-04-25T00:21:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86000ee70 con 0x7ff878103490 2024-04-25T00:21:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781a0410 con 0x7ff878103490 2024-04-25T00:21:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.959+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860018660 con 0x7ff878103490 2024-04-25T00:21:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.960+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a09a0 con 0x7ff878103490 2024-04-25T00:21:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.960+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff878068f20 con 0x7ff878103490 2024-04-25T00:21:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.962+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff860010430 con 0x7ff878103490 2024-04-25T00:21:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.962+0000 7ff875ffb700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 0x7ff8640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.963+0000 7ff877fff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 0x7ff8640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.963+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7ff8600cf760 con 0x7ff878103490 2024-04-25T00:21:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.963+0000 7ff877fff700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 0x7ff8640a5160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b580 tx=0x7ff868005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:42.966+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff86009c340 con 0x7ff878103490 2024-04-25T00:21:43.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:42 smithi045 ceph-mon[29194]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T00:21:43.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.176+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff87802db60 con 0x7ff878103490 2024-04-25T00:21:43.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.177+0000 7ff875ffb700 1 -- 172.21.15.45:0/2170786496 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5629 (secure 0 0 0) 0x7ff86000cd70 con 0x7ff878103490 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 93 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-25T00:11:31.853921+0000 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-25T00:21:27.913220+0000 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 27 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-04-25T00:21:43.178 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/2748889100,v1:172.21.15.45:6803/2748889100] [v2:172.21.15.45:6804/2748889100,v1:172.21.15.45:6805/2748889100] exists,up 47ff99bd-1736-4c56-8965-bd5199120838 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/3734145643,v1:172.21.15.71:6801/3734145643] [v2:172.21.15.71:6802/3734145643,v1:172.21.15.71:6803/3734145643] exists,up f0c603f9-9f3e-4af1-9a59-e3e887208af5 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/3829837658,v1:172.21.15.122:6801/3829837658] [v2:172.21.15.122:6802/3829837658,v1:172.21.15.122:6803/3829837658] exists,up e528381f-f657-4e55-a279-3f45e35fa585 2024-04-25T00:21:43.179 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/2950579455,v1:172.21.15.45:6811/2950579455] [v2:172.21.15.45:6812/2950579455,v1:172.21.15.45:6813/2950579455] exists,up d0f7644b-671b-4e42-b161-8895d46c48a7 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/2155408782,v1:172.21.15.71:6809/2155408782] [v2:172.21.15.71:6810/2155408782,v1:172.21.15.71:6811/2155408782] exists,up 40f29fc7-dd8a-45cc-b8b5-d837364a6001 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/2119403630,v1:172.21.15.122:6809/2119403630] [v2:172.21.15.122:6810/2119403630,v1:172.21.15.122:6811/2119403630] exists,up 1fb2eebb-1460-46c7-bdf2-aa80df2b595d 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/4007679117,v1:172.21.15.45:6819/4007679117] [v2:172.21.15.45:6820/4007679117,v1:172.21.15.45:6821/4007679117] exists,up 6624e936-d5b3-40f7-9a81-cdca44201b04 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/2471128651,v1:172.21.15.71:6817/2471128651] [v2:172.21.15.71:6818/2471128651,v1:172.21.15.71:6819/2471128651] exists,up 1e86e97d-8982-4535-9be0-961d692f1472 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/2592930089,v1:172.21.15.122:6817/2592930089] [v2:172.21.15.122:6818/2592930089,v1:172.21.15.122:6819/2592930089] exists,up 4a36bdfc-d527-4419-a2d2-03a09912a395 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/3783590854,v1:172.21.15.45:6827/3783590854] [v2:172.21.15.45:6828/3783590854,v1:172.21.15.45:6829/3783590854] exists,up 8856abc5-be19-468e-9205-df339f75e018 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/2371437490,v1:172.21.15.71:6825/2371437490] [v2:172.21.15.71:6826/2371437490,v1:172.21.15.71:6827/2371437490] exists,up d7602a6e-0267-4732-90f6-794201613999 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/1901931479,v1:172.21.15.122:6825/1901931479] [v2:172.21.15.122:6826/1901931479,v1:172.21.15.122:6827/1901931479] exists,up f89083ac-5fb2-4e72-ad19-20637464344f 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.16 [2,4] 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1a [8,1] 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1f [7,11] 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.e [5,9] 2024-04-25T00:21:43.180 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.2c [4,9] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.6 [5,8] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.f [9,8] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.17 [9,8] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.19 [9,10] 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1299966585 expires 2024-04-26T00:12:25.175459+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1299966585 expires 2024-04-26T00:12:25.175459+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3254971976 expires 2024-04-26T00:12:25.175459+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1950515838 expires 2024-04-26T00:11:58.120353+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2906156154 expires 2024-04-26T00:11:58.120353+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3376575480 expires 2024-04-26T00:12:25.175459+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/200010672 expires 2024-04-26T00:11:58.120353+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/200010672 expires 2024-04-26T00:11:58.120353+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2002798989 expires 2024-04-26T00:11:58.120353+0000 2024-04-25T00:21:43.181 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2389946368 expires 2024-04-26T00:12:25.175459+0000 2024-04-25T00:21:43.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 msgr2=0x7ff8640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:43.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 0x7ff8640a5160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b580 tx=0x7ff868005fb0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 msgr2=0x7ff878198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:43.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff86000e9d0 tx=0x7ff86000ece0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 shutdown_connections 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff878103e30 0x7ff87819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff8640a2ca0 0x7ff8640a5160 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878102290 0x7ff8781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 --2- 172.21.15.45:0/2170786496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff878103490 0x7ff878198a00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 >> 172.21.15.45:0/2170786496 conn(0x7ff8780fd830 msgr2=0x7ff878106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 shutdown_connections 2024-04-25T00:21:43.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:43.179+0000 7ff87e333700 1 -- 172.21.15.45:0/2170786496 wait complete. 2024-04-25T00:21:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:42 smithi122 ceph-mon[32959]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T00:21:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:42 smithi071 ceph-mon[32667]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T00:21:43.919 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T00:21:43.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:44.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/4272294700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc1074e0 msgr2=0x7fcbdc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/4272294700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc1098d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009ca0 tx=0x7fcbd402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/4272294700 shutdown_connections 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/4272294700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc1098d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/4272294700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/4272294700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 0x7fcbdc104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/4272294700 >> 172.21.15.45:0/4272294700 conn(0x7fcbdc0fbc30 msgr2=0x7fcbdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/4272294700 shutdown_connections 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/4272294700 wait complete. 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 Processor -- start 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.042+0000 7fcbe1ecd640 1 -- start start 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 0x7fcbdc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10d490 con 0x7fcbdc104bb0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10d310 con 0x7fcbdc102280 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbe1ecd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc10d610 con 0x7fcbdc1074e0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38610/0 (socket says 172.21.15.45:38610) 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 -- 172.21.15.45:0/560042800 learned_addr learned my addr 172.21.15.45:0/560042800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdb7fe640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 0x7fcbdc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdbfff640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 -- 172.21.15.45:0/560042800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc1074e0 msgr2=0x7fcbdc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 -- 172.21.15.45:0/560042800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 msgr2=0x7fcbdc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 0x7fcbdc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 -- 172.21.15.45:0/560042800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc19f6c0 con 0x7fcbdc104bb0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbdaffd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b450 tx=0x7fcbd000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.043+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0015020 con 0x7fcbdc104bb0 2024-04-25T00:21:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.044+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0011d70 con 0x7fcbdc104bb0 2024-04-25T00:21:44.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.044+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc19f9b0 con 0x7fcbdc104bb0 2024-04-25T00:21:44.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.044+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000f340 con 0x7fcbdc104bb0 2024-04-25T00:21:44.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.044+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc19ff30 con 0x7fcbdc104bb0 2024-04-25T00:21:44.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.045+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbdc0fd720 con 0x7fcbdc104bb0 2024-04-25T00:21:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.046+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbd0018020 con 0x7fcbdc104bb0 2024-04-25T00:21:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.046+0000 7fcbd8ff9640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 0x7fcbc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.047+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fcbd00cee80 con 0x7fcbdc104bb0 2024-04-25T00:21:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.047+0000 7fcbdb7fe640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 0x7fcbc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.047+0000 7fcbdb7fe640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 0x7fcbc40a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0058c0 tx=0x7fcbcc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.051+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcbd001f020 con 0x7fcbdc104bb0 2024-04-25T00:21:44.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:43 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2170786496' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:21:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.237+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbdc062340 con 0x7fcbdc104bb0 2024-04-25T00:21:44.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.239+0000 7fcbd8ff9640 1 -- 172.21.15.45:0/560042800 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24709 (secure 0 0 0) 0x7fcbd000d040 con 0x7fcbdc104bb0 2024-04-25T00:21:44.239 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:44.240 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:27.913220+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 msgr2=0x7fcbc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 0x7fcbc40a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0058c0 tx=0x7fcbcc00a2d0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 msgr2=0x7fcbdc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b450 tx=0x7fcbd000b910 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 shutdown_connections 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbc40a2b90 0x7fcbc40a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbdc1074e0 0x7fcbdc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbdc104bb0 0x7fcbdc197fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 --2- 172.21.15.45:0/560042800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbdc102280 0x7fcbdc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 >> 172.21.15.45:0/560042800 conn(0x7fcbdc0fbc30 msgr2=0x7fcbdc102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 shutdown_connections 2024-04-25T00:21:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.241+0000 7fcbe1ecd640 1 -- 172.21.15.45:0/560042800 wait complete. 2024-04-25T00:21:44.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:44.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:44.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:43 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2170786496' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:21:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.358+0000 7f318804d640 1 -- 172.21.15.45:0/2208233905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180104b90 msgr2=0x7f3180106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.358+0000 7f318804d640 1 --2- 172.21.15.45:0/2208233905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180104b90 0x7f3180106f80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3170009900 tx=0x7f317002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- 172.21.15.45:0/2208233905 shutdown_connections 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- 172.21.15.45:0/2208233905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 0x7f31801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- 172.21.15.45:0/2208233905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180104b90 0x7f3180106f80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- 172.21.15.45:0/2208233905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180102260 0x7f3180104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- 172.21.15.45:0/2208233905 >> 172.21.15.45:0/2208233905 conn(0x7f31800fbbf0 msgr2=0x7f31800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- 172.21.15.45:0/2208233905 shutdown_connections 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- 172.21.15.45:0/2208233905 wait complete. 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 Processor -- start 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- start start 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 0x7f3180100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 0x7f3180100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f318010d400 con 0x7f3180102260 2024-04-25T00:21:44.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f318010d280 con 0x7f3180104b90 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.359+0000 7f318804d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f318010d580 con 0x7f31801074c0 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38632/0 (socket says 172.21.15.45:38632) 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 -- 172.21.15.45:0/1580310598 learned_addr learned my addr 172.21.15.45:0/1580310598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f31865ea640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 0x7f3180100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f31855e8640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 0x7f3180100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 -- 172.21.15.45:0/1580310598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 msgr2=0x7f3180100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 0x7f3180100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 -- 172.21.15.45:0/1580310598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 msgr2=0x7f3180100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 0x7f3180100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 -- 172.21.15.45:0/1580310598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3180100fe0 con 0x7f3180102260 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f31855e8640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 0x7f3180100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3185de9640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d950 tx=0x7f317c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c017070 con 0x7f3180102260 2024-04-25T00:21:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317c009c30 con 0x7f3180102260 2024-04-25T00:21:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31801a1bf0 con 0x7f3180102260 2024-04-25T00:21:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c00f6b0 con 0x7f3180102260 2024-04-25T00:21:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.360+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31801a2110 con 0x7f3180102260 2024-04-25T00:21:44.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.361+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31800fd6e0 con 0x7f3180102260 2024-04-25T00:21:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.362+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f317c0109d0 con 0x7f3180102260 2024-04-25T00:21:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.363+0000 7f3176ffd640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 0x7f316c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.363+0000 7f31855e8640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 0x7f316c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.363+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f317c0cf010 con 0x7f3180102260 2024-04-25T00:21:44.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.363+0000 7f31855e8640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 0x7f316c0a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3170006010 tx=0x7f3170032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.367+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f317c009de0 con 0x7f3180102260 2024-04-25T00:21:44.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:43 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2170786496' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:21:44.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.551+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f318002d170 con 0x7f3180102260 2024-04-25T00:21:44.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.551+0000 7f3176ffd640 1 -- 172.21.15.45:0/1580310598 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f317c09bbf0 con 0x7f3180102260 2024-04-25T00:21:44.552 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:44.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.553+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 msgr2=0x7f316c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.553+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 0x7f316c0a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3170006010 tx=0x7f3170032cf0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.553+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 msgr2=0x7f3180101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.553+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d950 tx=0x7f317c00de10 comp rx=0 tx=0).stop 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 shutdown_connections 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f316c0a2b90 0x7f316c0a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31801074c0 0x7f3180100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3180104b90 0x7f3180100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 --2- 172.21.15.45:0/1580310598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3180102260 0x7f3180101bd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 >> 172.21.15.45:0/1580310598 conn(0x7f31800fbbf0 msgr2=0x7f3180102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 shutdown_connections 2024-04-25T00:21:44.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.554+0000 7f318804d640 1 -- 172.21.15.45:0/1580310598 wait complete. 2024-04-25T00:21:44.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:44.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 -- 172.21.15.45:0/2737882325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484102200 msgr2=0x7fe4841045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 --2- 172.21.15.45:0/2737882325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484102200 0x7fe4841045f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe474009900 tx=0x7fe47402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 -- 172.21.15.45:0/2737882325 shutdown_connections 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 --2- 172.21.15.45:0/2737882325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484107460 0x7fe4841098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 --2- 172.21.15.45:0/2737882325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 0x7fe484106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 --2- 172.21.15.45:0/2737882325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484102200 0x7fe4841045f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 -- 172.21.15.45:0/2737882325 >> 172.21.15.45:0/2737882325 conn(0x7fe4840fbbf0 msgr2=0x7fe4840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 -- 172.21.15.45:0/2737882325 shutdown_connections 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.670+0000 7fe48a742640 1 -- 172.21.15.45:0/2737882325 wait complete. 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 Processor -- start 2024-04-25T00:21:44.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 -- start start 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484102200 0x7fe484101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 0x7fe4841001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410d470 con 0x7fe484107460 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410d2f0 con 0x7fe484104b30 2024-04-25T00:21:44.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.671+0000 7fe48a742640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410d5f0 con 0x7fe484102200 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38650/0 (socket says 172.21.15.45:38650) 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 -- 172.21.15.45:0/1254822754 learned_addr learned my addr 172.21.15.45:0/1254822754 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe4837fe640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 0x7fe4841001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe483fff640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484102200 0x7fe484101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 -- 172.21.15.45:0/1254822754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484102200 msgr2=0x7fe484101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484102200 0x7fe484101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 -- 172.21.15.45:0/1254822754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 msgr2=0x7fe4841001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 0x7fe4841001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 -- 172.21.15.45:0/1254822754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe484100f80 con 0x7fe484107460 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe488cdf640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00cad0 tx=0x7fe47c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00ae20 con 0x7fe484107460 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47c005020 con 0x7fe484107460 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4841a1c20 con 0x7fe484107460 2024-04-25T00:21:44.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00f6f0 con 0x7fe484107460 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.672+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841a21c0 con 0x7fe484107460 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.673+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4840fd6e0 con 0x7fe484107460 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.674+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe47c0051d0 con 0x7fe484107460 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.675+0000 7fe4817fa640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 0x7fe46c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.675+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fe47c0cf670 con 0x7fe484107460 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.675+0000 7fe483fff640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 0x7fe46c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.675+0000 7fe483fff640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 0x7fe46c0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe474031440 tx=0x7fe4740315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.679+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe47c014030 con 0x7fe484107460 2024-04-25T00:21:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.861+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe4840623e0 con 0x7fe484107460 2024-04-25T00:21:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.862+0000 7fe4817fa640 1 -- 172.21.15.45:0/1254822754 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe47c09bc00 con 0x7fe484107460 2024-04-25T00:21:44.862 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.863+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 msgr2=0x7fe46c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.863+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 0x7fe46c0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe474031440 tx=0x7fe4740315b0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.863+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 msgr2=0x7fe4841006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.863+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00cad0 tx=0x7fe47c00cf90 comp rx=0 tx=0).stop 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 shutdown_connections 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe46c0a2c10 0x7fe46c0a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe484107460 0x7fe4841006f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe484104b30 0x7fe4841001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 --2- 172.21.15.45:0/1254822754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484102200 0x7fe484101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 >> 172.21.15.45:0/1254822754 conn(0x7fe4840fbbf0 msgr2=0x7fe484102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 shutdown_connections 2024-04-25T00:21:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.864+0000 7fe48a742640 1 -- 172.21.15.45:0/1254822754 wait complete. 2024-04-25T00:21:44.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 -- 172.21.15.45:0/4211457492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658101890 msgr2=0x7fe658101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/4211457492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658101890 0x7fe658101d10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 -- 172.21.15.45:0/4211457492 shutdown_connections 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/4211457492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658102250 0x7fe658108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/4211457492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658101890 0x7fe658101d10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/4211457492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6581006a0 0x7fe658100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 -- 172.21.15.45:0/4211457492 >> 172.21.15.45:0/4211457492 conn(0x7fe6580fbc30 msgr2=0x7fe6580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 -- 172.21.15.45:0/4211457492 shutdown_connections 2024-04-25T00:21:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.981+0000 7fe65f1b0640 1 -- 172.21.15.45:0/4211457492 wait complete. 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 Processor -- start 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 -- start start 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6581006a0 0x7fe65806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658101890 0x7fe65819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe65810c300 con 0x7fe658102250 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe65810c180 con 0x7fe6581006a0 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65f1b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe65810c480 con 0x7fe658101890 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38676/0 (socket says 172.21.15.45:38676) 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 -- 172.21.15.45:0/3670249346 learned_addr learned my addr 172.21.15.45:0/3670249346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe657fff640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658101890 0x7fe65819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65cf4c640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6581006a0 0x7fe65806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 -- 172.21.15.45:0/3670249346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658101890 msgr2=0x7fe65819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658101890 0x7fe65819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 -- 172.21.15.45:0/3670249346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6581006a0 msgr2=0x7fe65806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.982+0000 7fe65d74d640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6581006a0 0x7fe65806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe65d74d640 1 -- 172.21.15.45:0/3670249346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65819f670 con 0x7fe658102250 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe65d74d640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe65000eaa0 tx=0x7fe65000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65000cdb0 con 0x7fe658102250 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe650005020 con 0x7fe658102250 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6500116b0 con 0x7fe658102250 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65819f960 con 0x7fe658102250 2024-04-25T00:21:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.983+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65819fe30 con 0x7fe658102250 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.984+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658107a10 con 0x7fe658102250 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.985+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe650005560 con 0x7fe658102250 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.986+0000 7fe655ffb640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 0x7fe6400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.986+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fe6500cf560 con 0x7fe658102250 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.986+0000 7fe65cf4c640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 0x7fe6400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.986+0000 7fe65cf4c640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 0x7fe6400a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6581016f0 tx=0x7fe648005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:44.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:44.990+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe650016030 con 0x7fe658102250 2024-04-25T00:21:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:44 smithi045 ceph-mon[29194]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:44 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/560042800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:44 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1580310598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:44 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1254822754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.175+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe658062340 con 0x7fe658102250 2024-04-25T00:21:45.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.175+0000 7fe655ffb640 1 -- 172.21.15.45:0/3670249346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fe65009baf0 con 0x7fe658102250 2024-04-25T00:21:45.176 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 msgr2=0x7fe6400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 0x7fe6400a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6581016f0 tx=0x7fe648005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 msgr2=0x7fe65819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe65000eaa0 tx=0x7fe65000ef60 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 shutdown_connections 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe6400a2b90 0x7fe6400a5050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe658102250 0x7fe65819edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658101890 0x7fe65819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 --2- 172.21.15.45:0/3670249346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6581006a0 0x7fe65806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 >> 172.21.15.45:0/3670249346 conn(0x7fe6580fbc30 msgr2=0x7fe658104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 shutdown_connections 2024-04-25T00:21:45.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.177+0000 7fe65f1b0640 1 -- 172.21.15.45:0/3670249346 wait complete. 2024-04-25T00:21:45.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 -- 172.21.15.45:0/871669232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60101890 msgr2=0x7fbc60101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 --2- 172.21.15.45:0/871669232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60101890 0x7fbc60101d10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009900 tx=0x7fbc5002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 -- 172.21.15.45:0/871669232 shutdown_connections 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 --2- 172.21.15.45:0/871669232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60102250 0x7fbc60108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 --2- 172.21.15.45:0/871669232 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60101890 0x7fbc60101d10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 --2- 172.21.15.45:0/871669232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 0x7fbc60100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 -- 172.21.15.45:0/871669232 >> 172.21.15.45:0/871669232 conn(0x7fbc600fbc30 msgr2=0x7fbc600fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 -- 172.21.15.45:0/871669232 shutdown_connections 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.293+0000 7fbc682f5640 1 -- 172.21.15.45:0/871669232 wait complete. 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 Processor -- start 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 -- start start 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 0x7fbc601969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60101890 0x7fbc60196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010c320 con 0x7fbc60102250 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010c1a0 con 0x7fbc601006a0 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc682f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6010c4a0 con 0x7fbc60101890 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66892640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66892640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38700/0 (socket says 172.21.15.45:38700) 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66892640 1 -- 172.21.15.45:0/3395598194 learned_addr learned my addr 172.21.15.45:0/3395598194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66091640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 0x7fbc601969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc65890640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60101890 0x7fbc60196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66892640 1 -- 172.21.15.45:0/3395598194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60101890 msgr2=0x7fbc60196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.294+0000 7fbc66892640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60101890 0x7fbc60196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc66892640 1 -- 172.21.15.45:0/3395598194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 msgr2=0x7fbc601969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc66892640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 0x7fbc601969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc66892640 1 -- 172.21.15.45:0/3395598194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6019e5e0 con 0x7fbc60102250 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc66892640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800eb10 tx=0x7fbc5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58004b20 con 0x7fbc60102250 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc58004cc0 con 0x7fbc60102250 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6019e8d0 con 0x7fbc60102250 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58010760 con 0x7fbc60102250 2024-04-25T00:21:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.295+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6019ee70 con 0x7fbc60102250 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.297+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc601079c0 con 0x7fbc60102250 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.297+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc58005020 con 0x7fbc60102250 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.298+0000 7fbc577fe640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 0x7fbc4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.298+0000 7fbc66091640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 0x7fbc4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.298+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fbc580cfa30 con 0x7fbc60102250 2024-04-25T00:21:45.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.298+0000 7fbc66091640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 0x7fbc4c0a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.302+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc5809c610 con 0x7fbc60102250 2024-04-25T00:21:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:44 smithi122 ceph-mon[32959]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:44 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/560042800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:44 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1580310598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:44 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1254822754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:44 smithi071 ceph-mon[32667]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:21:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:44 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/560042800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:44 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1580310598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:44 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1254822754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.486+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbc6002d170 con 0x7fbc60102250 2024-04-25T00:21:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.487+0000 7fbc577fe640 1 -- 172.21.15.45:0/3395598194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbc5809bf10 con 0x7fbc60102250 2024-04-25T00:21:45.488 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 msgr2=0x7fbc4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 0x7fbc4c0a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c00b410 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 msgr2=0x7fbc6019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800eb10 tx=0x7fbc5800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 shutdown_connections 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbc4c0a2c10 0x7fbc4c0a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc60102250 0x7fbc6019df70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc60101890 0x7fbc60196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 --2- 172.21.15.45:0/3395598194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc601006a0 0x7fbc601969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 >> 172.21.15.45:0/3395598194 conn(0x7fbc600fbc30 msgr2=0x7fbc60104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 shutdown_connections 2024-04-25T00:21:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.489+0000 7fbc682f5640 1 -- 172.21.15.45:0/3395598194 wait complete. 2024-04-25T00:21:45.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3368857844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c841022c0 msgr2=0x7f9c841046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3368857844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c841022c0 0x7f9c841046b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3368857844 shutdown_connections 2024-04-25T00:21:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3368857844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c84107520 0x7f9c84109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3368857844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84104bf0 0x7f9c84106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3368857844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c841022c0 0x7f9c841046b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3368857844 >> 172.21.15.45:0/3368857844 conn(0x7f9c840fbc30 msgr2=0x7f9c840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.602+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3368857844 shutdown_connections 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3368857844 wait complete. 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 Processor -- start 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 -- start start 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c841022c0 0x7f9c84101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 0x7f9c841007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d410 con 0x7f9c84104bf0 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d290 con 0x7f9c84107520 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c8bf09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410d590 con 0x7f9c841022c0 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c894a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c894a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38720/0 (socket says 172.21.15.45:38720) 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.603+0000 7f9c894a4640 1 -- 172.21.15.45:0/3387915566 learned_addr learned my addr 172.21.15.45:0/3387915566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c89ca5640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c841022c0 0x7f9c84101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c8a4a6640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 0x7f9c841007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 -- 172.21.15.45:0/3387915566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c841022c0 msgr2=0x7f9c84101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c841022c0 0x7f9c84101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 -- 172.21.15.45:0/3387915566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 msgr2=0x7f9c841007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 0x7f9c841007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 -- 172.21.15.45:0/3387915566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84101040 con 0x7f9c84104bf0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c8a4a6640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 0x7f9c841007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c894a4640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800d770 tx=0x7f9c7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78017070 con 0x7f9c84104bf0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c78009c30 con 0x7f9c84104bf0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841a1c00 con 0x7f9c84104bf0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800f620 con 0x7f9c84104bf0 2024-04-25T00:21:45.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.604+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a2150 con 0x7f9c84104bf0 2024-04-25T00:21:45.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.605+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c840fd720 con 0x7f9c84104bf0 2024-04-25T00:21:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.607+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c7801a070 con 0x7f9c84104bf0 2024-04-25T00:21:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.608+0000 7f9c82ffd640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 0x7f9c6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.608+0000 7f9c89ca5640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 0x7f9c6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.608+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f9c780cef90 con 0x7f9c84104bf0 2024-04-25T00:21:45.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.608+0000 7f9c89ca5640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 0x7f9c6c0a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000c00 tx=0x7f9c7c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.611+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c78009de0 con 0x7f9c84104bf0 2024-04-25T00:21:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.836+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c8402d170 con 0x7f9c84104bf0 2024-04-25T00:21:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.837+0000 7f9c82ffd640 1 -- 172.21.15.45:0/3387915566 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7f9c7809bb10 con 0x7f9c84104bf0 2024-04-25T00:21:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:45.838 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:21:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 msgr2=0x7f9c6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 0x7f9c6c0a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000c00 tx=0x7f9c7c0315d0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 msgr2=0x7f9c84100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800d770 tx=0x7f9c7800dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 shutdown_connections 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9c6c0a2b90 0x7f9c6c0a5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c84107520 0x7f9c841007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c84104bf0 0x7f9c84100270 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 --2- 172.21.15.45:0/3387915566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c841022c0 0x7f9c84101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 >> 172.21.15.45:0/3387915566 conn(0x7f9c840fbc30 msgr2=0x7f9c84102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 shutdown_connections 2024-04-25T00:21:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.839+0000 7f9c8bf09640 1 -- 172.21.15.45:0/3387915566 wait complete. 2024-04-25T00:21:45.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 -- 172.21.15.45:0/2068440321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc1006a0 msgr2=0x7f9ccc100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/2068440321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc100ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009900 tx=0x7f9cc802a410 comp rx=0 tx=0).stop 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 -- 172.21.15.45:0/2068440321 shutdown_connections 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/2068440321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 0x7f9ccc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/2068440321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc101890 0x7f9ccc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/2068440321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc100ac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 -- 172.21.15.45:0/2068440321 >> 172.21.15.45:0/2068440321 conn(0x7f9ccc0fbc30 msgr2=0x7f9ccc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:45.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 -- 172.21.15.45:0/2068440321 shutdown_connections 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.955+0000 7f9cd35b0640 1 -- 172.21.15.45:0/2068440321 wait complete. 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 Processor -- start 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 -- start start 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 0x7f9ccc1a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10c2f0 con 0x7f9ccc101890 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10c170 con 0x7f9ccc1006a0 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd35b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10c470 con 0x7f9ccc102250 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38738/0 (socket says 172.21.15.45:38738) 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 -- 172.21.15.45:0/4082722261 learned_addr learned my addr 172.21.15.45:0/4082722261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd134c640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd1b4d640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 0x7f9ccc1a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 -- 172.21.15.45:0/4082722261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 msgr2=0x7f9ccc1a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 0x7f9ccc1a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 -- 172.21.15.45:0/4082722261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc1006a0 msgr2=0x7f9ccc06a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc06a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.956+0000 7f9cd0b4b640 1 -- 172.21.15.45:0/4082722261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1a1900 con 0x7f9ccc101890 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cd0b4b640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e9f0 tx=0x7f9cbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00cd80 con 0x7f9ccc101890 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc010910 con 0x7f9ccc101890 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1a1b90 con 0x7f9ccc101890 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc018990 con 0x7f9ccc101890 2024-04-25T00:21:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.957+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a20e0 con 0x7f9ccc101890 2024-04-25T00:21:45.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.958+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc1079c0 con 0x7f9ccc101890 2024-04-25T00:21:45.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.959+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9cbc010ac0 con 0x7f9ccc101890 2024-04-25T00:21:45.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.960+0000 7f9cc27fc640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 0x7f9cb80a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:45.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.960+0000 7f9cd134c640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 0x7f9cb80a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:45.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.960+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f9cbc0cf720 con 0x7f9ccc101890 2024-04-25T00:21:45.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.961+0000 7f9cd134c640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 0x7f9cb80a5080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8005ab0 tx=0x7f9cc8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:45.965+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9cbc09c300 con 0x7f9ccc101890 2024-04-25T00:21:46.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3670249346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3395598194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3387915566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.152+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ccc062340 con 0x7f9ccc101890 2024-04-25T00:21:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.154+0000 7f9cc27fc640 1 -- 172.21.15.45:0/4082722261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24709 (secure 0 0 0) 0x7f9cbc09bc00 con 0x7f9ccc101890 2024-04-25T00:21:46.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:46.155 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:27.913220+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 msgr2=0x7f9cb80a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 0x7f9cb80a5080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8005ab0 tx=0x7f9cc8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 msgr2=0x7f9ccc19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e9f0 tx=0x7f9cbc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 shutdown_connections 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f9cb80a2bc0 0x7f9cb80a5080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ccc102250 0x7f9ccc1a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ccc101890 0x7f9ccc19ebc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 --2- 172.21.15.45:0/4082722261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc1006a0 0x7f9ccc06a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 >> 172.21.15.45:0/4082722261 conn(0x7f9ccc0fbc30 msgr2=0x7f9ccc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 shutdown_connections 2024-04-25T00:21:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.156+0000 7f9cd35b0640 1 -- 172.21.15.45:0/4082722261 wait complete. 2024-04-25T00:21:46.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:46.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- 172.21.15.45:0/1146416247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4102230 msgr2=0x7fcbe4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/1146416247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4102230 0x7fcbe4108ac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009ca0 tx=0x7fcbdc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- 172.21.15.45:0/1146416247 shutdown_connections 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/1146416247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4102230 0x7fcbe4108ac0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/1146416247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4101870 0x7fcbe4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/1146416247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4100680 0x7fcbe4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- 172.21.15.45:0/1146416247 >> 172.21.15.45:0/1146416247 conn(0x7fcbe40fbbf0 msgr2=0x7fcbe40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- 172.21.15.45:0/1146416247 shutdown_connections 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- 172.21.15.45:0/1146416247 wait complete. 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 Processor -- start 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 -- start start 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.273+0000 7fcbec5ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4101870 0x7fcbe4109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 0x7fcbe4109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe410d420 con 0x7fcbe4100680 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe410d2a0 con 0x7fcbe4101870 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe410d5a0 con 0x7fcbe4102230 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38748/0 (socket says 172.21.15.45:38748) 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 -- 172.21.15.45:0/2963260525 learned_addr learned my addr 172.21.15.45:0/2963260525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbe9b69640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4101870 0x7fcbe4109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbeab6b640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 0x7fcbe4109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 -- 172.21.15.45:0/2963260525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 msgr2=0x7fcbe4109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 0x7fcbe4109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 -- 172.21.15.45:0/2963260525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4101870 msgr2=0x7fcbe4109640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4101870 0x7fcbe4109640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 -- 172.21.15.45:0/2963260525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe410a410 con 0x7fcbe4100680 2024-04-25T00:21:46.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbeab6b640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 0x7fcbe4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbea36a640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c970 tx=0x7fcbe000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0007c10 con 0x7fcbe4100680 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0007db0 con 0x7fcbe4100680 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe41a1b10 con 0x7fcbe4100680 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000f940 con 0x7fcbe4100680 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.274+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe41a2060 con 0x7fcbe4100680 2024-04-25T00:21:46.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.275+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe41079a0 con 0x7fcbe4100680 2024-04-25T00:21:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.276+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcbe000fae0 con 0x7fcbe4100680 2024-04-25T00:21:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.277+0000 7fcbdb7fe640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 0x7fcbd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.277+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fcbe00cf820 con 0x7fcbe4100680 2024-04-25T00:21:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.277+0000 7fcbe9b69640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 0x7fcbd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.277+0000 7fcbe9b69640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 0x7fcbd00a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbe410ad70 tx=0x7fcbd400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.281+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcbe009c490 con 0x7fcbe4100680 2024-04-25T00:21:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3670249346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3395598194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3387915566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:46.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3670249346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3395598194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:46.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3387915566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:46.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.465+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbe4062340 con 0x7fcbe4100680 2024-04-25T00:21:46.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.466+0000 7fcbdb7fe640 1 -- 172.21.15.45:0/2963260525 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fcbe009bd90 con 0x7fcbe4100680 2024-04-25T00:21:46.466 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 msgr2=0x7fcbd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 0x7fcbd00a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbe410ad70 tx=0x7fcbd400b390 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 msgr2=0x7fcbe410b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c970 tx=0x7fcbe000ce30 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 shutdown_connections 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fcbd00a2b90 0x7fcbd00a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4102230 0x7fcbe4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbe4101870 0x7fcbe4109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 --2- 172.21.15.45:0/2963260525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe4100680 0x7fcbe410b000 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 >> 172.21.15.45:0/2963260525 conn(0x7fcbe40fbbf0 msgr2=0x7fcbe4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 shutdown_connections 2024-04-25T00:21:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.468+0000 7fcbec5ce640 1 -- 172.21.15.45:0/2963260525 wait complete. 2024-04-25T00:21:46.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 -- 172.21.15.45:0/2760412837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494101890 msgr2=0x7f9494101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 --2- 172.21.15.45:0/2760412837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494101890 0x7f9494101d10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f947c009900 tx=0x7f947c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 -- 172.21.15.45:0/2760412837 shutdown_connections 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 --2- 172.21.15.45:0/2760412837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9494102250 0x7f9494108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 --2- 172.21.15.45:0/2760412837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494101890 0x7f9494101d10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 --2- 172.21.15.45:0/2760412837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94941006a0 0x7f9494100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 -- 172.21.15.45:0/2760412837 >> 172.21.15.45:0/2760412837 conn(0x7f94940fbc30 msgr2=0x7f94940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 -- 172.21.15.45:0/2760412837 shutdown_connections 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.584+0000 7f94997a2640 1 -- 172.21.15.45:0/2760412837 wait complete. 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 Processor -- start 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 -- start start 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 0x7f9494104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 0x7f9494102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f949410c350 con 0x7f9494102250 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f949410c1d0 con 0x7f9494101890 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94997a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f949410c4d0 con 0x7f94941006a0 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38756/0 (socket says 172.21.15.45:38756) 2024-04-25T00:21:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 -- 172.21.15.45:0/102753147 learned_addr learned my addr 172.21.15.45:0/102753147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f948bfff640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 0x7f9494102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f9492ffd640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 0x7f9494104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 -- 172.21.15.45:0/102753147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 msgr2=0x7f9494104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 0x7f9494104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 -- 172.21.15.45:0/102753147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 msgr2=0x7f9494102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 0x7f9494102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f94937fe640 1 -- 172.21.15.45:0/102753147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9494103a40 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f948bfff640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 0x7f9494102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.585+0000 7f9492ffd640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 0x7f9494104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f94937fe640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f948c00cce0 tx=0x7f948c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c00f450 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c00fad0 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941a1be0 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c018c10 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.586+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94941a20b0 con 0x7f9494102250 2024-04-25T00:21:46.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.587+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940689b0 con 0x7f9494102250 2024-04-25T00:21:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.588+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f948c00f5f0 con 0x7f9494102250 2024-04-25T00:21:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.589+0000 7f9490ff9640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 0x7f94800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.589+0000 7f9492ffd640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 0x7f94800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.589+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f948c0cfc00 con 0x7f9494102250 2024-04-25T00:21:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.589+0000 7f9492ffd640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 0x7f94800a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94941016f0 tx=0x7f9484005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.593+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f948c014030 con 0x7f9494102250 2024-04-25T00:21:46.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.776+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94941a2a50 con 0x7f9494102250 2024-04-25T00:21:46.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.777+0000 7f9490ff9640 1 -- 172.21.15.45:0/102753147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f948c09c780 con 0x7f9494102250 2024-04-25T00:21:46.777 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 msgr2=0x7f94800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 0x7f94800a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94941016f0 tx=0x7f9484005d30 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 msgr2=0x7f94941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f948c00cce0 tx=0x7f948c00a620 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 shutdown_connections 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f94800a2b90 0x7f94800a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9494102250 0x7f94941031b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9494101890 0x7f9494102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 --2- 172.21.15.45:0/102753147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94941006a0 0x7f9494104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 >> 172.21.15.45:0/102753147 conn(0x7f94940fbc30 msgr2=0x7f94940fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 shutdown_connections 2024-04-25T00:21:46.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.779+0000 7f94997a2640 1 -- 172.21.15.45:0/102753147 wait complete. 2024-04-25T00:21:46.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 -- 172.21.15.45:0/1467149471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c101890 msgr2=0x7f2b6c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 --2- 172.21.15.45:0/1467149471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c101890 0x7f2b6c101d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009900 tx=0x7f2b5c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 -- 172.21.15.45:0/1467149471 shutdown_connections 2024-04-25T00:21:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 --2- 172.21.15.45:0/1467149471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c102250 0x7f2b6c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 --2- 172.21.15.45:0/1467149471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c101890 0x7f2b6c101d10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 --2- 172.21.15.45:0/1467149471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 -- 172.21.15.45:0/1467149471 >> 172.21.15.45:0/1467149471 conn(0x7f2b6c0fbc30 msgr2=0x7f2b6c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.896+0000 7f2b73c29640 1 -- 172.21.15.45:0/1467149471 shutdown_connections 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 -- 172.21.15.45:0/1467149471 wait complete. 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 Processor -- start 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 -- start start 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 0x7f2b6c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10c320 con 0x7f2b6c102250 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10c1a0 con 0x7f2b6c101890 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b73c29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10c4a0 con 0x7f2b6c1006a0 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b721c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b721c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38778/0 (socket says 172.21.15.45:38778) 2024-04-25T00:21:46.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b721c6640 1 -- 172.21.15.45:0/3026121707 learned_addr learned my addr 172.21.15.45:0/3026121707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.897+0000 7f2b719c5640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b711c4640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 0x7f2b6c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 -- 172.21.15.45:0/3026121707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 msgr2=0x7f2b6c06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 -- 172.21.15.45:0/3026121707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 msgr2=0x7f2b6c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 0x7f2b6c19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 -- 172.21.15.45:0/3026121707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c19f740 con 0x7f2b6c102250 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b719c5640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b711c4640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 0x7f2b6c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b721c6640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400eaa0 tx=0x7f2b6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400cdb0 con 0x7f2b6c102250 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b64005020 con 0x7f2b6c102250 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c19fa30 con 0x7f2b6c102250 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b640116b0 con 0x7f2b6c102250 2024-04-25T00:21:46.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.898+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c19ff00 con 0x7f2b6c102250 2024-04-25T00:21:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.899+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c107a80 con 0x7f2b6c102250 2024-04-25T00:21:46.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.900+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b64005560 con 0x7f2b6c102250 2024-04-25T00:21:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.901+0000 7f2b62ffd640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 0x7f2b580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.901+0000 7f2b719c5640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 0x7f2b580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.901+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f2b640cf720 con 0x7f2b6c102250 2024-04-25T00:21:46.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.901+0000 7f2b719c5640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 0x7f2b580a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c1016f0 tx=0x7f2b68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:46.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:46.906+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2b64016030 con 0x7f2b6c102250 2024-04-25T00:21:47.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.091+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b6c062340 con 0x7f2b6c102250 2024-04-25T00:21:47.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.092+0000 7f2b62ffd640 1 -- 172.21.15.45:0/3026121707 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f2b6409c2a0 con 0x7f2b6c102250 2024-04-25T00:21:47.093 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 msgr2=0x7f2b580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 0x7f2b580a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c1016f0 tx=0x7f2b68005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 msgr2=0x7f2b6c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400eaa0 tx=0x7f2b6400ef60 comp rx=0 tx=0).stop 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 shutdown_connections 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2b580a2b90 0x7f2b580a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b6c102250 0x7f2b6c19ee80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b6c101890 0x7f2b6c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 --2- 172.21.15.45:0/3026121707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c1006a0 0x7f2b6c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 >> 172.21.15.45:0/3026121707 conn(0x7f2b6c0fbc30 msgr2=0x7f2b6c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 shutdown_connections 2024-04-25T00:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.094+0000 7f2b73c29640 1 -- 172.21.15.45:0/3026121707 wait complete. 2024-04-25T00:21:47.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:46 smithi045 ceph-mon[29194]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:47.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4082722261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:47.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2963260525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:46 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/102753147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 -- 172.21.15.45:0/2025717248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 msgr2=0x7f1720108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 --2- 172.21.15.45:0/2025717248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f1720108ae0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1718009ca0 tx=0x7f171802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 -- 172.21.15.45:0/2025717248 shutdown_connections 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 --2- 172.21.15.45:0/2025717248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f1720108ae0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 --2- 172.21.15.45:0/2025717248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1720101890 0x7f1720101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 --2- 172.21.15.45:0/2025717248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17201006a0 0x7f1720100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 -- 172.21.15.45:0/2025717248 >> 172.21.15.45:0/2025717248 conn(0x7f17200fbc30 msgr2=0x7f17200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 -- 172.21.15.45:0/2025717248 shutdown_connections 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.211+0000 7f1728638640 1 -- 172.21.15.45:0/2025717248 wait complete. 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 Processor -- start 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 -- start start 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17201006a0 0x7f1720197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 0x7f1720197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f172010c320 con 0x7f1720102250 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f172010c1a0 con 0x7f17201006a0 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1728638640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f172010c4a0 con 0x7f1720101890 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38802/0 (socket says 172.21.15.45:38802) 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 -- 172.21.15.45:0/1206539592 learned_addr learned my addr 172.21.15.45:0/1206539592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f17263d4640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17201006a0 0x7f1720197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 -- 172.21.15.45:0/1206539592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 msgr2=0x7f1720197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1725bd3640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 0x7f1720197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 0x7f1720197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 -- 172.21.15.45:0/1206539592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17201006a0 msgr2=0x7f1720197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17201006a0 0x7f1720197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 -- 172.21.15.45:0/1206539592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172019f6a0 con 0x7f1720102250 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1725bd3640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 0x7f1720197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.212+0000 7f1726bd5640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1718031ae0 tx=0x7f1718031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.213+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718003c30 con 0x7f1720102250 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.213+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1718003dd0 con 0x7f1720102250 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.213+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172019f930 con 0x7f1720102250 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.213+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171802fcb0 con 0x7f1720102250 2024-04-25T00:21:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.213+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172019fe80 con 0x7f1720102250 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.214+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17201a00b0 con 0x7f1720102250 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.215+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1718004050 con 0x7f1720102250 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.216+0000 7f17177fe640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 0x7f170c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.216+0000 7f17263d4640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 0x7f170c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.216+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f17180f0710 con 0x7f1720102250 2024-04-25T00:21:47.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.216+0000 7f17263d4640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 0x7f170c0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.220+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f17180bd2f0 con 0x7f1720102250 2024-04-25T00:21:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:46 smithi122 ceph-mon[32959]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4082722261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2963260525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:46 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/102753147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:46 smithi071 ceph-mon[32667]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T00:21:47.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4082722261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:47.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2963260525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:46 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/102753147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:47.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.405+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f17201a0280 con 0x7f1720102250 2024-04-25T00:21:47.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.406+0000 7f17177fe640 1 -- 172.21.15.45:0/1206539592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f17180bcbf0 con 0x7f1720102250 2024-04-25T00:21:47.407 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 msgr2=0x7f170c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 0x7f170c0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005850 comp rx=0 tx=0).stop 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 msgr2=0x7f172019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1718031ae0 tx=0x7f1718031ee0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 shutdown_connections 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f170c0a2b90 0x7f170c0a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1720102250 0x7f172019f030 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1720101890 0x7f1720197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 --2- 172.21.15.45:0/1206539592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17201006a0 0x7f1720197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 >> 172.21.15.45:0/1206539592 conn(0x7f17200fbc30 msgr2=0x7f17200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 shutdown_connections 2024-04-25T00:21:47.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.408+0000 7f1728638640 1 -- 172.21.15.45:0/1206539592 wait complete. 2024-04-25T00:21:47.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 -- 172.21.15.45:0/2184126755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c102230 msgr2=0x7f472c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 --2- 172.21.15.45:0/2184126755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c102230 0x7f472c108ac0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ca0 tx=0x7f472402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 -- 172.21.15.45:0/2184126755 shutdown_connections 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 --2- 172.21.15.45:0/2184126755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c102230 0x7f472c108ac0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 --2- 172.21.15.45:0/2184126755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c101870 0x7f472c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 --2- 172.21.15.45:0/2184126755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c100680 0x7f472c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 -- 172.21.15.45:0/2184126755 >> 172.21.15.45:0/2184126755 conn(0x7f472c0fbbf0 msgr2=0x7f472c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 -- 172.21.15.45:0/2184126755 shutdown_connections 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.527+0000 7f47332b3640 1 -- 172.21.15.45:0/2184126755 wait complete. 2024-04-25T00:21:47.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 Processor -- start 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 -- start start 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c101870 0x7f472c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 0x7f472c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10d420 con 0x7f472c100680 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10d2a0 con 0x7f472c102230 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f47332b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10d5a0 con 0x7f472c101870 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38822/0 (socket says 172.21.15.45:38822) 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 -- 172.21.15.45:0/2346099199 learned_addr learned my addr 172.21.15.45:0/2346099199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473084e640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c101870 0x7f472c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f4731850640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 0x7f472c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 -- 172.21.15.45:0/2346099199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c101870 msgr2=0x7f472c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c101870 0x7f472c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 -- 172.21.15.45:0/2346099199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 msgr2=0x7f472c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 0x7f472c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 -- 172.21.15.45:0/2346099199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19f630 con 0x7f472c100680 2024-04-25T00:21:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f4731850640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 0x7f472c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.528+0000 7f473104f640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e9c0 tx=0x7f471c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.529+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00cd80 con 0x7f472c100680 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.529+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471c0035f0 con 0x7f472c100680 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.529+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19f920 con 0x7f472c100680 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.529+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c010690 con 0x7f472c100680 2024-04-25T00:21:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.529+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c19fe20 con 0x7f472c100680 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.530+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c1079a0 con 0x7f472c100680 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.531+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f471c0037a0 con 0x7f472c100680 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.531+0000 7f472a7fc640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 0x7f47140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.532+0000 7f473084e640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 0x7f47140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.532+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f471c0cf8b0 con 0x7f472c100680 2024-04-25T00:21:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.532+0000 7f473084e640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 0x7f47140a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f472c198f20 tx=0x7f4720005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.536+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f471c09c490 con 0x7f472c100680 2024-04-25T00:21:47.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.758+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f472c02d170 con 0x7f472c100680 2024-04-25T00:21:47.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.759+0000 7f472a7fc640 1 -- 172.21.15.45:0/2346099199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11932 (secure 0 0 0) 0x7f471c09bd90 con 0x7f472c100680 2024-04-25T00:21:47.759 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:47.760 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:20:01.835169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24467,24460,24478,24500,24493]},"id":1}]} 2024-04-25T00:21:47.760 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T00:21:47.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 msgr2=0x7f47140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 0x7f47140a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f472c198f20 tx=0x7f4720005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 msgr2=0x7f472c197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e9c0 tx=0x7f471c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 shutdown_connections 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f47140a2b90 0x7f47140a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f472c102230 0x7f472c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f472c101870 0x7f472c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 --2- 172.21.15.45:0/2346099199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f472c100680 0x7f472c197a10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 >> 172.21.15.45:0/2346099199 conn(0x7f472c0fbbf0 msgr2=0x7f472c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 shutdown_connections 2024-04-25T00:21:47.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.761+0000 7f47332b3640 1 -- 172.21.15.45:0/2346099199 wait complete. 2024-04-25T00:21:47.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 -- 172.21.15.45:0/2206094649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf141006a0 msgr2=0x7faf14100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 --2- 172.21.15.45:0/2206094649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf141006a0 0x7faf14100ac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf0402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 -- 172.21.15.45:0/2206094649 shutdown_connections 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 --2- 172.21.15.45:0/2206094649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14102250 0x7faf14108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 --2- 172.21.15.45:0/2206094649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf14101890 0x7faf14101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 --2- 172.21.15.45:0/2206094649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf141006a0 0x7faf14100ac0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 -- 172.21.15.45:0/2206094649 >> 172.21.15.45:0/2206094649 conn(0x7faf140fbc30 msgr2=0x7faf140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 -- 172.21.15.45:0/2206094649 shutdown_connections 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.874+0000 7faf19b90640 1 -- 172.21.15.45:0/2206094649 wait complete. 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 Processor -- start 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 -- start start 2024-04-25T00:21:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141006a0 0x7faf14104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 0x7faf14102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410c2f0 con 0x7faf14102250 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410c170 con 0x7faf141006a0 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf19b90640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410c470 con 0x7faf14101890 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf13fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38836/0 (socket says 172.21.15.45:38836) 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf13fff640 1 -- 172.21.15.45:0/3685734574 learned_addr learned my addr 172.21.15.45:0/3685734574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf12ffd640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 0x7faf14102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.875+0000 7faf137fe640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141006a0 0x7faf14104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 -- 172.21.15.45:0/3685734574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 msgr2=0x7faf14102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 0x7faf14102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 -- 172.21.15.45:0/3685734574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141006a0 msgr2=0x7faf14104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141006a0 0x7faf14104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 -- 172.21.15.45:0/3685734574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf14103a40 con 0x7faf14102250 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf12ffd640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 0x7faf14102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf13fff640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00eaa0 tx=0x7faf0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00cdb0 con 0x7faf14102250 2024-04-25T00:21:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0c005020 con 0x7faf14102250 2024-04-25T00:21:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a1c60 con 0x7faf14102250 2024-04-25T00:21:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0116b0 con 0x7faf14102250 2024-04-25T00:21:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.876+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a2160 con 0x7faf14102250 2024-04-25T00:21:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.877+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf141a2390 con 0x7faf14102250 2024-04-25T00:21:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.878+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf0c005560 con 0x7faf14102250 2024-04-25T00:21:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.879+0000 7faf10ff9640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 0x7faefc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.879+0000 7faf137fe640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 0x7faefc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.879+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7faf0c0cf600 con 0x7faf14102250 2024-04-25T00:21:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.879+0000 7faf137fe640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 0x7faefc0a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ab0 tx=0x7faf04000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:47.883+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf0c016030 con 0x7faf14102250 2024-04-25T00:21:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.070+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf141a2720 con 0x7faf14102250 2024-04-25T00:21:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.071+0000 7faf10ff9640 1 -- 172.21.15.45:0/3685734574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24709 (secure 0 0 0) 0x7faf0c09bb90 con 0x7faf14102250 2024-04-25T00:21:48.072 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:27.913220+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 msgr2=0x7faefc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 0x7faefc0a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ab0 tx=0x7faf04000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 msgr2=0x7faf141031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00eaa0 tx=0x7faf0c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 shutdown_connections 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7faefc0a2b90 0x7faefc0a5050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14102250 0x7faf141031b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf14101890 0x7faf14102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 --2- 172.21.15.45:0/3685734574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141006a0 0x7faf14104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.073+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 >> 172.21.15.45:0/3685734574 conn(0x7faf140fbc30 msgr2=0x7faf140fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.074+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 shutdown_connections 2024-04-25T00:21:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.074+0000 7faf19b90640 1 -- 172.21.15.45:0/3685734574 wait complete. 2024-04-25T00:21:48.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3026121707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1206539592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:47 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2346099199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:48.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:19:33.221862+0000', 'modified': '2024-04-25T00:20:01.835169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24493, 'mds_2': 24478, 'mds_3': 24467, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/2848612989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 2848612989}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 2848612989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6841/3316648689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 3316648689}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 3316648689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/1511686553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 1511686553}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 1511686553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6833/3066271880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 3066271880}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 3066271880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/741071549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 741071549}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 741071549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.122:6835/1025393130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 1025393130}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 1025393130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.71:6837/2635562743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 2635562743}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 2635562743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.122:6837/3810708134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 3810708134}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 3810708134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.122:6839/2298542043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 2298542043}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 2298542043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/102345855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 102345855}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 102345855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24467, 24460, 24478, 24500, 24493]} 2024-04-25T00:21:48.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T00:21:48.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T00:21:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 -- 172.21.15.45:0/2397593814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d4102990 msgr2=0x7f99d4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 --2- 172.21.15.45:0/2397593814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d4102990 0x7f99d4102e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009900 tx=0x7f99c402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 -- 172.21.15.45:0/2397593814 shutdown_connections 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 --2- 172.21.15.45:0/2397593814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4103350 0x7f99d4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 --2- 172.21.15.45:0/2397593814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d4102990 0x7f99d4102e10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 --2- 172.21.15.45:0/2397593814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d41017a0 0x7f99d4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 -- 172.21.15.45:0/2397593814 >> 172.21.15.45:0/2397593814 conn(0x7f99d40fcd10 msgr2=0x7f99d40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 -- 172.21.15.45:0/2397593814 shutdown_connections 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.188+0000 7f99db293640 1 -- 172.21.15.45:0/2397593814 wait complete. 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 Processor -- start 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 -- start start 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 0x7f99d4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 0x7f99d419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99d410d400 con 0x7f99d41017a0 2024-04-25T00:21:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f99d410d280 con 0x7f99d4103350 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99db293640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99d410d580 con 0x7f99d4102990 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38860/0 (socket says 172.21.15.45:38860) 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 -- 172.21.15.45:0/66682198 learned_addr learned my addr 172.21.15.45:0/66682198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d882e640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 0x7f99d4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d9830640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 0x7f99d419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 -- 172.21.15.45:0/66682198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 msgr2=0x7f99d4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 0x7f99d4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 -- 172.21.15.45:0/66682198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 msgr2=0x7f99d419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 0x7f99d419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d902f640 1 -- 172.21.15.45:0/66682198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d419f6e0 con 0x7f99d41017a0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d9830640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 0x7f99d419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.189+0000 7f99d882e640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 0x7f99d4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99d902f640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99d000d950 tx=0x7f99d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99ca7fc640 1 -- 172.21.15.45:0/66682198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0017020 con 0x7f99d41017a0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99ca7fc640 1 -- 172.21.15.45:0/66682198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d0009c30 con 0x7f99d41017a0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d419f9d0 con 0x7f99d41017a0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99ca7fc640 1 -- 172.21.15.45:0/66682198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d000f720 con 0x7f99d41017a0 2024-04-25T00:21:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.190+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d419fef0 con 0x7f99d41017a0 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.192+0000 7f99ca7fc640 1 -- 172.21.15.45:0/66682198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99d001a020 con 0x7f99d41017a0 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.192+0000 7f99ddf9f740 1 Processor -- start 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.192+0000 7f99ddf9f740 1 -- start start 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480eeec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480ef6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559480e4df00 con 0x559480eef240 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559480df6750 con 0x559480eee850 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559480e49ec0 con 0x559480ef43b0 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38872/0 (socket says 172.21.15.45:38872) 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 -- 172.21.15.45:0/1897073388 learned_addr learned my addr 172.21.15.45:0/1897073388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d882e640 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480ef6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d9830640 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480eeec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ca7fc640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 0x7f99c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 -- 172.21.15.45:0/1897073388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 msgr2=0x559480ef6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480ef6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 -- 172.21.15.45:0/1897073388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 msgr2=0x559480eeec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d9830640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 0x7f99c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480eeec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 -- 172.21.15.45:0/1897073388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559480b622d0 con 0x559480eef240 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99ca7fc640 1 -- 172.21.15.45:0/66682198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f99d00cf330 con 0x7f99d41017a0 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.193+0000 7f99d902f640 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f99d00cca90 tx=0x7f99d0020b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ed56352bbbd5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99d9830640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 0x7f99c00a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x559480eefaa0 tx=0x7f99cc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99bf7fe640 1 -- 172.21.15.45:0/1897073388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d009b3f0 con 0x559480eef240 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99bf7fe640 1 -- 172.21.15.45:0/1897073388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d007f1f0 con 0x559480eef240 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99bf7fe640 1 -- 172.21.15.45:0/1897073388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d009b210 con 0x559480eef240 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1897073388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 msgr2=0x559480ef3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f99d00cca90 tx=0x7f99d0020b80 comp rx=0 tx=0).stop 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1897073388 shutdown_connections 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480ef6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef3d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1897073388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480eeec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1897073388 >> 172.21.15.45:0/1897073388 conn(0x559480ee5ab0 msgr2=0x559480ee5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1897073388 shutdown_connections 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.194+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1897073388 wait complete. 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 -- start start 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 Processor -- start 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559480ef7300 con 0x559480ef43b0 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559480e09cf0 con 0x559480eef240 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559480ef7480 con 0x559480eee850 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99ddf9f740 10 client.0 _ll_register_callbacks cb 0x559480e096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99d9830640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99d9830640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38882/0 (socket says 172.21.15.45:38882) 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99d9830640 1 -- 172.21.15.45:0/2219921176 learned_addr learned my addr 172.21.15.45:0/2219921176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99d902f640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.195+0000 7f99d882e640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 -- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480eee850 msgr2=0x559480ef8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 -- 172.21.15.45:0/2219921176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 msgr2=0x559480ef9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 -- 172.21.15.45:0/2219921176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559480c2c6b0 con 0x559480ef43b0 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d882e640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99d9830640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f99cc011b80 tx=0x7f99cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb0b84e396b33f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc024070 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f99cc010cd0 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d007f1f0 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99cc019070 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99ddf9f740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559480f03200 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d007f1f0 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.196+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f99cc0123f0 con 0x559480ef43b0 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 10 client.15444 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 20 client.15444 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f058e0 0x559480f06d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 4 client.15444 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559480e5b440 con 0x559480f058e0 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99d902f640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f058e0 0x559480f06d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99d902f640 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f058e0 0x559480f06d50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45691be6d9aa51ec server_cookie=84fdae9fc1b26c90 in_seq=0 out_seq=0 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.197+0000 7f99bcff9640 10 client.15444 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99bcff9640 1 -- 172.21.15.45:0/2219921176 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559480e5b440 con 0x559480f058e0 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99bcff9640 10 client.15444 handle_command_reply: tid=0 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 client.15444 shutdown 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 msgr2=0x559480f00150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f99cc011b80 tx=0x7f99cc004a40 comp rx=0 tx=0).stop 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 shutdown_connections 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f058e0 0x559480f06d50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480ef43b0 0x559480f00150 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99bcff9640 0 client.15444 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2219921176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 >> 172.21.15.45:0/2219921176 conn(0x559480ee5ab0 msgr2=0x559480ee4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 shutdown_connections 2024-04-25T00:21:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2219921176 wait complete. 2024-04-25T00:21:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.198+0000 7f99ddf9f740 20 client.15444 trim_cache size 0 max 16384 2024-04-25T00:21:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 Processor -- start 2024-04-25T00:21:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 -- start start 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480f00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559480f05090 con 0x559480eef240 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559480f05760 con 0x559480eee850 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559480f05e30 con 0x559480ef43b0 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38886/0 (socket says 172.21.15.45:38886) 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 -- 172.21.15.45:0/2421069887 learned_addr learned my addr 172.21.15.45:0/2421069887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d902f640 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 -- 172.21.15.45:0/2421069887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 msgr2=0x559480f00070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480f00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 -- 172.21.15.45:0/2421069887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 msgr2=0x559480ef8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 -- 172.21.15.45:0/2421069887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559480eefad0 con 0x559480eef240 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d902f640 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.217+0000 7f99d882e640 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f99c4005940 tx=0x7f99c4032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd00793e80316a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99beffd640 1 -- 172.21.15.45:0/2421069887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4035040 con 0x559480eef240 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99beffd640 1 -- 172.21.15.45:0/2421069887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c40318b0 con 0x559480eef240 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99beffd640 1 -- 172.21.15.45:0/2421069887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4005650 con 0x559480eef240 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2421069887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 msgr2=0x559480ef9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f99c4005940 tx=0x7f99c4032f00 comp rx=0 tx=0).stop 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2421069887 shutdown_connections 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480f00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eef240 0x559480ef9180 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/2421069887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eee850 0x559480ef8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2421069887 >> 172.21.15.45:0/2421069887 conn(0x559480ee5ab0 msgr2=0x559480ee5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2421069887 shutdown_connections 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.218+0000 7f99ddf9f740 1 -- 172.21.15.45:0/2421069887 wait complete. 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 -- start start 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 Processor -- start 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480efc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480eff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559480f06bd0 con 0x559480eee850 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559480f06500 con 0x559480eef240 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559480f072a0 con 0x559480ef43b0 2024-04-25T00:21:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99ddf9f740 10 client.0 _ll_register_callbacks cb 0x7f99cc011850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38900/0 (socket says 172.21.15.45:38900) 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 -- 172.21.15.45:0/1946482 learned_addr learned my addr 172.21.15.45:0/1946482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d9830640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480eff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d882e640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480efc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 -- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 msgr2=0x559480eff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480eff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 -- 172.21.15.45:0/1946482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 msgr2=0x559480efc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480efc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d902f640 1 -- 172.21.15.45:0/1946482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559480eff9b0 con 0x559480eee850 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.219+0000 7f99d882e640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480efc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99d902f640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f99d0010e70 tx=0x7f99d0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087804abdd3e6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d007f1f0 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f99d0097740 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c40318b0 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99d0009de0 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99ddf9f740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559480f6be00 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.220+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c40318b0 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f99d0097f20 con 0x559480eee850 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 10 client.15450 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 20 client.15450 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f705b0 0x559480f6f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 4 client.15450 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "status"}) -- 0x559480e55790 con 0x559480f705b0 2024-04-25T00:21:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99d882e640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f705b0 0x559480f6f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99d882e640 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f705b0 0x559480f6f2c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31072aab56e8cdaa server_cookie=c760a56596961cd7 in_seq=0 out_seq=0 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.221+0000 7f99bcff9640 10 client.15450 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99bcff9640 1 -- 172.21.15.45:0/1946482 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559480e55790 con 0x559480f705b0 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99bcff9640 10 client.15450 handle_command_reply: tid=0 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 client.15450 shutdown 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 msgr2=0x559480efc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f99d0010e70 tx=0x7f99d0086040 comp rx=0 tx=0).stop 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 shutdown_connections 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x559480f705b0 0x559480f6f2c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559480ef43b0 0x559480eff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559480eef240 0x559480efc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99bcff9640 0 client.15450 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 --2- 172.21.15.45:0/1946482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559480eee850 0x559480efc2a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 >> 172.21.15.45:0/1946482 conn(0x559480ee5ab0 msgr2=0x559480ef0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 shutdown_connections 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 1 -- 172.21.15.45:0/1946482 wait complete. 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99ddf9f740 20 client.15450 trim_cache size 0 max 16384 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 4, 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24460, 2024-04-25T00:21:48.230 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 107.24213419599999, 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 157.78347030200001 2024-04-25T00:21:48.231 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 msgr2=0x7f99c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 0x7f99c00a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x559480eefaa0 tx=0x7f99cc004ab0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 msgr2=0x7f99d4197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.222+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99d000d950 tx=0x7f99d000de10 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 shutdown_connections 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f99c00a2b90 0x7f99c00a5050 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99d4103350 0x7f99d419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99d4102990 0x7f99d4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 --2- 172.21.15.45:0/66682198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99d41017a0 0x7f99d4197ac0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 >> 172.21.15.45:0/66682198 conn(0x7f99d40fcd10 msgr2=0x7f99d4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 shutdown_connections 2024-04-25T00:21:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.223+0000 7f99db293640 1 -- 172.21.15.45:0/66682198 wait complete. 2024-04-25T00:21:48.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:21:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3026121707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1206539592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:47 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2346099199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:48.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 -- 172.21.15.45:0/3072088711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1022c0 msgr2=0x7f66bc1046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 --2- 172.21.15.45:0/3072088711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1022c0 0x7f66bc1046b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009900 tx=0x7f66ac02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 -- 172.21.15.45:0/3072088711 shutdown_connections 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 --2- 172.21.15.45:0/3072088711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 0x7f66bc109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 --2- 172.21.15.45:0/3072088711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc104bf0 0x7f66bc106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 --2- 172.21.15.45:0/3072088711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1022c0 0x7f66bc1046b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 -- 172.21.15.45:0/3072088711 >> 172.21.15.45:0/3072088711 conn(0x7f66bc0fbbf0 msgr2=0x7f66bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 -- 172.21.15.45:0/3072088711 shutdown_connections 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.336+0000 7f66c1134640 1 -- 172.21.15.45:0/3072088711 wait complete. 2024-04-25T00:21:48.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 Processor -- start 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 -- start start 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc1022c0 0x7f66bc101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 0x7f66bc1007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10d400 con 0x7f66bc104bf0 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10d280 con 0x7f66bc1022c0 2024-04-25T00:21:48.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.337+0000 7f66c1134640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10d580 con 0x7f66bc107520 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38910/0 (socket says 172.21.15.45:38910) 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 -- 172.21.15.45:0/2546573293 learned_addr learned my addr 172.21.15.45:0/2546573293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc1022c0 0x7f66bc101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66bb59e640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 0x7f66bc1007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 -- 172.21.15.45:0/2546573293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 msgr2=0x7f66bc1007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 0x7f66bc1007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 -- 172.21.15.45:0/2546573293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc1022c0 msgr2=0x7f66bc101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc1022c0 0x7f66bc101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 -- 172.21.15.45:0/2546573293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc101040 con 0x7f66bc104bf0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f66b000e9c0 tx=0x7f66b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66a3fff640 1 -- 172.21.15.45:0/2546573293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000cd80 con 0x7f66bc104bf0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc1a1b90 con 0x7f66bc104bf0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66a3fff640 1 -- 172.21.15.45:0/2546573293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b0005500 con 0x7f66bc104bf0 2024-04-25T00:21:48.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66a3fff640 1 -- 172.21.15.45:0/2546573293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0010690 con 0x7f66bc104bf0 2024-04-25T00:21:48.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.338+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1a2130 con 0x7f66bc104bf0 2024-04-25T00:21:48.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.340+0000 7f66a3fff640 1 -- 172.21.15.45:0/2546573293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66b00046c0 con 0x7f66bc104bf0 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 Processor -- start 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 -- start start 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bdb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc94b36f00 con 0x55cc94bd3e30 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cc94adf750 con 0x55cc94bd8fa0 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66c3e41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc94b32ec0 con 0x55cc94bd3440 2024-04-25T00:21:48.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38912/0 (socket says 172.21.15.45:38912) 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 -- 172.21.15.45:0/1953743894 learned_addr learned my addr 172.21.15.45:0/1953743894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bb59e640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66a3fff640 1 -- 172.21.15.45:0/2546573293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f66b00cf900 con 0x7f66bc104bf0 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bb59e640 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66ba59c640 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bdb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 -- 172.21.15.45:0/1953743894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 msgr2=0x55cc94bd3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 -- 172.21.15.45:0/1953743894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 msgr2=0x55cc94bdb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bdb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.341+0000 7f66bad9d640 1 -- 172.21.15.45:0/1953743894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9484b2d0 con 0x55cc94bd3e30 2024-04-25T00:21:48.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66bad9d640 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02a930 tx=0x7f66ac032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df6910ff581d112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66bb59e640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 0x7f66a40a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cc94bd4690 tx=0x7f66b4007410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66a17fa640 1 -- 172.21.15.45:0/1953743894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac03f070 con 0x55cc94bd3e30 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66a17fa640 1 -- 172.21.15.45:0/1953743894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ac0318b0 con 0x55cc94bd3e30 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66a17fa640 1 -- 172.21.15.45:0/1953743894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac035070 con 0x55cc94bd3e30 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66c3e41740 1 -- 172.21.15.45:0/1953743894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 msgr2=0x55cc94bd8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.342+0000 7f66c3e41740 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66ac02a930 tx=0x7f66ac032cb0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- 172.21.15.45:0/1953743894 shutdown_connections 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bdb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bd8930 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- 172.21.15.45:0/1953743894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- 172.21.15.45:0/1953743894 >> 172.21.15.45:0/1953743894 conn(0x55cc94bceab0 msgr2=0x55cc94bceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- 172.21.15.45:0/1953743894 shutdown_connections 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- 172.21.15.45:0/1953743894 wait complete. 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- start start 2024-04-25T00:21:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 Processor -- start 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3440 0x55cc94bdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc94bdbef0 con 0x55cc94bd3e30 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cc94af2cf0 con 0x55cc94bd3440 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc94bdc070 con 0x55cc94bd8fa0 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66c3e41740 10 client.0 _ll_register_callbacks cb 0x55cc94af26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38924/0 (socket says 172.21.15.45:38924) 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66ba59c640 1 -- 172.21.15.45:0/2674976246 learned_addr learned my addr 172.21.15.45:0/2674976246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.343+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3440 0x55cc94bdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66bb59e640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 -- 172.21.15.45:0/2674976246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 msgr2=0x55cc94be4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 -- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3440 msgr2=0x55cc94bdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3440 0x55cc94bdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 -- 172.21.15.45:0/2674976246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc949156b0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f66b00cd3f0 tx=0x7f66b009b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce43000ee8691566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00b9060 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f66b00a47c0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ac0318b0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66b009bde0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc94be7e60 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ac0318b0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66b009aca0 con 0x55cc94bd3e30 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 10 client.15462 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 20 client.15462 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94bea4c0 0x55cc94beb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 4 client.15462 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.344+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc94b44440 con 0x55cc94bea4c0 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.345+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94bea4c0 0x55cc94beb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.345+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94bea4c0 0x55cc94beb930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f26700aac7265c9 server_cookie=14cdd28c563d191b in_seq=0 out_seq=0 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.345+0000 7f669eff5640 10 client.15462 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f669eff5640 1 -- 172.21.15.45:0/2674976246 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc94b44440 con 0x55cc94bea4c0 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f669eff5640 10 client.15462 handle_command_reply: tid=0 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 client.15462 shutdown 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 msgr2=0x55cc94bddd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f66b00cd3f0 tx=0x7f66b009b9e0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 shutdown_connections 2024-04-25T00:21:48.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94bea4c0 0x55cc94beb930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94bddd70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f669eff5640 0 client.15462 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2674976246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3440 0x55cc94bdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 >> 172.21.15.45:0/2674976246 conn(0x55cc94bceab0 msgr2=0x55cc94bcdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 shutdown_connections 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 1 -- 172.21.15.45:0/2674976246 wait complete. 2024-04-25T00:21:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.346+0000 7f66c3e41740 20 client.15462 trim_cache size 0 max 16384 2024-04-25T00:21:48.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3026121707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1206539592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:48.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:47 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2346099199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 Processor -- start 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 -- start start 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bc9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc94be9c70 con 0x55cc94bd3e30 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cc94bea340 con 0x55cc94bd8fa0 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66c3e41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc94beaa10 con 0x55cc94bd3440 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38932/0 (socket says 172.21.15.45:38932) 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.364+0000 7f66ba59c640 1 -- 172.21.15.45:0/3579505718 learned_addr learned my addr 172.21.15.45:0/3579505718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66bad9d640 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 -- 172.21.15.45:0/3579505718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 msgr2=0x55cc94bd8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 -- 172.21.15.45:0/3579505718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 msgr2=0x55cc94bc9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bc9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 -- 172.21.15.45:0/3579505718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc94bd46c0 con 0x55cc94bd3e30 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66bad9d640 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66ba59c640 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f66b00987b0 tx=0x7f66b009a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900acb50a92b5610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66a0ff9640 1 -- 172.21.15.45:0/3579505718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00a16c0 con 0x55cc94bd3e30 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66a0ff9640 1 -- 172.21.15.45:0/3579505718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b00a0190 con 0x55cc94bd3e30 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66a0ff9640 1 -- 172.21.15.45:0/3579505718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b00a45c0 con 0x55cc94bd3e30 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 -- 172.21.15.45:0/3579505718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 msgr2=0x55cc94be4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f66b00987b0 tx=0x7f66b009a2f0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 -- 172.21.15.45:0/3579505718 shutdown_connections 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd8fa0 0x55cc94bc9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3e30 0x55cc94be4d70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.365+0000 7f66c3e41740 1 --2- 172.21.15.45:0/3579505718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd3440 0x55cc94bd8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- 172.21.15.45:0/3579505718 >> 172.21.15.45:0/3579505718 conn(0x55cc94bceab0 msgr2=0x55cc94bcee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- 172.21.15.45:0/3579505718 shutdown_connections 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- 172.21.15.45:0/3579505718 wait complete. 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- start start 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 Processor -- start 2024-04-25T00:21:48.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 0x55cc94be0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cc94beb7b0 con 0x55cc94bd3440 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cc94beb0e0 con 0x55cc94bd3e30 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc94bebe80 con 0x55cc94bd8fa0 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66c3e41740 10 client.0 _ll_register_callbacks cb 0x7f66b0098100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38938/0 (socket says 172.21.15.45:38938) 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66bad9d640 1 -- 172.21.15.45:0/2322584709 learned_addr learned my addr 172.21.15.45:0/2322584709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66bb59e640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.366+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 0x55cc94be0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 -- 172.21.15.45:0/2322584709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 msgr2=0x55cc94be3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 -- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 msgr2=0x55cc94be0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 0x55cc94be0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 -- 172.21.15.45:0/2322584709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc94be4520 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 0x55cc94be0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66bad9d640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009900 tx=0x7f66ac031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba88678438698654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac03f040 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f66ac004450 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b00a0190 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66ac059050 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66c3e41740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc94c50950 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b00a0190 con 0x55cc94bd3440 2024-04-25T00:21:48.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.367+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f66ac054030 con 0x55cc94bd3440 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 10 client.15468 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 20 client.15468 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94c55110 0x55cc94c53e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 4 client.15468 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "status"}) -- 0x55cc94b3e790 con 0x55cc94c55110 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94c55110 0x55cc94c53e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f669eff5640 10 client.15468 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66ba59c640 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94c55110 0x55cc94c53e20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5fa594a260ed872e server_cookie=1e716dc8d7f7f7fb in_seq=0 out_seq=0 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f669eff5640 1 -- 172.21.15.45:0/2322584709 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cc94b3e790 con 0x55cc94c55110 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f669eff5640 10 client.15468 handle_command_reply: tid=0 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 1 client.15468 shutdown 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 msgr2=0x55cc94be0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.368+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009900 tx=0x7f66ac031850 comp rx=0 tx=0).stop 2024-04-25T00:21:48.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 shutdown_connections 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55cc94c55110 0x55cc94c53e20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc94bd8fa0 0x55cc94be3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc94bd3e30 0x55cc94be0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 --2- 172.21.15.45:0/2322584709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cc94bd3440 0x55cc94be0a40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f669eff5640 0 client.15468 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 >> 172.21.15.45:0/2322584709 conn(0x55cc94bceab0 msgr2=0x55cc94bc9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 shutdown_connections 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 1 -- 172.21.15.45:0/2322584709 wait complete. 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c3e41740 20 client.15468 trim_cache size 0 max 16384 2024-04-25T00:21:48.378 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 3, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24467, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 108.395750939, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 152.13701275099999 2024-04-25T00:21:48.379 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 msgr2=0x7f66a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 0x7f66a40a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cc94bd4690 tx=0x7f66b4007410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 msgr2=0x7f66bc100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f66b000e9c0 tx=0x7f66b000ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 shutdown_connections 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f66a40a2b90 0x7f66a40a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66bc107520 0x7f66bc1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc104bf0 0x7f66bc100270 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 --2- 172.21.15.45:0/2546573293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66bc1022c0 0x7f66bc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 >> 172.21.15.45:0/2546573293 conn(0x7f66bc0fbbf0 msgr2=0x7f66bc102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 shutdown_connections 2024-04-25T00:21:48.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.369+0000 7f66c1134640 1 -- 172.21.15.45:0/2546573293 wait complete. 2024-04-25T00:21:48.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:21:48.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- 172.21.15.45:0/1646109693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe450102990 msgr2=0x7fe450102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- 172.21.15.45:0/1646109693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe450102990 0x7fe450102e10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe440009900 tx=0x7fe44002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- 172.21.15.45:0/1646109693 shutdown_connections 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- 172.21.15.45:0/1646109693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 0x7fe450109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- 172.21.15.45:0/1646109693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe450102990 0x7fe450102e10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- 172.21.15.45:0/1646109693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4501017a0 0x7fe450101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- 172.21.15.45:0/1646109693 >> 172.21.15.45:0/1646109693 conn(0x7fe4500fcd10 msgr2=0x7fe4500ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- 172.21.15.45:0/1646109693 shutdown_connections 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- 172.21.15.45:0/1646109693 wait complete. 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 Processor -- start 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- start start 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe450102990 0x7fe450197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 0x7fe45019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d470 con 0x7fe4501017a0 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d2f0 con 0x7fe450102990 2024-04-25T00:21:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.484+0000 7fe457309640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d5f0 con 0x7fe450103350 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38948/0 (socket says 172.21.15.45:38948) 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 -- 172.21.15.45:0/1413892452 learned_addr learned my addr 172.21.15.45:0/1413892452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 0x7fe45019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4548a4640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe450102990 0x7fe450197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 -- 172.21.15.45:0/1413892452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 msgr2=0x7fe45019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 0x7fe45019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 -- 172.21.15.45:0/1413892452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe450102990 msgr2=0x7fe450197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe450102990 0x7fe450197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 -- 172.21.15.45:0/1413892452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45019f6d0 con 0x7fe4501017a0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9c0 tx=0x7fe44c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4467fc640 1 -- 172.21.15.45:0/1413892452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00cd80 con 0x7fe4501017a0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4467fc640 1 -- 172.21.15.45:0/1413892452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44c0035f0 con 0x7fe4501017a0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe4467fc640 1 -- 172.21.15.45:0/1413892452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c010690 con 0x7fe4501017a0 2024-04-25T00:21:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45019f9c0 con 0x7fe4501017a0 2024-04-25T00:21:48.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.485+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45019fee0 con 0x7fe4501017a0 2024-04-25T00:21:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.487+0000 7fe4467fc640 1 -- 172.21.15.45:0/1413892452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe44c0037a0 con 0x7fe4501017a0 2024-04-25T00:21:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 Processor -- start 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 -- start start 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e26f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e274930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56050e1d2f00 con 0x56050e274fa0 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56050e17b750 con 0x56050e26f440 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe45a013740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56050e1ceec0 con 0x56050e26fe30 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4467fc640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4548a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4548a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38956/0 (socket says 172.21.15.45:38956) 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4548a4640 1 -- 172.21.15.45:0/1958241491 learned_addr learned my addr 172.21.15.45:0/1958241491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4467fc640 1 -- 172.21.15.45:0/1413892452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fe44c0cf930 con 0x7fe4501017a0 2024-04-25T00:21:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e274930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.488+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e26f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 -- 172.21.15.45:0/1958241491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 msgr2=0x56050e274930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e274930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 -- 172.21.15.45:0/1958241491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 msgr2=0x56050e26f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e26f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 -- 172.21.15.45:0/1958241491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050dee72d0 con 0x56050e274fa0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e274930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 0x7fe43c0a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe448009270 tx=0x7fe448012c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe4548a4640 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe440005ab0 tx=0x7fe4400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8272d2c3770d485d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe43b7fe640 1 -- 172.21.15.45:0/1958241491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440035030 con 0x56050e274fa0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe43b7fe640 1 -- 172.21.15.45:0/1958241491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe440005520 con 0x56050e274fa0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.489+0000 7fe43b7fe640 1 -- 172.21.15.45:0/1958241491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400316b0 con 0x56050e274fa0 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- 172.21.15.45:0/1958241491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 msgr2=0x56050e277430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe440005ab0 tx=0x7fe4400040c0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- 172.21.15.45:0/1958241491 shutdown_connections 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e277430 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e274930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 --2- 172.21.15.45:0/1958241491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e26f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- 172.21.15.45:0/1958241491 >> 172.21.15.45:0/1958241491 conn(0x56050e26aab0 msgr2=0x56050e26aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- 172.21.15.45:0/1958241491 shutdown_connections 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- 172.21.15.45:0/1958241491 wait complete. 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 -- start start 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.490+0000 7fe45a013740 1 Processor -- start 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e279840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 0x56050e280dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56050e277ef0 con 0x56050e26fe30 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56050e18ecf0 con 0x56050e26f440 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56050e278070 con 0x56050e274fa0 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 10 client.0 _ll_register_callbacks cb 0x56050e18e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38958/0 (socket says 172.21.15.45:38958) 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 -- 172.21.15.45:0/2468218521 learned_addr learned my addr 172.21.15.45:0/2468218521 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4550a5640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e279840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4558a6640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 0x56050e280dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 -- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 msgr2=0x56050e280dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 0x56050e280dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 -- 172.21.15.45:0/2468218521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 msgr2=0x56050e279840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e279840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 -- 172.21.15.45:0/2468218521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050dfb16b0 con 0x56050e26fe30 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4558a6640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 0x56050e280dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe4548a4640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe440031610 tx=0x7fe4400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c3bec95f39c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440052060 con 0x56050e26fe30 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fe440004fe0 con 0x56050e26fe30 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.491+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe440005520 con 0x56050e26fe30 2024-04-25T00:21:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe440059050 con 0x56050e26fe30 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56050e283d10 con 0x56050e26fe30 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe440005520 con 0x56050e26fe30 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4400035f0 con 0x56050e26fe30 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 10 client.15480 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 20 client.15480 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e285ef0 0x56050e287360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 4 client.15480 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050e1e0440 con 0x56050e285ef0 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.492+0000 7fe4550a5640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e285ef0 0x56050e287360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe438ff9640 10 client.15480 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:48.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe4550a5640 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e285ef0 0x56050e287360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99f409b86186a24b server_cookie=8271e456fcc6d77f in_seq=0 out_seq=0 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe438ff9640 1 -- 172.21.15.45:0/2468218521 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050e1e0440 con 0x56050e285ef0 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe438ff9640 10 client.15480 handle_command_reply: tid=0 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe45a013740 1 client.15480 shutdown 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 msgr2=0x56050e279d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.493+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe440031610 tx=0x7fe4400048f0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 shutdown_connections 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e285ef0 0x56050e287360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e274fa0 0x56050e280dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e26fe30 0x56050e279d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 --2- 172.21.15.45:0/2468218521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e279840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 >> 172.21.15.45:0/2468218521 conn(0x56050e26aab0 msgr2=0x56050e269ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe438ff9640 0 client.15480 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 shutdown_connections 2024-04-25T00:21:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 1 -- 172.21.15.45:0/2468218521 wait complete. 2024-04-25T00:21:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.494+0000 7fe45a013740 20 client.15480 trim_cache size 0 max 16384 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 Processor -- start 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 -- start start 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e265690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e280d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56050e2856a0 con 0x56050e274fa0 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56050e285d70 con 0x56050e26f440 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe45a013740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56050e286440 con 0x56050e26fe30 2024-04-25T00:21:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38970/0 (socket says 172.21.15.45:38970) 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 -- 172.21.15.45:0/1215726484 learned_addr learned my addr 172.21.15.45:0/1215726484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e265690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4548a4640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e280d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 -- 172.21.15.45:0/1215726484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 msgr2=0x56050e280d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e280d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 -- 172.21.15.45:0/1215726484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 msgr2=0x56050e265690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e265690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.512+0000 7fe4558a6640 1 -- 172.21.15.45:0/1215726484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050e270210 con 0x56050e274fa0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe4550a5640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e265690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe4558a6640 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe448012190 tx=0x7fe44801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb463aafb28b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe43affd640 1 -- 172.21.15.45:0/1215726484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448010b20 con 0x56050e274fa0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe43affd640 1 -- 172.21.15.45:0/1215726484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448010cc0 con 0x56050e274fa0 2024-04-25T00:21:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe43affd640 1 -- 172.21.15.45:0/1215726484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448011730 con 0x56050e274fa0 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 -- 172.21.15.45:0/1215726484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 msgr2=0x56050e269250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe448012190 tx=0x7fe44801e000 comp rx=0 tx=0).stop 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 -- 172.21.15.45:0/1215726484 shutdown_connections 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e269250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26fe30 0x56050e280d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 --2- 172.21.15.45:0/1215726484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26f440 0x56050e265690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.513+0000 7fe45a013740 1 -- 172.21.15.45:0/1215726484 >> 172.21.15.45:0/1215726484 conn(0x56050e26aab0 msgr2=0x56050e26ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- 172.21.15.45:0/1215726484 shutdown_connections 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- 172.21.15.45:0/1215726484 wait complete. 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- start start 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 Processor -- start 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26f440 0x56050e27c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26fe30 0x56050e27cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56050e2871e0 con 0x56050e274fa0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56050e286b10 con 0x56050e26fe30 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56050e2878b0 con 0x56050e26f440 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe45a013740 10 client.0 _ll_register_callbacks cb 0x7fe440009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe4558a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe4558a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38978/0 (socket says 172.21.15.45:38978) 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe4558a6640 1 -- 172.21.15.45:0/3436229572 learned_addr learned my addr 172.21.15.45:0/3436229572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.514+0000 7fe4550a5640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26f440 0x56050e27c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4548a4640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26fe30 0x56050e27cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 -- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26f440 msgr2=0x56050e27c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26f440 0x56050e27c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 -- 172.21.15.45:0/3436229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26fe30 msgr2=0x56050e27cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26fe30 0x56050e27cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 -- 172.21.15.45:0/3436229572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050e2801b0 con 0x56050e274fa0 2024-04-25T00:21:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe4558a6640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe4480099e0 tx=0x7fe448010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc42da0141172c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448039060 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fe4480041c0 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe448010cc0 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe448040050 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe45a013740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56050e2ec0b0 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.515+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448010cc0 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe44801e070 con 0x56050e274fa0 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 10 client.15486 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 20 client.15486 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e2f0a00 0x56050e2ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 4 client.15486 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "status"}) -- 0x56050e1da790 con 0x56050e2f0a00 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe4550a5640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e2f0a00 0x56050e2ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe438ff9640 10 client.15486 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe4550a5640 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e2f0a00 0x56050e2ef710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a28b5ac64be3309 server_cookie=ece4535ad0ab6e6f in_seq=0 out_seq=0 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe438ff9640 1 -- 172.21.15.45:0/3436229572 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56050e1da790 con 0x56050e2f0a00 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe438ff9640 10 client.15486 handle_command_reply: tid=0 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.516+0000 7fe45a013740 1 client.15486 shutdown 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 msgr2=0x56050e27fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe4480099e0 tx=0x7fe448010b50 comp rx=0 tx=0).stop 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 shutdown_connections 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56050e2f0a00 0x56050e2ef710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56050e274fa0 0x56050e27fbc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56050e26fe30 0x56050e27cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 --2- 172.21.15.45:0/3436229572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56050e26f440 0x56050e27c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 >> 172.21.15.45:0/3436229572 conn(0x56050e26aab0 msgr2=0x56050e265690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe438ff9640 0 client.15486 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 shutdown_connections 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 1 -- 172.21.15.45:0/3436229572 wait complete. 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe45a013740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24478, 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:48.524 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 109.554134514, 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 155.12978054800001 2024-04-25T00:21:48.525 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 msgr2=0x7fe43c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 0x7fe43c0a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe448009270 tx=0x7fe448012c60 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 msgr2=0x7fe450197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9c0 tx=0x7fe44c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 shutdown_connections 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fe43c0a2b90 0x7fe43c0a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450103350 0x7fe45019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe450102990 0x7fe450197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 --2- 172.21.15.45:0/1413892452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4501017a0 0x7fe450197ab0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.517+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 >> 172.21.15.45:0/1413892452 conn(0x7fe4500fcd10 msgr2=0x7fe450105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.518+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 shutdown_connections 2024-04-25T00:21:48.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.518+0000 7fe457309640 1 -- 172.21.15.45:0/1413892452 wait complete. 2024-04-25T00:21:48.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:21:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 -- 172.21.15.45:0/2283356381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 msgr2=0x7fbbec100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 --2- 172.21.15.45:0/2283356381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec100ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009900 tx=0x7fbbdc02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 -- 172.21.15.45:0/2283356381 shutdown_connections 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 --2- 172.21.15.45:0/2283356381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec102250 0x7fbbec108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 --2- 172.21.15.45:0/2283356381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec101890 0x7fbbec101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 --2- 172.21.15.45:0/2283356381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec100ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 -- 172.21.15.45:0/2283356381 >> 172.21.15.45:0/2283356381 conn(0x7fbbec0fbc30 msgr2=0x7fbbec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 -- 172.21.15.45:0/2283356381 shutdown_connections 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.631+0000 7fbbf319b640 1 -- 172.21.15.45:0/2283356381 wait complete. 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 Processor -- start 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 -- start start 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec101890 0x7fbbec196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 0x7fbbec19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec10c370 con 0x7fbbec1006a0 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec10c1f0 con 0x7fbbec102250 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf319b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec10c4f0 con 0x7fbbec101890 2024-04-25T00:21:48.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf0f37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf0f37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38998/0 (socket says 172.21.15.45:38998) 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.632+0000 7fbbf0f37640 1 -- 172.21.15.45:0/3595053400 learned_addr learned my addr 172.21.15.45:0/3595053400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbebfff640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec101890 0x7fbbec196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf1738640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 0x7fbbec19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 -- 172.21.15.45:0/3595053400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec101890 msgr2=0x7fbbec196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec101890 0x7fbbec196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 -- 172.21.15.45:0/3595053400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 msgr2=0x7fbbec19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 0x7fbbec19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 -- 172.21.15.45:0/3595053400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec19e5e0 con 0x7fbbec1006a0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf1738640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 0x7fbbec19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031610 tx=0x7fbbdc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbe9ffb640 1 -- 172.21.15.45:0/3595053400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc03f030 con 0x7fbbec1006a0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbe9ffb640 1 -- 172.21.15.45:0/3595053400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc003ce0 con 0x7fbbec1006a0 2024-04-25T00:21:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec069d40 con 0x7fbbec1006a0 2024-04-25T00:21:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbe9ffb640 1 -- 172.21.15.45:0/3595053400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035070 con 0x7fbbec1006a0 2024-04-25T00:21:48.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.633+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec06a2b0 con 0x7fbbec1006a0 2024-04-25T00:21:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.635+0000 7fbbe9ffb640 1 -- 172.21.15.45:0/3595053400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbbdc042030 con 0x7fbbec1006a0 2024-04-25T00:21:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 Processor -- start 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 -- start start 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbe9ffb640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 0x7fbbd40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91e30 0x560a7dd96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dd99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a7dcf4f00 con 0x560a7dd91440 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a7dc9d750 con 0x560a7dd91e30 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a7dcf0ec0 con 0x560a7dd96fa0 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf0f37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf1738640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 0x7fbbd40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf0f37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39012/0 (socket says 172.21.15.45:39012) 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf0f37640 1 -- 172.21.15.45:0/152934471 learned_addr learned my addr 172.21.15.45:0/152934471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbf1738640 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dd99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbebfff640 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91e30 0x560a7dd96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.636+0000 7fbbe9ffb640 1 -- 172.21.15.45:0/3595053400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fbbdc0efb80 con 0x7fbbec1006a0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 -- 172.21.15.45:0/152934471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 msgr2=0x560a7dd99430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dd99430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 -- 172.21.15.45:0/152934471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91e30 msgr2=0x560a7dd96930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91e30 0x560a7dd96930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 -- 172.21.15.45:0/152934471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7da092d0 con 0x560a7dd91440 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf1738640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 0x7fbbd40a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40156e0 tx=0x7fbbe4015ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000c00 tx=0x7fbbdc04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de129eb2b89aed06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbd2ffd640 1 -- 172.21.15.45:0/152934471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0bd130 con 0x560a7dd91440 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbd2ffd640 1 -- 172.21.15.45:0/152934471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc0dc0c0 con 0x560a7dd91440 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.637+0000 7fbbd2ffd640 1 -- 172.21.15.45:0/152934471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0bc700 con 0x560a7dd91440 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- 172.21.15.45:0/152934471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 msgr2=0x560a7dd91860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000c00 tx=0x7fbbdc04b000 comp rx=0 tx=0).stop 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- 172.21.15.45:0/152934471 shutdown_connections 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dd99430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91e30 0x560a7dd96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/152934471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91440 0x560a7dd91860 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- 172.21.15.45:0/152934471 >> 172.21.15.45:0/152934471 conn(0x560a7dd8cab0 msgr2=0x560a7dd8ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- 172.21.15.45:0/152934471 shutdown_connections 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- 172.21.15.45:0/152934471 wait complete. 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 -- start start 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 Processor -- start 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dd9bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.638+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a7dd99d10 con 0x560a7dd96fa0 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a7dcb0cf0 con 0x560a7dd91440 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a7dd99e90 con 0x560a7dd91e30 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf5f83740 10 client.0 _ll_register_callbacks cb 0x560a7dcb06e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39018/0 (socket says 172.21.15.45:39018) 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 -- 172.21.15.45:0/1457344135 learned_addr learned my addr 172.21.15.45:0/1457344135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbebfff640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dd9bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 -- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 msgr2=0x560a7dd9bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dd9bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 -- 172.21.15.45:0/1457344135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 msgr2=0x560a7dd9b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd9b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 -- 172.21.15.45:0/1457344135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7dad36b0 con 0x560a7dd96fa0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf1738640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4012db0 tx=0x7fbbe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5052173b39d7287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4026070 con 0x560a7dd96fa0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fbbe4020600 con 0x560a7dd96fa0 2024-04-25T00:21:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.639+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc0dc0c0 con 0x560a7dd96fa0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4032050 con 0x560a7dd96fa0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a7dda5b40 con 0x560a7dd96fa0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc0dc0c0 con 0x560a7dd96fa0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbe401f040 con 0x560a7dd96fa0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 10 client.15498 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 20 client.15498 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7dda8230 0x560a7dda96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 4 client.15498 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a7dd02440 con 0x560a7dda8230 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.640+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7dda8230 0x560a7dda96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.641+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7dda8230 0x560a7dda96a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22a4a6e9c50262d4 server_cookie=86a707b6e53cf3ea in_seq=0 out_seq=0 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.641+0000 7fbbcbfff640 10 client.15498 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.641+0000 7fbbcbfff640 1 -- 172.21.15.45:0/1457344135 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a7dd02440 con 0x560a7dda8230 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.641+0000 7fbbcbfff640 10 client.15498 handle_command_reply: tid=0 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.641+0000 7fbbf5f83740 1 client.15498 shutdown 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 msgr2=0x560a7dda2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4012db0 tx=0x7fbbe4004a40 comp rx=0 tx=0).stop 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 shutdown_connections 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7dda8230 0x560a7dda96a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dda2af0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dd9bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbcbfff640 0 client.15498 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/1457344135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd9b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 >> 172.21.15.45:0/1457344135 conn(0x560a7dd8cab0 msgr2=0x560a7dd8bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 shutdown_connections 2024-04-25T00:21:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 1 -- 172.21.15.45:0/1457344135 wait complete. 2024-04-25T00:21:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.642+0000 7fbbf5f83740 20 client.15498 trim_cache size 0 max 16384 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 Processor -- start 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 -- start start 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dda2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda79e0 con 0x560a7dd96fa0 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda80b0 con 0x560a7dd91440 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda8780 con 0x560a7dd91e30 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.660+0000 7fbbf1738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39022/0 (socket says 172.21.15.45:39022) 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 -- 172.21.15.45:0/2452609006 learned_addr learned my addr 172.21.15.45:0/2452609006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbebfff640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dda2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 -- 172.21.15.45:0/2452609006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 msgr2=0x560a7dda2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dda2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 -- 172.21.15.45:0/2452609006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 msgr2=0x560a7dd928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 -- 172.21.15.45:0/2452609006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7dd8d780 con 0x560a7dd96fa0 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbf1738640 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4017010 tx=0x7fbbe40154f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfdb82e4098d26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbd27fc640 1 -- 172.21.15.45:0/2452609006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe403b070 con 0x560a7dd96fa0 2024-04-25T00:21:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbd27fc640 1 -- 172.21.15.45:0/2452609006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe4013c30 con 0x560a7dd96fa0 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.661+0000 7fbbd27fc640 1 -- 172.21.15.45:0/2452609006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe402f030 con 0x560a7dd96fa0 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- 172.21.15.45:0/2452609006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 msgr2=0x560a7dd99390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4017010 tx=0x7fbbe40154f0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- 172.21.15.45:0/2452609006 shutdown_connections 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd96fa0 0x560a7dd99390 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd91e30 0x560a7dda2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/2452609006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dd928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- 172.21.15.45:0/2452609006 >> 172.21.15.45:0/2452609006 conn(0x560a7dd8cab0 msgr2=0x560a7dd8ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- 172.21.15.45:0/2452609006 shutdown_connections 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- 172.21.15.45:0/2452609006 wait complete. 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 -- start start 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.662+0000 7fbbf5f83740 1 Processor -- start 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dda0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dda17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda9520 con 0x560a7dd91e30 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda8e50 con 0x560a7dd91440 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a7dda9bf0 con 0x560a7dd96fa0 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 10 client.0 _ll_register_callbacks cb 0x7fbbe4012750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39038/0 (socket says 172.21.15.45:39038) 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 -- 172.21.15.45:0/3249769596 learned_addr learned my addr 172.21.15.45:0/3249769596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf1738640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dda17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dda0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 -- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 msgr2=0x560a7dda17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dda17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 -- 172.21.15.45:0/3249769596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 msgr2=0x560a7dda0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dda0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 -- 172.21.15.45:0/3249769596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7dda2000 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dda0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbebfff640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560a7dd928a0 tx=0x7fbbe00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3509fe5b72401ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0011650 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fbbe00117f0 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.663+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe4013c30 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbe00119f0 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a7de0e4c0 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe4013c30 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbe0010430 con 0x560a7dd91e30 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 10 client.15504 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 20 client.15504 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7de13150 0x560a7de11e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 4 client.15504 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "status"}) -- 0x560a7dcfc790 con 0x560a7de13150 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.664+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7de13150 0x560a7de11e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbcbfff640 10 client.15504 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf0f37640 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7de13150 0x560a7de11e60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9985be2050f5ffea server_cookie=87b0317662435a5 in_seq=0 out_seq=0 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbcbfff640 1 -- 172.21.15.45:0/3249769596 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560a7dcfc790 con 0x560a7de13150 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbcbfff640 10 client.15504 handle_command_reply: tid=0 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 client.15504 shutdown 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 msgr2=0x560a7dda1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560a7dd928a0 tx=0x7fbbe00039c0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 shutdown_connections 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a7de13150 0x560a7de11e60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a7dd96fa0 0x560a7dda17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbcbfff640 0 client.15504 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:21:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a7dd91e30 0x560a7dda1230 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 --2- 172.21.15.45:0/3249769596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a7dd91440 0x560a7dda0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 >> 172.21.15.45:0/3249769596 conn(0x560a7dd8cab0 msgr2=0x560a7dd96460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 shutdown_connections 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 1 -- 172.21.15.45:0/3249769596 wait complete. 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.665+0000 7fbbf5f83740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24493, 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:48.673 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 110.71347683800001, 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 149.70996506399999 2024-04-25T00:21:48.674 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 msgr2=0x7fbbd40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 0x7fbbd40a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40156e0 tx=0x7fbbe4015ae0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 msgr2=0x7fbbec1969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031610 tx=0x7fbbdc0043b0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 shutdown_connections 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fbbd40a2b90 0x7fbbd40a5050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbec102250 0x7fbbec19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbec101890 0x7fbbec196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 --2- 172.21.15.45:0/3595053400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbbec1006a0 0x7fbbec1969c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 >> 172.21.15.45:0/3595053400 conn(0x7fbbec0fbc30 msgr2=0x7fbbec104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 shutdown_connections 2024-04-25T00:21:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.666+0000 7fbbf319b640 1 -- 172.21.15.45:0/3595053400 wait complete. 2024-04-25T00:21:48.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 -- 172.21.15.45:0/2221390679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178101870 msgr2=0x7fb178101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 --2- 172.21.15.45:0/2221390679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178101870 0x7fb178101cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009900 tx=0x7fb16c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 -- 172.21.15.45:0/2221390679 shutdown_connections 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 --2- 172.21.15.45:0/2221390679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb178108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 --2- 172.21.15.45:0/2221390679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178101870 0x7fb178101cf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 --2- 172.21.15.45:0/2221390679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178100680 0x7fb178100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 -- 172.21.15.45:0/2221390679 >> 172.21.15.45:0/2221390679 conn(0x7fb1780fbbf0 msgr2=0x7fb1780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 -- 172.21.15.45:0/2221390679 shutdown_connections 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.780+0000 7fb17d977640 1 -- 172.21.15.45:0/2221390679 wait complete. 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 Processor -- start 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 -- start start 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 0x7fb17819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb1781a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d420 con 0x7fb178100680 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d2a0 con 0x7fb178101870 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb17d977640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d5a0 con 0x7fb178102230 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb176ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb176ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39046/0 (socket says 172.21.15.45:39046) 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.781+0000 7fb176ffd640 1 -- 172.21.15.45:0/1729221404 learned_addr learned my addr 172.21.15.45:0/1729221404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb1777fe640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb1781a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb1767fc640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 0x7fb17819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 -- 172.21.15.45:0/1729221404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 msgr2=0x7fb1781a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb1781a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 -- 172.21.15.45:0/1729221404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 msgr2=0x7fb17819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 0x7fb17819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 -- 172.21.15.45:0/1729221404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1781a18b0 con 0x7fb178100680 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb1777fe640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb1781a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb1767fc640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 0x7fb17819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb176ffd640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c970 tx=0x7fb16800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb17c975640 1 -- 172.21.15.45:0/1729221404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168007c10 con 0x7fb178100680 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1781a1ba0 con 0x7fb178100680 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb17c975640 1 -- 172.21.15.45:0/1729221404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb168007db0 con 0x7fb178100680 2024-04-25T00:21:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb17c975640 1 -- 172.21.15.45:0/1729221404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16800f940 con 0x7fb178100680 2024-04-25T00:21:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.782+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1781a20c0 con 0x7fb178100680 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.784+0000 7fb17c975640 1 -- 172.21.15.45:0/1729221404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb16800fae0 con 0x7fb178100680 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 Processor -- start 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 -- start start 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eb0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562eb5e0ef00 con 0x562eb5eb0fa0 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562eb5db7750 con 0x562eb5eabe30 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb180763740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562eb5e0aec0 con 0x562eb5eab440 2024-04-25T00:21:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb17c975640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 0x7fb1600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb1767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb1767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39056/0 (socket says 172.21.15.45:39056) 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb1767fc640 1 -- 172.21.15.45:0/1508468753 learned_addr learned my addr 172.21.15.45:0/1508468753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb1777fe640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 0x7fb1600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb17c975640 1 -- 172.21.15.45:0/1729221404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fb1680cf8d0 con 0x7fb178100680 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb1777fe640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.785+0000 7fb176ffd640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eb0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 -- 172.21.15.45:0/1508468753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 msgr2=0x562eb5eab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 -- 172.21.15.45:0/1508468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 msgr2=0x562eb5eb0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eb0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 -- 172.21.15.45:0/1508468753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb5b232d0 con 0x562eb5eb0fa0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb176ffd640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eb0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1777fe640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 0x7fb1600a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb17000afd0 tx=0x7fb17000ad00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb1767fc640 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009900 tx=0x7fb16c031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a5cadbacbd9eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb15dffb640 1 -- 172.21.15.45:0/1508468753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c03f070 con 0x562eb5eb0fa0 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb15dffb640 1 -- 172.21.15.45:0/1508468753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16c004230 con 0x562eb5eb0fa0 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb15dffb640 1 -- 172.21.15.45:0/1508468753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c035070 con 0x562eb5eb0fa0 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb180763740 1 -- 172.21.15.45:0/1508468753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 msgr2=0x562eb5eb3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.786+0000 7fb180763740 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009900 tx=0x7fb16c031860 comp rx=0 tx=0).stop 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- 172.21.15.45:0/1508468753 shutdown_connections 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5eb3430 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eb0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- 172.21.15.45:0/1508468753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- 172.21.15.45:0/1508468753 >> 172.21.15.45:0/1508468753 conn(0x562eb5ea6ab0 msgr2=0x562eb5ea6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- 172.21.15.45:0/1508468753 shutdown_connections 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- 172.21.15.45:0/1508468753 wait complete. 2024-04-25T00:21:48.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- start start 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 Processor -- start 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eabe30 0x562eb5eb5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 0x562eb5ebcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562eb5eb3ef0 con 0x562eb5eab440 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562eb5dcacf0 con 0x562eb5eb0fa0 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562eb5eb4070 con 0x562eb5eabe30 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb180763740 10 client.0 _ll_register_callbacks cb 0x562eb5dca6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb176ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.787+0000 7fb176ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39058/0 (socket says 172.21.15.45:39058) 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 -- 172.21.15.45:0/2736241567 learned_addr learned my addr 172.21.15.45:0/2736241567 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb1767fc640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eabe30 0x562eb5eb5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb1777fe640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 0x562eb5ebcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 -- 172.21.15.45:0/2736241567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eabe30 msgr2=0x562eb5eb5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eabe30 0x562eb5eb5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 -- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 msgr2=0x562eb5ebcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 0x562eb5ebcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 -- 172.21.15.45:0/2736241567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb5bed6b0 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb1777fe640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 0x562eb5ebcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb176ffd640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb1680987b0 tx=0x7fb16809b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a436e94992ba71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680a0190 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fb168018b80 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16c004230 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1680a49c0 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb180763740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.788+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562eb5ebfe10 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16c004230 con 0x562eb5eab440 2024-04-25T00:21:48.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb168097b30 con 0x562eb5eab440 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 20 client.15516 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5ec24d0 0x562eb5ec3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 4 client.15516 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb5e1c440 con 0x562eb5ec24d0 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb1767fc640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5ec24d0 0x562eb5ec3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb156ffd640 10 client.15516 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.789+0000 7fb1767fc640 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5ec24d0 0x562eb5ec3940 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e79f6ae107c9c7c3 server_cookie=99dc41b62f4dcb9f in_seq=0 out_seq=0 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.790+0000 7fb156ffd640 1 -- 172.21.15.45:0/2736241567 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562eb5e1c440 con 0x562eb5ec24d0 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.790+0000 7fb156ffd640 10 client.15516 handle_command_reply: tid=0 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.790+0000 7fb180763740 1 client.15516 shutdown 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 msgr2=0x562eb5eb5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb1680987b0 tx=0x7fb16809b800 comp rx=0 tx=0).stop 2024-04-25T00:21:48.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 shutdown_connections 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eb0fa0 0x562eb5ebcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eabe30 0x562eb5eb5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5ec24d0 0x562eb5ec3940 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 --2- 172.21.15.45:0/2736241567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eab440 0x562eb5eb5800 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb156ffd640 0 client.15516 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 >> 172.21.15.45:0/2736241567 conn(0x562eb5ea6ab0 msgr2=0x562eb5ea5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 shutdown_connections 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 1 -- 172.21.15.45:0/2736241567 wait complete. 2024-04-25T00:21:48.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.791+0000 7fb180763740 20 client.15516 trim_cache size 0 max 16384 2024-04-25T00:21:48.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 Processor -- start 2024-04-25T00:21:48.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 -- start start 2024-04-25T00:21:48.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 0x562eb5eb3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eb0fa0 0x562eb5ea1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec1c80 con 0x562eb5eabe30 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec2350 con 0x562eb5eab440 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb180763740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec2a20 con 0x562eb5eb0fa0 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb1767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb1767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39074/0 (socket says 172.21.15.45:39074) 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.809+0000 7fb1767fc640 1 -- 172.21.15.45:0/3370822686 learned_addr learned my addr 172.21.15.45:0/3370822686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb176ffd640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 0x562eb5eb3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1777fe640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eb0fa0 0x562eb5ea1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 -- 172.21.15.45:0/3370822686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eb0fa0 msgr2=0x562eb5ea1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eb0fa0 0x562eb5ea1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 -- 172.21.15.45:0/3370822686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 msgr2=0x562eb5eb3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 0x562eb5eb3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 -- 172.21.15.45:0/3370822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb5eac210 con 0x562eb5eabe30 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb176ffd640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 0x562eb5eb3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb1767fc640 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb16c031440 tx=0x7fb16c031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975c3c4c7386fa6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb15d7fa640 1 -- 172.21.15.45:0/3370822686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c035040 con 0x562eb5eabe30 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb15d7fa640 1 -- 172.21.15.45:0/3370822686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16c04e860 con 0x562eb5eabe30 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.810+0000 7fb15d7fa640 1 -- 172.21.15.45:0/3370822686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c03f040 con 0x562eb5eabe30 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- 172.21.15.45:0/3370822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 msgr2=0x562eb5ebcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb16c031440 tx=0x7fb16c031b30 comp rx=0 tx=0).stop 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- 172.21.15.45:0/3370822686 shutdown_connections 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eb0fa0 0x562eb5ea1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eabe30 0x562eb5ebcca0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- 172.21.15.45:0/3370822686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eab440 0x562eb5eb3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- 172.21.15.45:0/3370822686 >> 172.21.15.45:0/3370822686 conn(0x562eb5ea6ab0 msgr2=0x562eb5ea6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- 172.21.15.45:0/3370822686 shutdown_connections 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- 172.21.15.45:0/3370822686 wait complete. 2024-04-25T00:21:48.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- start start 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 Processor -- start 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eaf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eaf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec37c0 con 0x562eb5eb0fa0 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec30f0 con 0x562eb5eabe30 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562eb5ec3e90 con 0x562eb5eab440 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb180763740 10 client.0 _ll_register_callbacks cb 0x7fb16809bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb1777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb1777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39086/0 (socket says 172.21.15.45:39086) 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.811+0000 7fb1777fe640 1 -- 172.21.15.45:0/3115013843 learned_addr learned my addr 172.21.15.45:0/3115013843 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb176ffd640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eaf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1767fc640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eaf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 -- 172.21.15.45:0/3115013843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 msgr2=0x562eb5eaf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eaf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 -- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 msgr2=0x562eb5eaf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eaf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 -- 172.21.15.45:0/3115013843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb5f269b0 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb1777fe640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb170010e70 tx=0x7fb17000a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89dfad2d507d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170004d20 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fb170003970 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16c04e860 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb170011e50 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb180763740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562eb5f27bc0 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.812+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16c04e860 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb170028070 con 0x562eb5eb0fa0 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:21:48.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 20 client.15522 populate_metadata read hostname 'smithi045' 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5f2c8b0 0x562eb5f2b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 4 client.15522 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "status"}) -- 0x562eb5e16790 con 0x562eb5f2c8b0 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb176ffd640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5f2c8b0 0x562eb5f2b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb156ffd640 10 client.15522 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.813+0000 7fb176ffd640 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5f2c8b0 0x562eb5f2b5c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d54f50ef3f7f043 server_cookie=9814a89c88433a47 in_seq=0 out_seq=0 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb156ffd640 1 -- 172.21.15.45:0/3115013843 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562eb5e16790 con 0x562eb5f2c8b0 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb156ffd640 10 client.15522 handle_command_reply: tid=0 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 client.15522 shutdown 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 msgr2=0x562eb5f26470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb170010e70 tx=0x7fb17000a3c0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 shutdown_connections 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562eb5f2c8b0 0x562eb5f2b5c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562eb5eb0fa0 0x562eb5f26470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562eb5eabe30 0x562eb5eaf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb156ffd640 0 client.15522 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 --2- 172.21.15.45:0/3115013843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562eb5eab440 0x562eb5eaf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 >> 172.21.15.45:0/3115013843 conn(0x562eb5ea6ab0 msgr2=0x562eb5ea1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 shutdown_connections 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 1 -- 172.21.15.45:0/3115013843 wait complete. 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.814+0000 7fb180763740 20 client.15522 trim_cache size 0 max 16384 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "24bb2e7a-0298-11ef-bc93-c7b262605968", 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24500, 2024-04-25T00:21:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 134.946223672, 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 93, 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 141.267338567 2024-04-25T00:21:48.822 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 msgr2=0x7fb1600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 0x7fb1600a5050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb17000afd0 tx=0x7fb17000ad00 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 msgr2=0x7fb178199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c970 tx=0x7fb16800ce30 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 shutdown_connections 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fb1600a2b90 0x7fb1600a5050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb178102230 0x7fb1781a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb178101870 0x7fb17819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 --2- 172.21.15.45:0/1729221404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb178100680 0x7fb178199c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 >> 172.21.15.45:0/1729221404 conn(0x7fb1780fbbf0 msgr2=0x7fb178109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 shutdown_connections 2024-04-25T00:21:48.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.815+0000 7fb17d977640 1 -- 172.21.15.45:0/1729221404 wait complete. 2024-04-25T00:21:48.826 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:21:48.835 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:21:48.835 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:21:48.835 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:21:48.835 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T00:21:48.836 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:21:48.836 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:21:48.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T00:21:48.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- 172.21.15.45:0/450449810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024100750 msgr2=0x7f1024100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 --2- 172.21.15.45:0/450449810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024100750 0x7f1024100b70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1020009900 tx=0x7f102002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- 172.21.15.45:0/450449810 shutdown_connections 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 --2- 172.21.15.45:0/450449810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 0x7f1024108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 --2- 172.21.15.45:0/450449810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024101940 0x7f1024101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 --2- 172.21.15.45:0/450449810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024100750 0x7f1024100b70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- 172.21.15.45:0/450449810 >> 172.21.15.45:0/450449810 conn(0x7f10240fbcc0 msgr2=0x7f10240fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- 172.21.15.45:0/450449810 shutdown_connections 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- 172.21.15.45:0/450449810 wait complete. 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 Processor -- start 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.939+0000 7f102b4f5640 1 -- start start 2024-04-25T00:21:48.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 0x7f10241979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 0x7f102419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c350 con 0x7f1024101940 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c1d0 con 0x7f1024102300 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f102410c4d0 con 0x7f1024100750 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39100/0 (socket says 172.21.15.45:39100) 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 -- 172.21.15.45:0/2661305596 learned_addr learned my addr 172.21.15.45:0/2661305596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1029291640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 0x7f10241979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1029a92640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 0x7f102419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 -- 172.21.15.45:0/2661305596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 msgr2=0x7f10241979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 0x7f10241979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 -- 172.21.15.45:0/2661305596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 msgr2=0x7f102419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 0x7f102419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:48.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 -- 172.21.15.45:0/2661305596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102419f5e0 con 0x7f1024101940 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1029291640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 0x7f10241979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f1028a90640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400cd80 con 0x7f1024101940 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102419f8d0 con 0x7f1024101940 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1014005500 con 0x7f1024101940 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014010690 con 0x7f1024101940 2024-04-25T00:21:48.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.940+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102419fdd0 con 0x7f1024101940 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.941+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10241a0030 con 0x7f1024101940 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.943+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10140046c0 con 0x7f1024101940 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.943+0000 7f101a7fc640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 0x7f10100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.943+0000 7f1029291640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 0x7f10100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.943+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f10140cf920 con 0x7f1024101940 2024-04-25T00:21:48.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.944+0000 7f1029291640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 0x7f10100a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f1020032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:48.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:48.948+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f101409c500 con 0x7f1024101940 2024-04-25T00:21:49.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.183+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f10241a0200 con 0x7f1024101940 2024-04-25T00:21:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:48 smithi122 ceph-mon[32959]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:48 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3685734574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:49.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:48 smithi071 ceph-mon[32667]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:49.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:48 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3685734574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:48 smithi045 ceph-mon[29194]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:21:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:48 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3685734574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:49.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.984+0000 7f101a7fc640 1 -- 172.21.15.45:0/2661305596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f101409be00 con 0x7f1024101940 2024-04-25T00:21:49.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 msgr2=0x7f10100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:49.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 0x7f10100a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1020006010 tx=0x7f1020032dd0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 msgr2=0x7f1024197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400edf0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 shutdown_connections 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10100a2c30 0x7f10100a50f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102300 0x7f102419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1024101940 0x7f1024197f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 --2- 172.21.15.45:0/2661305596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1024100750 0x7f10241979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.987+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 >> 172.21.15.45:0/2661305596 conn(0x7f10240fbcc0 msgr2=0x7f10240fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.988+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 shutdown_connections 2024-04-25T00:21:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:49.988+0000 7f102b4f5640 1 -- 172.21.15.45:0/2661305596 wait complete. 2024-04-25T00:21:50.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 -- 172.21.15.45:0/3758710645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 msgr2=0x7f108c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 --2- 172.21.15.45:0/3758710645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c101d10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1080009900 tx=0x7f108002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 -- 172.21.15.45:0/3758710645 shutdown_connections 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 --2- 172.21.15.45:0/3758710645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 --2- 172.21.15.45:0/3758710645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c101d10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 --2- 172.21.15.45:0/3758710645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 0x7f108c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 -- 172.21.15.45:0/3758710645 >> 172.21.15.45:0/3758710645 conn(0x7f108c0fbc10 msgr2=0x7f108c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:50.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 -- 172.21.15.45:0/3758710645 shutdown_connections 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.117+0000 7f1093472640 1 -- 172.21.15.45:0/3758710645 wait complete. 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 Processor -- start 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 -- start start 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 0x7f108c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f108c10c300 con 0x7f108c101890 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f108c10c180 con 0x7f108c102250 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1093472640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f108c10c480 con 0x7f108c1006a0 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1090a0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1090a0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39140/0 (socket says 172.21.15.45:39140) 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f109120e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 0x7f108c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1090a0d640 1 -- 172.21.15.45:0/1377541210 learned_addr learned my addr 172.21.15.45:0/1377541210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.118+0000 7f1091a0f640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 -- 172.21.15.45:0/1377541210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 msgr2=0x7f108c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 0x7f108c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 -- 172.21.15.45:0/1377541210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 msgr2=0x7f108c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 -- 172.21.15.45:0/1377541210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c19f630 con 0x7f108c101890 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1091a0f640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1090a0d640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1080009230 tx=0x7f1080031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108003f030 con 0x7f108c101890 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c19f8c0 con 0x7f108c101890 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1080003ad0 con 0x7f108c101890 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080035070 con 0x7f108c101890 2024-04-25T00:21:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.119+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c19fe00 con 0x7f108c101890 2024-04-25T00:21:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.120+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c1a0030 con 0x7f108c101890 2024-04-25T00:21:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.121+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1080042030 con 0x7f108c101890 2024-04-25T00:21:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.122+0000 7f108a7fc640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 0x7f10740a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.122+0000 7f109120e640 1 -- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 msgr2=0x7f10740a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.122+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f10800ef930 con 0x7f108c101890 2024-04-25T00:21:50.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.122+0000 7f109120e640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 0x7f10740a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:50.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.126+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f10800bbec0 con 0x7f108c101890 2024-04-25T00:21:50.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:49 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:21:50.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: ignoring --setuser ceph since I am not root 2024-04-25T00:21:50.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: ignoring --setgroup ceph since I am not root 2024-04-25T00:21:50.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-mgr[34686]: -- 172.21.15.122:0/2524873397 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596262de5a0 con 0x5596262b2800 2024-04-25T00:21:50.307 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:50.163+0000 7f16afd75200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:21:50.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.314+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f108c1a0730 con 0x7f108c101890 2024-04-25T00:21:50.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.318+0000 7f108a7fc640 1 -- 172.21.15.45:0/1377541210 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f10800c01c0 con 0x7f108c101890 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 msgr2=0x7f10740a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 0x7f10740a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 msgr2=0x7f108c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1080009230 tx=0x7f1080031b60 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 shutdown_connections 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f10740a2c80 0x7f10740a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f108c102250 0x7f108c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f108c101890 0x7f108c197f50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 --2- 172.21.15.45:0/1377541210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f108c1006a0 0x7f108c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 >> 172.21.15.45:0/1377541210 conn(0x7f108c0fbc10 msgr2=0x7f108c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:50.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 shutdown_connections 2024-04-25T00:21:50.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.320+0000 7f1093472640 1 -- 172.21.15.45:0/1377541210 wait complete. 2024-04-25T00:21:50.331 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:49 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:21:50.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setuser ceph since I am not root 2024-04-25T00:21:50.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: ignoring --setgroup ceph since I am not root 2024-04-25T00:21:50.331 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:50.179+0000 7fe42c30e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:21:50.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T00:21:50.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:49 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:21:50.347 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: ignoring --setuser ceph since I am not root 2024-04-25T00:21:50.348 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: ignoring --setgroup ceph since I am not root 2024-04-25T00:21:50.348 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-mgr[34094]: -- 172.21.15.71:0/1551465627 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9e582c5a0 con 0x55f9e5800800 2024-04-25T00:21:50.348 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:50.164+0000 7f536bab7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:21:50.348 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:50.347+0000 7f536bab7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 -- 172.21.15.45:0/2005304875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1017f0 msgr2=0x7f5b9c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 --2- 172.21.15.45:0/2005304875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c101c10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009900 tx=0x7f5b9002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 -- 172.21.15.45:0/2005304875 shutdown_connections 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 --2- 172.21.15.45:0/2005304875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 --2- 172.21.15.45:0/2005304875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 --2- 172.21.15.45:0/2005304875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c101c10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 -- 172.21.15.45:0/2005304875 >> 172.21.15.45:0/2005304875 conn(0x7f5b9c0fcd60 msgr2=0x7f5b9c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 -- 172.21.15.45:0/2005304875 shutdown_connections 2024-04-25T00:21:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 -- 172.21.15.45:0/2005304875 wait complete. 2024-04-25T00:21:50.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.437+0000 7f5ba087b640 1 Processor -- start 2024-04-25T00:21:50.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 -- start start 2024-04-25T00:21:50.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10d460 con 0x7f5b9c1029e0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10d2e0 con 0x7f5b9c1017f0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5ba087b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10d5e0 con 0x7f5b9c1033a0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39154/0 (socket says 172.21.15.45:39154) 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 -- 172.21.15.45:0/1899252965 learned_addr learned my addr 172.21.15.45:0/1899252965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b9a59c640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b9ad9d640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 -- 172.21.15.45:0/1899252965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 msgr2=0x7f5b9c1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 -- 172.21.15.45:0/1899252965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1017f0 msgr2=0x7f5b9c199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 -- 172.21.15.45:0/1899252965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1a18e0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b9ad9d640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.438+0000 7f5b99d9b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e9f0 tx=0x7f5b8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.439+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00cd80 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.439+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1a1bd0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.439+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c005500 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.439+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c010690 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.439+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a20f0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.440+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c1a2320 con 0x7f5b9c1029e0 2024-04-25T00:21:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.441+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f5b8c0046c0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.441+0000 7f5b8b7fe640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 0x7f5b840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.441+0000 7f5b9a59c640 1 -- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 msgr2=0x7f5b840a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.442+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f5b8c0cf7a0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.442+0000 7f5b9a59c640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 0x7f5b840a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.446+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b8c09c380 con 0x7f5b9c1029e0 2024-04-25T00:21:50.618 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:50.449+0000 7f536bab7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:21:50.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.632+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f5b9c1a24f0 con 0x7f5b9c1029e0 2024-04-25T00:21:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.636+0000 7f5b8b7fe640 1 -- 172.21.15.45:0/1899252965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f5b8c09bc80 con 0x7f5b9c1029e0 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 msgr2=0x7f5b840a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 0x7f5b840a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 msgr2=0x7f5b9c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e9f0 tx=0x7f5b8c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 shutdown_connections 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f5b840a2d00 0x7f5b840a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b9c1033a0 0x7f5b9c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b9c1029e0 0x7f5b9c19a1e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 --2- 172.21.15.45:0/1899252965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b9c1017f0 0x7f5b9c199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 >> 172.21.15.45:0/1899252965 conn(0x7f5b9c0fcd60 msgr2=0x7f5b9c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 shutdown_connections 2024-04-25T00:21:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:50.638+0000 7f5ba087b640 1 -- 172.21.15.45:0/1899252965 wait complete. 2024-04-25T00:21:50.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:50.375+0000 7fe42c30e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:50.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:50.487+0000 7fe42c30e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:21:50.650 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T00:21:50.663 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T00:21:50.663 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T00:21:50.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:50.663 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T00:21:50.680 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T00:21:50.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:50.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:21:50.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:50.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T00:21:50.747 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:50.344+0000 7f16afd75200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:50.747 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:50.445+0000 7f16afd75200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:21:50.747 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:50.747+0000 7f16afd75200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:21:50.851 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T00:21:50.851 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:21:50.851 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:21:50.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:21:50.915 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:50.821+0000 7fe42c30e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:21:50.915 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:50.914+0000 7fe42c30e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:21:51.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:50.831+0000 7f16afd75200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:21:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:21:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-mon[32959]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:21:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1377541210' entity='client.admin' 2024-04-25T00:21:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:50 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1899252965' entity='client.admin' 2024-04-25T00:21:51.111 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:21:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:21:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-mon[32667]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:21:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1377541210' entity='client.admin' 2024-04-25T00:21:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1899252965' entity='client.admin' 2024-04-25T00:21:51.119 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:50.756+0000 7f536bab7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:21:51.119 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:50.839+0000 7f536bab7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:21:51.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2661305596' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:21:51.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-mon[29194]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:21:51.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1377541210' entity='client.admin' 2024-04-25T00:21:51.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:50 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1899252965' entity='client.admin' 2024-04-25T00:21:52.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:51 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:51.995+0000 7f16afd75200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:21:52.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:52.163+0000 7f16afd75200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:52.310 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:52.207+0000 7fe42c30e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:21:52.347 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.013+0000 7f536bab7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:21:52.347 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.180+0000 7f536bab7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:52.572 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:52.391+0000 7fe42c30e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:52.604 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:52.570+0000 7fe42c30e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:21:52.612 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:52.329+0000 7f16afd75200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:21:52.613 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:52.611+0000 7f16afd75200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:21:52.618 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.347+0000 7f536bab7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:21:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 -- 172.21.15.45:0/4031028444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca81082b0 msgr2=0x7f2ca8108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/4031028444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca81082b0 0x7f2ca8108690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009990 tx=0x7f2c98009ca0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 -- 172.21.15.45:0/4031028444 shutdown_connections 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/4031028444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/4031028444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca81082b0 0x7f2ca8108690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/4031028444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca8069f40 0x7f2ca806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 -- 172.21.15.45:0/4031028444 >> 172.21.15.45:0/4031028444 conn(0x7f2ca80780e0 msgr2=0x7f2ca80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.827+0000 7f2cae9ef700 1 -- 172.21.15.45:0/4031028444 shutdown_connections 2024-04-25T00:21:52.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.828+0000 7f2cae9ef700 1 -- 172.21.15.45:0/4031028444 wait complete. 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.828+0000 7f2cae9ef700 1 Processor -- start 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 -- start start 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca8101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca81082b0 0x7f2ca8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca81124f0 con 0x7f2ca8069f40 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8112370 con 0x7f2ca81082b0 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cae9ef700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8112670 con 0x7f2ca806a8e0 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:52.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39176/0 (socket says 172.21.15.45:39176) 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2ca7fff700 1 -- 172.21.15.45:0/346004450 learned_addr learned my addr 172.21.15.45:0/346004450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2ca77fe700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca8101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.829+0000 7f2cacf8c700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca81082b0 0x7f2ca8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 msgr2=0x7f2ca8101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca8101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca81082b0 msgr2=0x7f2ca8101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca81082b0 0x7f2ca8101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 -- 172.21.15.45:0/346004450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81a6f90 con 0x7f2ca8069f40 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca77fe700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca7fff700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98027070 con 0x7f2ca8069f40 2024-04-25T00:21:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81a7220 con 0x7f2ca8069f40 2024-04-25T00:21:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c98003e40 con 0x7f2ca8069f40 2024-04-25T00:21:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98022d10 con 0x7f2ca8069f40 2024-04-25T00:21:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.830+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a7680 con 0x7f2ca8069f40 2024-04-25T00:21:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.831+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca8068f20 con 0x7f2ca8069f40 2024-04-25T00:21:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.833+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f2c9802a070 con 0x7f2ca8069f40 2024-04-25T00:21:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.833+0000 7f2ca57fa700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 0x7f2c900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.833+0000 7f2ca77fe700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 msgr2=0x7f2c900a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.833+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f2c980d0860 con 0x7f2ca8069f40 2024-04-25T00:21:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.833+0000 7f2ca77fe700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 0x7f2c900a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:52.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:52.841+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c98031070 con 0x7f2ca8069f40 2024-04-25T00:21:52.902 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:52.702+0000 7f16afd75200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:21:52.902 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:52.900+0000 7f16afd75200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:21:52.920 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.631+0000 7f536bab7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:21:52.921 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.721+0000 7f536bab7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:21:53.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.034+0000 7f2ca77fe700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 msgr2=0x7f2c900a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.034+0000 7f2ca77fe700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 0x7f2c900a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:21:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.097+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2ca802db30 con 0x7f2ca8069f40 2024-04-25T00:21:53.098 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:52.885+0000 7fe42c30e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:21:53.098 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:52.983+0000 7fe42c30e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:21:53.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.098+0000 7f2ca57fa700 1 -- 172.21.15.45:0/346004450 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f2c98013070 con 0x7f2ca8069f40 2024-04-25T00:21:53.099 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 msgr2=0x7f2c900a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 0x7f2c900a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 msgr2=0x7f2ca8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980047b0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 shutdown_connections 2024-04-25T00:21:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f2c900a2cf0 0x7f2c900a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca81082b0 0x7f2ca8101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ca8069f40 0x7f2ca8101200 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 --2- 172.21.15.45:0/346004450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca806a8e0 0x7f2ca8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 >> 172.21.15.45:0/346004450 conn(0x7f2ca80780e0 msgr2=0x7f2ca80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 shutdown_connections 2024-04-25T00:21:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53.100+0000 7f2cae9ef700 1 -- 172.21.15.45:0/346004450 wait complete. 2024-04-25T00:21:53.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:53.093+0000 7f16afd75200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:53 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/346004450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:21:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:53 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/346004450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:21:53.369 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:52.920+0000 7f536bab7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:21:53.369 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:53 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:53.111+0000 7f536bab7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:53 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/346004450' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:21:53.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:53.201+0000 7fe42c30e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:21:53.734 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:53.404+0000 7fe42c30e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:21:53.764 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:21:53.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:53.765 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:21:53.781 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T00:21:53.782 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T00:21:53.782 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:21:53.783 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:21:53.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:53.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:53.928 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:21:53.953 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T00:21:53.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:53.955 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T00:21:54.012 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T00:21:54.012 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:21:54.012 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T00:21:54.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T00:21:54.013 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:21:54.013 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:21:54.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:53.682+0000 7f16afd75200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:21:54.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:54.118 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:53 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:53.705+0000 7f536bab7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:21:54.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:54.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:54.142 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T00:21:54.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:54.158 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-25T00:21:54.215 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:21:54.215 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:21:54.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:21:54.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:21:54.215 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:21:54.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:54.314 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:54.048+0000 7fe42c30e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:21:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:54.336 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:21:54.337 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:21:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:54.395 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:54.102+0000 7f16afd75200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:21:54.395 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:54.197+0000 7f16afd75200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:21:54.396 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:54.394+0000 7f16afd75200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:21:54.415 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:54.124+0000 7f536bab7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:21:54.415 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:54.218+0000 7f536bab7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:21:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:54.453 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:21:54.453 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T00:21:54.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T00:21:54.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:21:54.453 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T00:21:54.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:54.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:54.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:21:54.532 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:21:54.547 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:21:54.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:21:54.547 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:21:54.601 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T00:21:54.601 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T00:21:54.601 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/c7bcad05-6684-43f3-a676-47abd672942f 2024-04-25T00:21:54.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:54.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/c7bcad05-6684-43f3-a676-47abd672942f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T00:21:54.622 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:54.513+0000 7fe42c30e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:21:54.622 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:54.621+0000 7fe42c30e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:21:54.664 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T00:21:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T00:21:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T00:21:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T00:21:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.71:3300,172.21.15.122:3300" 2024-04-25T00:21:54.786 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@24bb2e7a-0298-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/c7bcad05-6684-43f3-a676-47abd672942f 2024-04-25T00:21:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.45:3300/172.21.15.71:3300/172.21.15.122:3300" will pass to kernel 2024-04-25T00:21:54.787 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:21:54.787 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:21:54.788 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:21:54.788 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:21:54.788 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:21:54.790 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:21:54.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:54.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:21:54.859 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:21:54.868 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:54.414+0000 7f536bab7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:21:54.899 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:54.839+0000 7fe42c30e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:21:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:21:54.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:54.957 INFO:teuthology.orchestra.run.smithi045.stdout:client25184 2024-04-25T00:21:54.957 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:21:54.957 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:21:54.974 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:21:55.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:21:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:55.074 INFO:teuthology.orchestra.run.smithi045.stdout:client25184 2024-04-25T00:21:55.075 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:21:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:21:55.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:55.133 INFO:teuthology.orchestra.run.smithi045.stdout:client25184 2024-04-25T00:21:55.133 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968.client25184/status') 2024-04-25T00:21:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:21:55.192 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.25184 (3)172.21.15.45:0/3162367997 2024-04-25T00:21:55.192 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-04-25T00:21:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-04-25T00:21:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-04-25T00:21:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 2.508e-05 s, 2.8 MB/s 2024-04-25T00:21:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:21:55.200 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:21:55.210 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:21:55.210 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:21:55.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T00:21:55.243 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:54.910+0000 7f16afd75200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:21:55.243 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.002+0000 7f16afd75200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:21:55.263 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:54.932+0000 7f536bab7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:21:55.263 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.023+0000 7f536bab7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 -- 172.21.15.45:0/1092688275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718107480 msgr2=0x7ff7181098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 --2- 172.21.15.45:0/1092688275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718107480 0x7ff7181098d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff710009ca0 tx=0x7ff71002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 -- 172.21.15.45:0/1092688275 shutdown_connections 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 --2- 172.21.15.45:0/1092688275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718107480 0x7ff7181098d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 --2- 172.21.15.45:0/1092688275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718104b50 0x7ff718106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 --2- 172.21.15.45:0/1092688275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718102220 0x7ff718104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 -- 172.21.15.45:0/1092688275 >> 172.21.15.45:0/1092688275 conn(0x7ff7180fbc10 msgr2=0x7ff7180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 -- 172.21.15.45:0/1092688275 shutdown_connections 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.314+0000 7ff7202a5640 1 -- 172.21.15.45:0/1092688275 wait complete. 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 Processor -- start 2024-04-25T00:21:55.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 -- start start 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718102220 0x7ff718197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718107480 0x7ff71819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d420 con 0x7ff718104b50 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d2a0 con 0x7ff718102220 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff7202a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d5a0 con 0x7ff718107480 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52396/0 (socket says 172.21.15.45:52396) 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 -- 172.21.15.45:0/622044781 learned_addr learned my addr 172.21.15.45:0/622044781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71e041640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718102220 0x7ff718197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71e842640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718107480 0x7ff71819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718107480 msgr2=0x7ff71819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718107480 0x7ff71819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718102220 msgr2=0x7ff718197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718102220 0x7ff718197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 -- 172.21.15.45:0/622044781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71819f600 con 0x7ff718104b50 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.315+0000 7ff71d840640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff71400e9c0 tx=0x7ff71400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.316+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400cd80 con 0x7ff718104b50 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.316+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff714005500 con 0x7ff718104b50 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.316+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71819f8f0 con 0x7ff718104b50 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.316+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714010690 con 0x7ff718104b50 2024-04-25T00:21:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.316+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71819fe90 con 0x7ff718104b50 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.317+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7180fd700 con 0x7ff718104b50 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.318+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7ff7140046c0 con 0x7ff718104b50 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.318+0000 7ff70f7fe640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 0x7ff7040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.319+0000 7ff71e041640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 msgr2=0x7ff7040a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.319+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7ff7140cf6e0 con 0x7ff718104b50 2024-04-25T00:21:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.319+0000 7ff71e041640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 0x7ff7040a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:55.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.323+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff71409c2c0 con 0x7ff718104b50 2024-04-25T00:21:55.504 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:55.406+0000 7fe42c30e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:21:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.520+0000 7ff71e041640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 msgr2=0x7ff7040a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.520+0000 7ff71e041640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 0x7ff7040a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:21:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.551+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff718062340 con 0x7ff718104b50 2024-04-25T00:21:55.556 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.241+0000 7f16afd75200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:21:55.556 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.373+0000 7f16afd75200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:21:55.556 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.466+0000 7f16afd75200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:21:55.601 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.262+0000 7f536bab7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:21:55.601 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.394+0000 7f536bab7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:21:55.601 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.487+0000 7f536bab7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:21:55.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.602+0000 7ff70f7fe640 1 -- 172.21.15.45:0/622044781 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7ff71409bbc0 con 0x7ff718104b50 2024-04-25T00:21:55.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 msgr2=0x7ff7040a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 0x7ff7040a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 msgr2=0x7ff718197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff71400e9c0 tx=0x7ff71400ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 shutdown_connections 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7ff7040a2d00 0x7ff7040a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff718107480 0x7ff71819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff718104b50 0x7ff718197f70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 --2- 172.21.15.45:0/622044781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff718102220 0x7ff718197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 >> 172.21.15.45:0/622044781 conn(0x7ff7180fbc10 msgr2=0x7ff718102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 shutdown_connections 2024-04-25T00:21:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.605+0000 7ff7202a5640 1 -- 172.21.15.45:0/622044781 wait complete. 2024-04-25T00:21:55.617 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T00:21:55.627 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T00:21:55.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:55.768 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:55 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:21:55.769 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:55.504+0000 7fe42c30e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:21:55.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 -- 172.21.15.45:0/299661342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 msgr2=0x7f7ef0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 --2- 172.21.15.45:0/299661342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef0108ac0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009ca0 tx=0x7f7ee802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 -- 172.21.15.45:0/299661342 shutdown_connections 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 --2- 172.21.15.45:0/299661342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef0108ac0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 --2- 172.21.15.45:0/299661342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0101870 0x7f7ef0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 --2- 172.21.15.45:0/299661342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0100680 0x7f7ef0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.774+0000 7f7ef6363640 1 -- 172.21.15.45:0/299661342 >> 172.21.15.45:0/299661342 conn(0x7f7ef00fbbf0 msgr2=0x7f7ef00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- 172.21.15.45:0/299661342 shutdown_connections 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- 172.21.15.45:0/299661342 wait complete. 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 Processor -- start 2024-04-25T00:21:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- start start 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0100680 0x7f7ef0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 0x7f7ef0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef010c300 con 0x7f7ef0102230 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef010c180 con 0x7f7ef0100680 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.775+0000 7f7ef6363640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef010c480 con 0x7f7ef0101870 2024-04-25T00:21:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52412/0 (socket says 172.21.15.45:52412) 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 -- 172.21.15.45:0/4032591860 learned_addr learned my addr 172.21.15.45:0/4032591860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7eef7fe640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 0x7f7ef0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7eeffff640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0100680 0x7f7ef0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 -- 172.21.15.45:0/4032591860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 msgr2=0x7f7ef0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 0x7f7ef0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 -- 172.21.15.45:0/4032591860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0100680 msgr2=0x7f7ef0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0100680 0x7f7ef0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 -- 172.21.15.45:0/4032591860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef019f6f0 con 0x7f7ef0102230 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7eef7fe640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 0x7f7ef0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef4900640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005cc0 tx=0x7f7ee8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003b70 con 0x7f7ef0102230 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef019f980 con 0x7f7ef0102230 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.776+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8003d10 con 0x7f7ef0102230 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.777+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee802fbd0 con 0x7f7ef0102230 2024-04-25T00:21:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.777+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef019fed0 con 0x7f7ef0102230 2024-04-25T00:21:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.777+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef01a0100 con 0x7f7ef0102230 2024-04-25T00:21:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.779+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7ee802fd70 con 0x7f7ef0102230 2024-04-25T00:21:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.780+0000 7f7eed7fa640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 0x7f7edc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.780+0000 7f7eeffff640 1 -- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 msgr2=0x7f7edc0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.780+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f7ee80f07a0 con 0x7f7ef0102230 2024-04-25T00:21:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.780+0000 7f7eeffff640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 0x7f7edc0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:55.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.784+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ee80bd380 con 0x7f7ef0102230 2024-04-25T00:21:55.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:21:55.869 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.601+0000 7f536bab7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:55.869 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:55.693+0000 7f536bab7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:21:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.971+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ef01a03f0 con 0x7f7ef0102230 2024-04-25T00:21:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.973+0000 7f7eed7fa640 1 -- 172.21.15.45:0/4032591860 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24709 (secure 0 0 0) 0x7f7ee80bcc80 con 0x7f7ef0102230 2024-04-25T00:21:55.974 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:55.974 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:27.913220+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 msgr2=0x7f7edc0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 0x7f7edc0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 msgr2=0x7f7ef019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005cc0 tx=0x7f7ee8004ab0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 shutdown_connections 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f7edc0a2be0 0x7f7edc0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ef0102230 0x7f7ef019f080 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0101870 0x7f7ef0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 --2- 172.21.15.45:0/4032591860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ef0100680 0x7f7ef0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 >> 172.21.15.45:0/4032591860 conn(0x7f7ef00fbbf0 msgr2=0x7f7ef00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 shutdown_connections 2024-04-25T00:21:55.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:55.975+0000 7f7ef6363640 1 -- 172.21.15.45:0/4032591860 wait complete. 2024-04-25T00:21:55.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:55.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:56.019 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:55.767+0000 7fe42c30e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:21:56.019 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:55.913+0000 7fe42c30e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:21:56.019 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:56.018+0000 7fe42c30e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:21:56.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:21:56.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.580+0000 7f16afd75200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:56.056 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:55.672+0000 7f16afd75200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 -- 172.21.15.45:0/1597216016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120101870 msgr2=0x7fd120101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 --2- 172.21.15.45:0/1597216016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120101870 0x7fd120101cf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009900 tx=0x7fd11c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 -- 172.21.15.45:0/1597216016 shutdown_connections 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 --2- 172.21.15.45:0/1597216016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120102230 0x7fd120108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 --2- 172.21.15.45:0/1597216016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120101870 0x7fd120101cf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 --2- 172.21.15.45:0/1597216016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120100680 0x7fd120100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 -- 172.21.15.45:0/1597216016 >> 172.21.15.45:0/1597216016 conn(0x7fd1200fbbf0 msgr2=0x7fd1200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 -- 172.21.15.45:0/1597216016 shutdown_connections 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.095+0000 7fd127864640 1 -- 172.21.15.45:0/1597216016 wait complete. 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 Processor -- start 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 -- start start 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120101870 0x7fd12019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120102230 0x7fd12019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd12010c300 con 0x7fd120100680 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd12010c180 con 0x7fd120102230 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd127864640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd12010c480 con 0x7fd120101870 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd125600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd125600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52428/0 (socket says 172.21.15.45:52428) 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd125600640 1 -- 172.21.15.45:0/4023389441 learned_addr learned my addr 172.21.15.45:0/4023389441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd125e01640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120102230 0x7fd12019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.096+0000 7fd124dff640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120101870 0x7fd12019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 -- 172.21.15.45:0/4023389441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120101870 msgr2=0x7fd12019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120101870 0x7fd12019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 -- 172.21.15.45:0/4023389441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120102230 msgr2=0x7fd12019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120102230 0x7fd12019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 -- 172.21.15.45:0/4023389441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12019f700 con 0x7fd120100680 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd125600640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e9c0 tx=0x7fd11000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000cd80 con 0x7fd120100680 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12019f9f0 con 0x7fd120100680 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1100035f0 con 0x7fd120100680 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110010690 con 0x7fd120100680 2024-04-25T00:21:56.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.097+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12019ff60 con 0x7fd120100680 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.098+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1201079c0 con 0x7fd120100680 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.099+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd1100037a0 con 0x7fd120100680 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.100+0000 7fd1167fc640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 0x7fd10c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.100+0000 7fd124dff640 1 -- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 msgr2=0x7fd10c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.100+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fd1100cf7f0 con 0x7fd120100680 2024-04-25T00:21:56.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.100+0000 7fd124dff640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 0x7fd10c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:56.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.105+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd11009c3d0 con 0x7fd120100680 2024-04-25T00:21:56.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.293+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd120062370 con 0x7fd120100680 2024-04-25T00:21:56.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.293+0000 7fd1167fc640 1 -- 172.21.15.45:0/4023389441 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd11009bcd0 con 0x7fd120100680 2024-04-25T00:21:56.294 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:56.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 msgr2=0x7fd10c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:56.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 0x7fd10c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 msgr2=0x7fd1200694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd11000e9c0 tx=0x7fd11000ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 shutdown_connections 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fd10c0a2be0 0x7fd10c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd120102230 0x7fd12019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd120101870 0x7fd12019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 --2- 172.21.15.45:0/4023389441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd120100680 0x7fd1200694a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 >> 172.21.15.45:0/4023389441 conn(0x7fd1200fbbf0 msgr2=0x7fd120104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 shutdown_connections 2024-04-25T00:21:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.296+0000 7fd127864640 1 -- 172.21.15.45:0/4023389441 wait complete. 2024-04-25T00:21:56.307 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:56.147+0000 7fe42c30e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:21:56.307 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:56.246+0000 7fe42c30e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:21:56.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 -- 172.21.15.45:0/3380576870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c104b30 msgr2=0x7fa35c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 --2- 172.21.15.45:0/3380576870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c104b30 0x7fa35c106f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02a410 comp rx=0 tx=0).stop 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 -- 172.21.15.45:0/3380576870 shutdown_connections 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 --2- 172.21.15.45:0/3380576870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c107460 0x7fa35c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 --2- 172.21.15.45:0/3380576870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c104b30 0x7fa35c106f20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 --2- 172.21.15.45:0/3380576870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c102200 0x7fa35c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 -- 172.21.15.45:0/3380576870 >> 172.21.15.45:0/3380576870 conn(0x7fa35c0fbbf0 msgr2=0x7fa35c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 -- 172.21.15.45:0/3380576870 shutdown_connections 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.457+0000 7fa362615640 1 -- 172.21.15.45:0/3380576870 wait complete. 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 Processor -- start 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 -- start start 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c102200 0x7fa35c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c104b30 0x7fa35c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c10d410 con 0x7fa35c107460 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c10d290 con 0x7fa35c102200 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa362615640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c10d590 con 0x7fa35c104b30 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52444/0 (socket says 172.21.15.45:52444) 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 -- 172.21.15.45:0/1658264762 learned_addr learned my addr 172.21.15.45:0/1658264762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa35bfff640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c102200 0x7fa35c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa35b7fe640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c104b30 0x7fa35c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 -- 172.21.15.45:0/1658264762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c104b30 msgr2=0x7fa35c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c104b30 0x7fa35c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 -- 172.21.15.45:0/1658264762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c102200 msgr2=0x7fa35c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c102200 0x7fa35c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.458+0000 7fa360bb2640 1 -- 172.21.15.45:0/1658264762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c19f730 con 0x7fa35c107460 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa360bb2640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa35400eb10 tx=0x7fa35400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354004b20 con 0x7fa35c107460 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa354004cc0 con 0x7fa35c107460 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c19fa20 con 0x7fa35c107460 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354010760 con 0x7fa35c107460 2024-04-25T00:21:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.459+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c19fef0 con 0x7fa35c107460 2024-04-25T00:21:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.460+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c0fd6e0 con 0x7fa35c107460 2024-04-25T00:21:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.461+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa354005020 con 0x7fa35c107460 2024-04-25T00:21:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.462+0000 7fa3597fa640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 0x7fa3480a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.462+0000 7fa35bfff640 1 -- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 msgr2=0x7fa3480a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.462+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7fa3540cfa30 con 0x7fa35c107460 2024-04-25T00:21:56.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.462+0000 7fa35bfff640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 0x7fa3480a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:56.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.466+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa35409c5b0 con 0x7fa35c107460 2024-04-25T00:21:56.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.650+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa35c0623e0 con 0x7fa35c107460 2024-04-25T00:21:56.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.650+0000 7fa3597fa640 1 -- 172.21.15.45:0/1658264762 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fa35409beb0 con 0x7fa35c107460 2024-04-25T00:21:56.651 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 msgr2=0x7fa3480a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 0x7fa3480a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 msgr2=0x7fa35c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa35400eb10 tx=0x7fa35400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 shutdown_connections 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7fa3480a2be0 0x7fa3480a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa35c107460 0x7fa35c19f0c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa35c104b30 0x7fa35c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 --2- 172.21.15.45:0/1658264762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c102200 0x7fa35c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 >> 172.21.15.45:0/1658264762 conn(0x7fa35c0fbbf0 msgr2=0x7fa35c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 shutdown_connections 2024-04-25T00:21:56.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.652+0000 7fa362615640 1 -- 172.21.15.45:0/1658264762 wait complete. 2024-04-25T00:21:56.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:56.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 -- 172.21.15.45:0/3173966810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984100680 msgr2=0x7f2984100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 --2- 172.21.15.45:0/3173966810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984100680 0x7f2984100aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 -- 172.21.15.45:0/3173966810 shutdown_connections 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 --2- 172.21.15.45:0/3173966810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 --2- 172.21.15.45:0/3173966810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984101870 0x7f2984101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 --2- 172.21.15.45:0/3173966810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984100680 0x7f2984100aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 -- 172.21.15.45:0/3173966810 >> 172.21.15.45:0/3173966810 conn(0x7f29840fbbf0 msgr2=0x7f29840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 -- 172.21.15.45:0/3173966810 shutdown_connections 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.770+0000 7f298c116640 1 -- 172.21.15.45:0/3173966810 wait complete. 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 Processor -- start 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 -- start start 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984100680 0x7f2984104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984101870 0x7f2984102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f298410c300 con 0x7f2984102230 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f298410c180 con 0x7f2984101870 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298c116640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f298410c480 con 0x7f2984100680 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298a6b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298a6b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52466/0 (socket says 172.21.15.45:52466) 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.771+0000 7f298a6b3640 1 -- 172.21.15.45:0/3806010995 learned_addr learned my addr 172.21.15.45:0/3806010995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f29896b1640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984101870 0x7f2984102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f2989eb2640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984100680 0x7f2984104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 -- 172.21.15.45:0/3806010995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984100680 msgr2=0x7f2984104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984100680 0x7f2984104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 -- 172.21.15.45:0/3806010995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984101870 msgr2=0x7f2984102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984101870 0x7f2984102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 -- 172.21.15.45:0/3806010995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2984103a20 con 0x7f2984102230 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298a6b3640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f297c00eb10 tx=0x7f297c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c004b20 con 0x7f2984102230 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841a1c70 con 0x7f2984102230 2024-04-25T00:21:56.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297c004cc0 con 0x7f2984102230 2024-04-25T00:21:56.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c010760 con 0x7f2984102230 2024-04-25T00:21:56.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.772+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a21c0 con 0x7f2984102230 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.773+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29841a23f0 con 0x7f2984102230 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.775+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f297c005020 con 0x7f2984102230 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.776+0000 7f297affd640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 0x7f29700a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.776+0000 7f2989eb2640 1 -- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 msgr2=0x7f29700a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.776+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f297c0cfa10 con 0x7f2984102230 2024-04-25T00:21:56.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.776+0000 7f2989eb2640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 0x7f29700a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:56.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.781+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f297c09c5f0 con 0x7f2984102230 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-mon[32959]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4032591860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4023389441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:56.480+0000 7f16afd75200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:21:56.806 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:21:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z[34661]: 2024-04-25T00:21:56.693+0000 7f16afd75200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:21:56.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:21:56.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-mon[32667]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:21:56.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4032591860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:56.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4023389441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:56.869 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:56.502+0000 7f536bab7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:21:56.869 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:21:56 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y[34090]: 2024-04-25T00:21:56.714+0000 7f536bab7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:21:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/622044781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:21:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-mon[29194]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T00:21:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4032591860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:56 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4023389441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:56.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.968+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29841a2780 con 0x7f2984102230 2024-04-25T00:21:56.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.968+0000 7f297affd640 1 -- 172.21.15.45:0/3806010995 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f297c09bef0 con 0x7f2984102230 2024-04-25T00:21:56.969 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 msgr2=0x7f29700a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 0x7f29700a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 msgr2=0x7f2984103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f297c00eb10 tx=0x7f297c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 shutdown_connections 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f29700a2be0 0x7f29700a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2984102230 0x7f2984103190 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2984101870 0x7f2984102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 --2- 172.21.15.45:0/3806010995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2984100680 0x7f2984104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 >> 172.21.15.45:0/3806010995 conn(0x7f29840fbbf0 msgr2=0x7f29840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 shutdown_connections 2024-04-25T00:21:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:56.970+0000 7f298c116640 1 -- 172.21.15.45:0/3806010995 wait complete. 2024-04-25T00:21:56.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 -- 172.21.15.45:0/581727988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 msgr2=0x7f8ca0073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/581727988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca0073b70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009900 tx=0x7f8c9c02a480 comp rx=0 tx=0).stop 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 -- 172.21.15.45:0/581727988 shutdown_connections 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/581727988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/581727988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 0x7f8ca00745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/581727988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca0073b70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 -- 172.21.15.45:0/581727988 >> 172.21.15.45:0/581727988 conn(0x7f8ca00fcd20 msgr2=0x7f8ca00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 -- 172.21.15.45:0/581727988 shutdown_connections 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.087+0000 7f8ca87ee640 1 -- 172.21.15.45:0/581727988 wait complete. 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 Processor -- start 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 -- start start 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 0x7f8ca0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca01a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010f600 con 0x7f8ca0075720 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010f480 con 0x7f8ca0109620 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca87ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca010f780 con 0x7f8ca0074140 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca5d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca5d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52490/0 (socket says 172.21.15.45:52490) 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca5d89640 1 -- 172.21.15.45:0/2152123350 learned_addr learned my addr 172.21.15.45:0/2152123350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.088+0000 7f8ca6d8b640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca01a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca658a640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 0x7f8ca0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 -- 172.21.15.45:0/2152123350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 msgr2=0x7f8ca0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 0x7f8ca0199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 -- 172.21.15.45:0/2152123350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 msgr2=0x7f8ca01a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca01a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 -- 172.21.15.45:0/2152123350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca01a1910 con 0x7f8ca0075720 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca6d8b640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca5d89640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d770 tx=0x7f8c9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90017070 con 0x7f8ca0075720 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c90005500 con 0x7f8ca0075720 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca01a1c00 con 0x7f8ca0075720 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004e00 con 0x7f8ca0075720 2024-04-25T00:21:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.089+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01a2100 con 0x7f8ca0075720 2024-04-25T00:21:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.090+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca01083e0 con 0x7f8ca0075720 2024-04-25T00:21:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.091+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8c90004fa0 con 0x7f8ca0075720 2024-04-25T00:21:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.092+0000 7f8c977fe640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 0x7f8c8c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.092+0000 7f8ca658a640 1 -- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 msgr2=0x7f8c8c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3305016128 2024-04-25T00:21:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.092+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9129+0+0 (secure 0 0 0) 0x7f8c900cf3c0 con 0x7f8ca0075720 2024-04-25T00:21:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.092+0000 7f8ca658a640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 0x7f8c8c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:21:57.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.100+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8c9009bf40 con 0x7f8ca0075720 2024-04-25T00:21:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.285+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8ca002d300 con 0x7f8ca0075720 2024-04-25T00:21:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.286+0000 7f8c977fe640 1 -- 172.21.15.45:0/2152123350 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f8c9009b840 con 0x7f8ca0075720 2024-04-25T00:21:57.286 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 msgr2=0x7f8c8c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:21:57.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 0x7f8c8c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 msgr2=0x7f8ca019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d770 tx=0x7f8c9000dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 shutdown_connections 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:6800/3305016128,v1:172.21.15.45:6801/3305016128] conn(0x7f8c8c0a2be0 0x7f8c8c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca0109620 0x7f8ca01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ca0075720 0x7f8ca019a290 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 --2- 172.21.15.45:0/2152123350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ca0074140 0x7f8ca0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 >> 172.21.15.45:0/2152123350 conn(0x7f8ca00fcd20 msgr2=0x7f8ca01055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 shutdown_connections 2024-04-25T00:21:57.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.288+0000 7f8ca87ee640 1 -- 172.21.15.45:0/2152123350 wait complete. 2024-04-25T00:21:57.298 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.125+0000 7fe42c30e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:21:57.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/3617496979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec104b30 msgr2=0x7f7cec106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/3617496979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec104b30 0x7f7cec106f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009900 tx=0x7f7ce002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/3617496979 shutdown_connections 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/3617496979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec107460 0x7f7cec1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/3617496979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec104b30 0x7f7cec106f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/3617496979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec102200 0x7f7cec1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/3617496979 >> 172.21.15.45:0/3617496979 conn(0x7f7cec0fbbf0 msgr2=0x7f7cec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/3617496979 shutdown_connections 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/3617496979 wait complete. 2024-04-25T00:21:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.410+0000 7f7cf1dc1640 1 Processor -- start 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 -- start start 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 0x7f7cec197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 0x7f7cec198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec107460 0x7f7cec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10d410 con 0x7f7cec102200 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10d290 con 0x7f7cec104b30 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cf1dc1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10d590 con 0x7f7cec107460 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 0x7f7cec197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 0x7f7cec197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52540/0 (socket says 172.21.15.45:52540) 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceb7fe640 1 -- 172.21.15.45:0/2340839609 learned_addr learned my addr 172.21.15.45:0/2340839609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 0x7f7cec198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7cebfff640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec107460 0x7f7cec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 -- 172.21.15.45:0/2340839609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec107460 msgr2=0x7f7cec19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec107460 0x7f7cec19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 -- 172.21.15.45:0/2340839609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 msgr2=0x7f7cec197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 0x7f7cec197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.411+0000 7f7ceaffd640 1 -- 172.21.15.45:0/2340839609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec19f700 con 0x7f7cec104b30 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7ceaffd640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 0x7f7cec198020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009230 tx=0x7f7ce00043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce003f070 con 0x7f7cec104b30 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec19f990 con 0x7f7cec104b30 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce003b460 con 0x7f7cec104b30 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035070 con 0x7f7cec104b30 2024-04-25T00:21:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.412+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec19fea0 con 0x7f7cec104b30 2024-04-25T00:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.413+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec0fd6e0 con 0x7f7cec104b30 2024-04-25T00:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.415+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f7ce003b910 con 0x7f7cec104b30 2024-04-25T00:21:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.416+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7ce00ef530 con 0x7f7cec104b30 2024-04-25T00:21:57.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.420+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ce00bbdc0 con 0x7f7cec104b30 2024-04-25T00:21:57.552 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.358+0000 7fe42c30e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:21:57.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.700+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cec062340 con 0x7f7cec104b30 2024-04-25T00:21:57.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.702+0000 7f7ce8ff9640 1 -- 172.21.15.45:0/2340839609 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25086 (secure 0 0 0) 0x7f7ce00c00c0 con 0x7f7cec104b30 2024-04-25T00:21:57.703 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:57.703 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:57.361695+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:21:57.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.705+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 msgr2=0x7f7cec198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.705+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 0x7f7cec198020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009230 tx=0x7f7ce00043b0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 shutdown_connections 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cec107460 0x7f7cec19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cec104b30 0x7f7cec198020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 --2- 172.21.15.45:0/2340839609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cec102200 0x7f7cec197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 >> 172.21.15.45:0/2340839609 conn(0x7f7cec0fbbf0 msgr2=0x7f7cec102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 shutdown_connections 2024-04-25T00:21:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.706+0000 7f7cf1dc1640 1 -- 172.21.15.45:0/2340839609 wait complete. 2024-04-25T00:21:57.720 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:21:57.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:21:57.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1658264762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Standby manager daemon z restarted 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Standby manager daemon z started 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Standby manager daemon y restarted 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Standby manager daemon y started 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3806010995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2152123350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Active manager daemon x restarted 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Activating manager daemon x 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: mgrmap e20: x(active, starting, since 0.0134673s), standbys: y, z 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:21:57.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:21:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: Manager daemon x is now available 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:57.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:57 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- 172.21.15.45:0/1817706334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 msgr2=0x7feabc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 --2- 172.21.15.45:0/1817706334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc100e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feab0009900 tx=0x7feab002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- 172.21.15.45:0/1817706334 shutdown_connections 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 --2- 172.21.15.45:0/1817706334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 --2- 172.21.15.45:0/1817706334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc100e10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 --2- 172.21.15.45:0/1817706334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- 172.21.15.45:0/1817706334 >> 172.21.15.45:0/1817706334 conn(0x7feabc0fbbf0 msgr2=0x7feabc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- 172.21.15.45:0/1817706334 shutdown_connections 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- 172.21.15.45:0/1817706334 wait complete. 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 Processor -- start 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.828+0000 7feac2438640 1 -- start start 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f5e0 con 0x7feabc101350 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f460 con 0x7feabc100050 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac2438640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10f760 con 0x7feabc1009b0 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac09d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac09d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52656/0 (socket says 172.21.15.45:52656) 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:36044/0 (socket says 172.21.15.45:36044) 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feac09d5640 1 -- 172.21.15.45:0/1340868929 learned_addr learned my addr 172.21.15.45:0/1340868929 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 -- 172.21.15.45:0/1340868929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 msgr2=0x7feabc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 -- 172.21.15.45:0/1340868929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 msgr2=0x7feabc1a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 -- 172.21.15.45:0/1340868929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1a18f0 con 0x7feabc1009b0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabbfff640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.829+0000 7feabb7fe640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feab0031440 tx=0x7feab0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.830+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab003f070 con 0x7feabc1009b0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.830+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a1b80 con 0x7feabc1009b0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.830+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a2090 con 0x7feabc1009b0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.830+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab0003cd0 con 0x7feabc1009b0 2024-04-25T00:21:57.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.830+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0035070 con 0x7feabc1009b0 2024-04-25T00:21:57.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.831+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc10afa0 con 0x7feabc1009b0 2024-04-25T00:21:57.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.832+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7feab003b4f0 con 0x7feabc1009b0 2024-04-25T00:21:57.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.833+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feab00ef7e0 con 0x7feabc1009b0 2024-04-25T00:21:57.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:57.836+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feab00bc010 con 0x7feabc1009b0 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1658264762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Standby manager daemon z restarted 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Standby manager daemon z started 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.843 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Standby manager daemon y restarted 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Standby manager daemon y started 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3806010995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2152123350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Active manager daemon x restarted 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Activating manager daemon x 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: mgrmap e20: x(active, starting, since 0.0134673s), standbys: y, z 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:21:57.844 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:57.845 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:21:57.846 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: Manager daemon x is now available 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:57.847 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:57 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:57.849 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.701+0000 7fe38ec92700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.849 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.701+0000 7fe38ec92700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.849 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.701+0000 7fe38ec92700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.849 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.701+0000 7fe38ec92700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.849 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.701+0000 7fe38ec92700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.703+0000 7fe38bc4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.703+0000 7fe38bc4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.703+0000 7fe38bc4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.703+0000 7fe38bc4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:21:57.703+0000 7fe38bc4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1658264762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Standby manager daemon z restarted 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Standby manager daemon z started 2024-04-25T00:21:57.850 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.122:0/2428036068' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Standby manager daemon y restarted 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Standby manager daemon y started 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.? 172.21.15.71:0/1522687466' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3806010995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2152123350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Active manager daemon x restarted 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Activating manager daemon x 2024-04-25T00:21:57.851 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: mgrmap e20: x(active, starting, since 0.0134673s), standbys: y, z 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:21:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:21:57.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: Manager daemon x is now available 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:57.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:57 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:21:58.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.055+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feabc02d170 con 0x7feabc1009b0 2024-04-25T00:21:58.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.055+0000 7feab97fa640 1 -- 172.21.15.45:0/1340868929 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7feab00c0310 con 0x7feabc1009b0 2024-04-25T00:21:58.056 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:21:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 msgr2=0x7feabc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feab0031440 tx=0x7feab0004270 comp rx=0 tx=0).stop 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 shutdown_connections 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc101350 0x7feabc1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc1009b0 0x7feabc19a1f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 --2- 172.21.15.45:0/1340868929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100050 0x7feabc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 >> 172.21.15.45:0/1340868929 conn(0x7feabc0fbbf0 msgr2=0x7feabc0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 shutdown_connections 2024-04-25T00:21:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.058+0000 7feac2438640 1 -- 172.21.15.45:0/1340868929 wait complete. 2024-04-25T00:21:58.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- 172.21.15.45:0/2436693554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840104b30 msgr2=0x7f6840106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 --2- 172.21.15.45:0/2436693554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840104b30 0x7f6840106f20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6834009900 tx=0x7f683402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- 172.21.15.45:0/2436693554 shutdown_connections 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 --2- 172.21.15.45:0/2436693554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f68401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 --2- 172.21.15.45:0/2436693554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840104b30 0x7f6840106f20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 --2- 172.21.15.45:0/2436693554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840102200 0x7f68401045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- 172.21.15.45:0/2436693554 >> 172.21.15.45:0/2436693554 conn(0x7f68400fbbf0 msgr2=0x7f68400fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- 172.21.15.45:0/2436693554 shutdown_connections 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- 172.21.15.45:0/2436693554 wait complete. 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 Processor -- start 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.178+0000 7f6845d29640 1 -- start start 2024-04-25T00:21:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840104b30 0x7f68401980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f684019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d3f0 con 0x7f6840102200 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d270 con 0x7f6840107460 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f6845d29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f684010d570 con 0x7f6840104b30 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52688/0 (socket says 172.21.15.45:52688) 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 -- 172.21.15.45:0/3541355278 learned_addr learned my addr 172.21.15.45:0/3541355278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683ffff640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f684019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683effd640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840104b30 0x7f68401980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 -- 172.21.15.45:0/3541355278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840104b30 msgr2=0x7f68401980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840104b30 0x7f68401980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 -- 172.21.15.45:0/3541355278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 msgr2=0x7f684019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f684019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 -- 172.21.15.45:0/3541355278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684019f740 con 0x7f6840102200 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683ffff640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f684019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.179+0000 7f683f7fe640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9f0 tx=0x7f683000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.180+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683000ccc0 con 0x7f6840102200 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.180+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684019fa30 con 0x7f6840102200 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.180+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6830010910 con 0x7f6840102200 2024-04-25T00:21:58.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.180+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68300188a0 con 0x7f6840102200 2024-04-25T00:21:58.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.180+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684019ff50 con 0x7f6840102200 2024-04-25T00:21:58.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.181+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68400fd6e0 con 0x7f6840102200 2024-04-25T00:21:58.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.182+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f6830010ac0 con 0x7f6840102200 2024-04-25T00:21:58.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.183+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f68300cf810 con 0x7f6840102200 2024-04-25T00:21:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.187+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f683009c690 con 0x7f6840102200 2024-04-25T00:21:58.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.381+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f6830097ce0 con 0x7f6840102200 2024-04-25T00:21:58.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.382+0000 7f683cff9640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 0x7f68280ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.382+0000 7f683effd640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 0x7f68280ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.386+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f684002d420 con 0x7f6840102200 2024-04-25T00:21:58.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.386+0000 7f683cff9640 1 -- 172.21.15.45:0/3541355278 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f6830028a00 con 0x7f6840102200 2024-04-25T00:21:58.388 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.388+0000 7f683effd640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 0x7f68280ad310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6834005ab0 tx=0x7f6834000bc0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 msgr2=0x7f68280ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 0x7f68280ad310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6834005ab0 tx=0x7f6834000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 msgr2=0x7f6840197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f683000e9f0 tx=0x7f683000edf0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 shutdown_connections 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f68280aaf20 0x7f68280ad310 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6840107460 0x7f684019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6840104b30 0x7f68401980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 --2- 172.21.15.45:0/3541355278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840102200 0x7f6840197b70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 >> 172.21.15.45:0/3541355278 conn(0x7f68400fbbf0 msgr2=0x7f6840102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 shutdown_connections 2024-04-25T00:21:58.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.390+0000 7f6845d29640 1 -- 172.21.15.45:0/3541355278 wait complete. 2024-04-25T00:21:58.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:21:58.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.517+0000 7fe420348640 1 -- 172.21.15.45:0/2497723513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418101790 msgr2=0x7fe418101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 --2- 172.21.15.45:0/2497723513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418101790 0x7fe418101bb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe414009900 tx=0x7fe41402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 -- 172.21.15.45:0/2497723513 shutdown_connections 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 --2- 172.21.15.45:0/2497723513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 0x7fe418109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 --2- 172.21.15.45:0/2497723513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418102980 0x7fe418102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 --2- 172.21.15.45:0/2497723513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418101790 0x7fe418101bb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 -- 172.21.15.45:0/2497723513 >> 172.21.15.45:0/2497723513 conn(0x7fe4180fcd20 msgr2=0x7fe4180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 -- 172.21.15.45:0/2497723513 shutdown_connections 2024-04-25T00:21:58.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.518+0000 7fe420348640 1 -- 172.21.15.45:0/2497723513 wait complete. 2024-04-25T00:21:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 Processor -- start 2024-04-25T00:21:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 -- start start 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418101790 0x7fe418197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 0x7fe41819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe41810d400 con 0x7fe418102980 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe41810d280 con 0x7fe418103340 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe420348640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe41810d580 con 0x7fe418101790 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52708/0 (socket says 172.21.15.45:52708) 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 -- 172.21.15.45:0/238494546 learned_addr learned my addr 172.21.15.45:0/238494546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41e0e4640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418101790 0x7fe418197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41e8e5640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 0x7fe41819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 -- 172.21.15.45:0/238494546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418101790 msgr2=0x7fe418197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418101790 0x7fe418197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 -- 172.21.15.45:0/238494546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 msgr2=0x7fe41819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.519+0000 7fe41d8e3640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 0x7fe41819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe41d8e3640 1 -- 172.21.15.45:0/238494546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41819f710 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe41d8e3640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d770 tx=0x7fe40800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408017070 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408005500 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408004e00 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41819fa00 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.520+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41819ff50 con 0x7fe418102980 2024-04-25T00:21:58.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.521+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe418108ba0 con 0x7fe418102980 2024-04-25T00:21:58.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.522+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fe408004fa0 con 0x7fe418102980 2024-04-25T00:21:58.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.523+0000 7fe40f7fe640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 0x7fe4040a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.523+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe4080cee50 con 0x7fe418102980 2024-04-25T00:21:58.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.523+0000 7fe41e0e4640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 0x7fe4040a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.524+0000 7fe41e0e4640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 0x7fe4040a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe414005ab0 tx=0x7fe414000bc0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.531+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe40809b650 con 0x7fe418102980 2024-04-25T00:21:58.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.725+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe41802d3c0 con 0x7fe418102980 2024-04-25T00:21:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.725+0000 7fe40f7fe640 1 -- 172.21.15.45:0/238494546 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fe40809f950 con 0x7fe418102980 2024-04-25T00:21:58.726 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:21:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 msgr2=0x7fe4040a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 0x7fe4040a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe414005ab0 tx=0x7fe414000bc0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 msgr2=0x7fe418198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d770 tx=0x7fe40800dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 shutdown_connections 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe4040a2a10 0x7fe4040a4ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418103340 0x7fe41819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe418102980 0x7fe418198030 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 --2- 172.21.15.45:0/238494546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe418101790 0x7fe418197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 >> 172.21.15.45:0/238494546 conn(0x7fe4180fcd20 msgr2=0x7fe418105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 shutdown_connections 2024-04-25T00:21:58.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.727+0000 7fe420348640 1 -- 172.21.15.45:0/238494546 wait complete. 2024-04-25T00:21:58.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:58 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2340839609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:58.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:58 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1340868929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:58.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:58 smithi045 ceph-mon[29194]: mgrmap e21: x(active, since 1.02194s), standbys: y, z 2024-04-25T00:21:58.737 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:58 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3541355278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:58.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:21:58.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 -- 172.21.15.45:0/323581643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0102200 msgr2=0x7f57f01045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 --2- 172.21.15.45:0/323581643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0102200 0x7f57f01045f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009900 tx=0x7f57e002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:58.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 -- 172.21.15.45:0/323581643 shutdown_connections 2024-04-25T00:21:58.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 --2- 172.21.15.45:0/323581643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 --2- 172.21.15.45:0/323581643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0104b30 0x7f57f0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 --2- 172.21.15.45:0/323581643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0102200 0x7f57f01045f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 -- 172.21.15.45:0/323581643 >> 172.21.15.45:0/323581643 conn(0x7f57f00fbbf0 msgr2=0x7f57f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:58.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 -- 172.21.15.45:0/323581643 shutdown_connections 2024-04-25T00:21:58.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.846+0000 7f57f50e7640 1 -- 172.21.15.45:0/323581643 wait complete. 2024-04-25T00:21:58.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 Processor -- start 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 -- start start 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0102200 0x7f57f0101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57f010d430 con 0x7f57f0104b30 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57f010d2b0 con 0x7f57f0107460 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57f50e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57f010d5b0 con 0x7f57f0102200 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52726/0 (socket says 172.21.15.45:52726) 2024-04-25T00:21:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57e659c640 1 -- 172.21.15.45:0/604158857 learned_addr learned my addr 172.21.15.45:0/604158857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57ef59e640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.847+0000 7f57eed9d640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0102200 0x7f57f0101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 -- 172.21.15.45:0/604158857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0102200 msgr2=0x7f57f0101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0102200 0x7f57f0101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 -- 172.21.15.45:0/604158857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 msgr2=0x7f57f01006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 -- 172.21.15.45:0/604158857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f0100f80 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57ef59e640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57e659c640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e9f0 tx=0x7f57d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d800ccc0 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d8005500 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f01a1be0 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8010690 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.848+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f01a20e0 con 0x7f57f0104b30 2024-04-25T00:21:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.849+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f00fd6e0 con 0x7f57f0104b30 2024-04-25T00:21:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.850+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f57d80046c0 con 0x7f57f0104b30 2024-04-25T00:21:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.851+0000 7f57ecd99640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 0x7f57dc0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.851+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f57d80cf710 con 0x7f57f0104b30 2024-04-25T00:21:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.851+0000 7f57eed9d640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 0x7f57dc0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.852+0000 7f57eed9d640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 0x7f57dc0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57e0000c00 tx=0x7f57e00315d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:58.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:58.855+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57d8053740 con 0x7f57f0104b30 2024-04-25T00:21:58.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:58 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2340839609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:58.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:58 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1340868929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:58.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:58 smithi071 ceph-mon[32667]: mgrmap e21: x(active, since 1.02194s), standbys: y, z 2024-04-25T00:21:58.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:58 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3541355278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:58.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:58 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2340839609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:21:58.946 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:58 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1340868929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:21:58.946 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:58 smithi122 ceph-mon[32959]: mgrmap e21: x(active, since 1.02194s), standbys: y, z 2024-04-25T00:21:58.946 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:58 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3541355278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:21:59.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.053+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f57f002d370 con 0x7f57f0104b30 2024-04-25T00:21:59.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.053+0000 7f57ecd99640 1 -- 172.21.15.45:0/604158857 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f57d809c4e0 con 0x7f57f0104b30 2024-04-25T00:21:59.054 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:21:59.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 msgr2=0x7f57dc0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 0x7f57dc0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57e0000c00 tx=0x7f57e00315d0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 msgr2=0x7f57f01001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e9f0 tx=0x7f57d800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 shutdown_connections 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57dc0a2990 0x7f57dc0a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57f0107460 0x7f57f01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57f0104b30 0x7f57f01001b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 --2- 172.21.15.45:0/604158857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57f0102200 0x7f57f0101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 >> 172.21.15.45:0/604158857 conn(0x7f57f00fbbf0 msgr2=0x7f57f0102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 shutdown_connections 2024-04-25T00:21:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.055+0000 7f57f50e7640 1 -- 172.21.15.45:0/604158857 wait complete. 2024-04-25T00:21:59.067 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T00:21:59.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- 172.21.15.45:0/1024129158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 msgr2=0x7f15bc1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 --2- 172.21.15.45:0/1024129158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc1045f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009900 tx=0x7f15b002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- 172.21.15.45:0/1024129158 shutdown_connections 2024-04-25T00:21:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 --2- 172.21.15.45:0/1024129158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc107460 0x7f15bc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 --2- 172.21.15.45:0/1024129158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc104b30 0x7f15bc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 --2- 172.21.15.45:0/1024129158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc1045f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- 172.21.15.45:0/1024129158 >> 172.21.15.45:0/1024129158 conn(0x7f15bc0fbbf0 msgr2=0x7f15bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- 172.21.15.45:0/1024129158 shutdown_connections 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- 172.21.15.45:0/1024129158 wait complete. 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 Processor -- start 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.171+0000 7f15c2af5640 1 -- start start 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc104b30 0x7f15bc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d450 con 0x7f15bc107460 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d2d0 con 0x7f15bc102200 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d5d0 con 0x7f15bc104b30 2024-04-25T00:21:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52750/0 (socket says 172.21.15.45:52750) 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c0891640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 -- 172.21.15.45:0/4055968370 learned_addr learned my addr 172.21.15.45:0/4055968370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15bbfff640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc104b30 0x7f15bc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 -- 172.21.15.45:0/4055968370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc104b30 msgr2=0x7f15bc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc104b30 0x7f15bc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 -- 172.21.15.45:0/4055968370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 msgr2=0x7f15bc199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 -- 172.21.15.45:0/4055968370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc1a18e0 con 0x7f15bc107460 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c1092640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f15b400efd0 tx=0x7f15b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4003b70 con 0x7f15bc107460 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.172+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc1a1bd0 con 0x7f15bc107460 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.173+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b4003d10 con 0x7f15bc107460 2024-04-25T00:21:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.173+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4010930 con 0x7f15bc107460 2024-04-25T00:21:59.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.173+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a20f0 con 0x7f15bc107460 2024-04-25T00:21:59.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.174+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc108fb0 con 0x7f15bc107460 2024-04-25T00:21:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.175+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f15b4004b40 con 0x7f15bc107460 2024-04-25T00:21:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.175+0000 7f15b9ffb640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 0x7f15ac0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.176+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f15b40cf940 con 0x7f15bc107460 2024-04-25T00:21:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.176+0000 7f15c0891640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 0x7f15ac0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.176+0000 7f15c0891640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 0x7f15ac0a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15b0031440 tx=0x7f15b00315b0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:59.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.179+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15b409c710 con 0x7f15bc107460 2024-04-25T00:21:59.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.467+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15bc02d3c0 con 0x7f15bc107460 2024-04-25T00:21:59.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.468+0000 7f15b9ffb640 1 -- 172.21.15.45:0/4055968370 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f15b40bc0d0 con 0x7f15bc107460 2024-04-25T00:21:59.468 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:59.469 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:21:59.469 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:21:59.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 msgr2=0x7f15ac0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 0x7f15ac0a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15b0031440 tx=0x7f15b00315b0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 msgr2=0x7f15bc1a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f15b400efd0 tx=0x7f15b400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 shutdown_connections 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f15ac0a2990 0x7f15ac0a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15bc107460 0x7f15bc1a1240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc104b30 0x7f15bc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.470+0000 7f15c2af5640 1 --2- 172.21.15.45:0/4055968370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15bc102200 0x7f15bc199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.471+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 >> 172.21.15.45:0/4055968370 conn(0x7f15bc0fbbf0 msgr2=0x7f15bc1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.471+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 shutdown_connections 2024-04-25T00:21:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.471+0000 7f15c2af5640 1 -- 172.21.15.45:0/4055968370 wait complete. 2024-04-25T00:21:59.485 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T00:21:59.485 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T00:21:59.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.631+0000 7feaa26cd640 1 -- 172.21.15.45:0/3656850761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c0741a0 msgr2=0x7fea9c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.631+0000 7feaa26cd640 1 --2- 172.21.15.45:0/3656850761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c0741a0 0x7fea9c074620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea90009860 tx=0x7fea9002a340 comp rx=0 tx=0).stop 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 -- 172.21.15.45:0/3656850761 shutdown_connections 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 --2- 172.21.15.45:0/3656850761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c109670 0x7fea9c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 --2- 172.21.15.45:0/3656850761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c0741a0 0x7fea9c074620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 --2- 172.21.15.45:0/3656850761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c075780 0x7fea9c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 -- 172.21.15.45:0/3656850761 >> 172.21.15.45:0/3656850761 conn(0x7fea9c0fcd40 msgr2=0x7fea9c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 -- 172.21.15.45:0/3656850761 shutdown_connections 2024-04-25T00:21:59.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.632+0000 7feaa26cd640 1 -- 172.21.15.45:0/3656850761 wait complete. 2024-04-25T00:21:59.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.633+0000 7feaa26cd640 1 Processor -- start 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.633+0000 7feaa26cd640 1 -- start start 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 0x7fea9c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c10f660 con 0x7fea9c0741a0 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c10f4e0 con 0x7fea9c109670 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa26cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c10f7e0 con 0x7fea9c075780 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7fea9b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7fea9b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:36120/0 (socket says 172.21.15.45:36120) 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52796/0 (socket says 172.21.15.45:52796) 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7fea9b7fe640 1 -- 172.21.15.45:0/847981793 learned_addr learned my addr 172.21.15.45:0/847981793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:21:59.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.634+0000 7feaa0c6a640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 0x7fea9c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 -- 172.21.15.45:0/847981793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 msgr2=0x7fea9c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 -- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 msgr2=0x7fea9c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 -- 172.21.15.45:0/847981793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a16f0 con 0x7fea9c109670 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7fea9b7fe640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7fea9bfff640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa0c6a640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 0x7fea9c1a1080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea9400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400cd60 con 0x7fea9c109670 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea94005020 con 0x7fea9c109670 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94010690 con 0x7fea9c109670 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1a19e0 con 0x7fea9c109670 2024-04-25T00:21:59.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.635+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a1ee0 con 0x7fea9c109670 2024-04-25T00:21:59.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.636+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c1083a0 con 0x7fea9c109670 2024-04-25T00:21:59.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.637+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fea94005540 con 0x7fea9c109670 2024-04-25T00:21:59.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.638+0000 7fea997fa640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 0x7fea840a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.638+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea940cfca0 con 0x7fea9c109670 2024-04-25T00:21:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.638+0000 7fea9bfff640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 0x7fea840a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:59.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.639+0000 7fea9bfff640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 0x7fea840a5340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005fd0 tx=0x7fea8c009260 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.642+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea9409c560 con 0x7fea9c109670 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: [25/Apr/2024:00:21:58] ENGINE Bus STARTING 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: [25/Apr/2024:00:21:58] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: [25/Apr/2024:00:21:58] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: [25/Apr/2024:00:21:58] ENGINE Bus STARTED 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/238494546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/604158857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4055968370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:59.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:21:59 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.851+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea9c061b80 con 0x7fea840a2e80 2024-04-25T00:21:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.862+0000 7fea997fa640 1 -- 172.21.15.45:0/847981793 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339719 (secure 0 0 0) 0x7fea9c061b80 con 0x7fea840a2e80 2024-04-25T00:21:59.867 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:21:59.870 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: [25/Apr/2024:00:21:58] ENGINE Bus STARTING 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: [25/Apr/2024:00:21:58] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: [25/Apr/2024:00:21:58] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: [25/Apr/2024:00:21:58] ENGINE Bus STARTED 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/238494546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/604158857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4055968370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:59.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:21:59 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:21:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 msgr2=0x7fea840a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 0x7fea840a5340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005fd0 tx=0x7fea8c009260 comp rx=0 tx=0).stop 2024-04-25T00:21:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 msgr2=0x7fea9c1a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 0x7fea9c1a1080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea94009fd0 tx=0x7fea9400ee50 comp rx=0 tx=0).stop 2024-04-25T00:21:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 shutdown_connections 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea840a2e80 0x7fea840a5340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea9c109670 0x7fea9c1a1080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea9c075780 0x7fea9c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 --2- 172.21.15.45:0/847981793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea9c0741a0 0x7fea9c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 >> 172.21.15.45:0/847981793 conn(0x7fea9c0fcd40 msgr2=0x7fea9c105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 shutdown_connections 2024-04-25T00:21:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:21:59.867+0000 7feaa26cd640 1 -- 172.21.15.45:0/847981793 wait complete. 2024-04-25T00:21:59.879 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-25T00:21:59.380607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189898,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1435,"num_read_kb":880,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391176,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550648,"statfs":{"total":1151940427776,"available":1151539863552,"internally_reserved":0,"allocated":13180928,"data_stored":9897572,"data_compressed":538584,"data_compressed_allocated":724992,"data_compressed_original":4300800,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.985744"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570237+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:21:57.570237+0000","last_peered":"2024-04-25T00:21:57.570237+0000","last_clean":"2024-04-25T00:21:57.570237+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:21:57.570237+0000","last_undegraded":"2024-04-25T00:21:57.570237+0000","last_fullsized":"2024-04-25T00:21:57.570237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569393+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:21:57.569393+0000","last_peered":"2024-04-25T00:21:57.569393+0000","last_clean":"2024-04-25T00:21:57.569393+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:21:57.569393+0000","last_undegraded":"2024-04-25T00:21:57.569393+0000","last_fullsized":"2024-04-25T00:21:57.569393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471340+0000","last_change":"2024-04-25T00:21:27.923300+0000","last_active":"2024-04-25T00:21:57.471340+0000","last_peered":"2024-04-25T00:21:57.471340+0000","last_clean":"2024-04-25T00:21:57.471340+0000","last_became_active":"2024-04-25T00:21:27.922951+0000","last_became_peered":"2024-04-25T00:21:27.922951+0000","last_unstale":"2024-04-25T00:21:57.471340+0000","last_undegraded":"2024-04-25T00:21:57.471340+0000","last_fullsized":"2024-04-25T00:21:57.471340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:50.728710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473760+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:21:57.473760+0000","last_peered":"2024-04-25T00:21:57.473760+0000","last_clean":"2024-04-25T00:21:57.473760+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:21:57.473760+0000","last_undegraded":"2024-04-25T00:21:57.473760+0000","last_fullsized":"2024-04-25T00:21:57.473760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445544+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:21:57.445544+0000","last_peered":"2024-04-25T00:21:57.445544+0000","last_clean":"2024-04-25T00:21:57.445544+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:21:57.445544+0000","last_undegraded":"2024-04-25T00:21:57.445544+0000","last_fullsized":"2024-04-25T00:21:57.445544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367303+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:21:57.367303+0000","last_peered":"2024-04-25T00:21:57.367303+0000","last_clean":"2024-04-25T00:21:57.367303+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:21:57.367303+0000","last_undegraded":"2024-04-25T00:21:57.367303+0000","last_fullsized":"2024-04-25T00:21:57.367303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556792+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:21:57.556792+0000","last_peered":"2024-04-25T00:21:57.556792+0000","last_clean":"2024-04-25T00:21:57.556792+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:21:57.556792+0000","last_undegraded":"2024-04-25T00:21:57.556792+0000","last_fullsized":"2024-04-25T00:21:57.556792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445954+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:21:57.445954+0000","last_peered":"2024-04-25T00:21:57.445954+0000","last_clean":"2024-04-25T00:21:57.445954+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:21:57.445954+0000","last_undegraded":"2024-04-25T00:21:57.445954+0000","last_fullsized":"2024-04-25T00:21:57.445954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368761+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:21:57.368761+0000","last_peered":"2024-04-25T00:21:57.368761+0000","last_clean":"2024-04-25T00:21:57.368761+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:21:57.368761+0000","last_undegraded":"2024-04-25T00:21:57.368761+0000","last_fullsized":"2024-04-25T00:21:57.368761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570829+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:21:57.570829+0000","last_peered":"2024-04-25T00:21:57.570829+0000","last_clean":"2024-04-25T00:21:57.570829+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:21:57.570829+0000","last_undegraded":"2024-04-25T00:21:57.570829+0000","last_fullsized":"2024-04-25T00:21:57.570829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571324+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:21:57.571324+0000","last_peered":"2024-04-25T00:21:57.571324+0000","last_clean":"2024-04-25T00:21:57.571324+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:21:57.571324+0000","last_undegraded":"2024-04-25T00:21:57.571324+0000","last_fullsized":"2024-04-25T00:21:57.571324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570961+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:21:57.570961+0000","last_peered":"2024-04-25T00:21:57.570961+0000","last_clean":"2024-04-25T00:21:57.570961+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:21:57.570961+0000","last_undegraded":"2024-04-25T00:21:57.570961+0000","last_fullsized":"2024-04-25T00:21:57.570961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555127+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:21:57.555127+0000","last_peered":"2024-04-25T00:21:57.555127+0000","last_clean":"2024-04-25T00:21:57.555127+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:21:57.555127+0000","last_undegraded":"2024-04-25T00:21:57.555127+0000","last_fullsized":"2024-04-25T00:21:57.555127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569250+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:21:57.569250+0000","last_peered":"2024-04-25T00:21:57.569250+0000","last_clean":"2024-04-25T00:21:57.569250+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:21:57.569250+0000","last_undegraded":"2024-04-25T00:21:57.569250+0000","last_fullsized":"2024-04-25T00:21:57.569250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471567+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:21:57.471567+0000","last_peered":"2024-04-25T00:21:57.471567+0000","last_clean":"2024-04-25T00:21:57.471567+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:21:57.471567+0000","last_undegraded":"2024-04-25T00:21:57.471567+0000","last_fullsized":"2024-04-25T00:21:57.471567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369493+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:21:57.369493+0000","last_peered":"2024-04-25T00:21:57.369493+0000","last_clean":"2024-04-25T00:21:57.369493+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:21:57.369493+0000","last_undegraded":"2024-04-25T00:21:57.369493+0000","last_fullsized":"2024-04-25T00:21:57.369493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368878+0000","last_change":"2024-04-25T00:20:27.153842+0000","last_active":"2024-04-25T00:21:57.368878+0000","last_peered":"2024-04-25T00:21:57.368878+0000","last_clean":"2024-04-25T00:21:57.368878+0000","last_became_active":"2024-04-25T00:20:27.152421+0000","last_became_peered":"2024-04-25T00:20:27.152421+0000","last_unstale":"2024-04-25T00:21:57.368878+0000","last_undegraded":"2024-04-25T00:21:57.368878+0000","last_fullsized":"2024-04-25T00:21:57.368878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:30.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":185,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471573+0000","last_change":"2024-04-25T00:20:27.056346+0000","last_active":"2024-04-25T00:21:57.471573+0000","last_peered":"2024-04-25T00:21:57.471573+0000","last_clean":"2024-04-25T00:21:57.471573+0000","last_became_active":"2024-04-25T00:20:27.046470+0000","last_became_peered":"2024-04-25T00:20:27.046470+0000","last_unstale":"2024-04-25T00:21:57.471573+0000","last_undegraded":"2024-04-25T00:21:57.471573+0000","last_fullsized":"2024-04-25T00:21:57.471573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:31.760140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367846+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:21:57.367846+0000","last_peered":"2024-04-25T00:21:57.367846+0000","last_clean":"2024-04-25T00:21:57.367846+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:21:57.367846+0000","last_undegraded":"2024-04-25T00:21:57.367846+0000","last_fullsized":"2024-04-25T00:21:57.367846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367941+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:21:57.367941+0000","last_peered":"2024-04-25T00:21:57.367941+0000","last_clean":"2024-04-25T00:21:57.367941+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:21:57.367941+0000","last_undegraded":"2024-04-25T00:21:57.367941+0000","last_fullsized":"2024-04-25T00:21:57.367941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570184+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:21:57.570184+0000","last_peered":"2024-04-25T00:21:57.570184+0000","last_clean":"2024-04-25T00:21:57.570184+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:21:57.570184+0000","last_undegraded":"2024-04-25T00:21:57.570184+0000","last_fullsized":"2024-04-25T00:21:57.570184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368450+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:21:57.368450+0000","last_peered":"2024-04-25T00:21:57.368450+0000","last_clean":"2024-04-25T00:21:57.368450+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:21:57.368450+0000","last_undegraded":"2024-04-25T00:21:57.368450+0000","last_fullsized":"2024-04-25T00:21:57.368450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569175+0000","last_change":"2024-04-25T00:20:27.173867+0000","last_active":"2024-04-25T00:21:57.569175+0000","last_peered":"2024-04-25T00:21:57.569175+0000","last_clean":"2024-04-25T00:21:57.569175+0000","last_became_active":"2024-04-25T00:20:27.154058+0000","last_became_peered":"2024-04-25T00:20:27.154058+0000","last_unstale":"2024-04-25T00:21:57.569175+0000","last_undegraded":"2024-04-25T00:21:57.569175+0000","last_fullsized":"2024-04-25T00:21:57.569175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:55.519210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555526+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:21:57.555526+0000","last_peered":"2024-04-25T00:21:57.555526+0000","last_clean":"2024-04-25T00:21:57.555526+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:21:57.555526+0000","last_undegraded":"2024-04-25T00:21:57.555526+0000","last_fullsized":"2024-04-25T00:21:57.555526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473816+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:21:57.473816+0000","last_peered":"2024-04-25T00:21:57.473816+0000","last_clean":"2024-04-25T00:21:57.473816+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:21:57.473816+0000","last_undegraded":"2024-04-25T00:21:57.473816+0000","last_fullsized":"2024-04-25T00:21:57.473816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479776+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:21:57.479776+0000","last_peered":"2024-04-25T00:21:57.479776+0000","last_clean":"2024-04-25T00:21:57.479776+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:21:57.479776+0000","last_undegraded":"2024-04-25T00:21:57.479776+0000","last_fullsized":"2024-04-25T00:21:57.479776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369325+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:21:57.369325+0000","last_peered":"2024-04-25T00:21:57.369325+0000","last_clean":"2024-04-25T00:21:57.369325+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:21:57.369325+0000","last_undegraded":"2024-04-25T00:21:57.369325+0000","last_fullsized":"2024-04-25T00:21:57.369325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006431+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:21:58.006431+0000","last_peered":"2024-04-25T00:21:58.006431+0000","last_clean":"2024-04-25T00:21:58.006431+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:21:58.006431+0000","last_undegraded":"2024-04-25T00:21:58.006431+0000","last_fullsized":"2024-04-25T00:21:58.006431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474170+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:21:57.474170+0000","last_peered":"2024-04-25T00:21:57.474170+0000","last_clean":"2024-04-25T00:21:57.474170+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:21:57.474170+0000","last_undegraded":"2024-04-25T00:21:57.474170+0000","last_fullsized":"2024-04-25T00:21:57.474170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369206+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:21:57.369206+0000","last_peered":"2024-04-25T00:21:57.369206+0000","last_clean":"2024-04-25T00:21:57.369206+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:21:57.369206+0000","last_undegraded":"2024-04-25T00:21:57.369206+0000","last_fullsized":"2024-04-25T00:21:57.369206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445520+0000","last_change":"2024-04-25T00:20:27.054883+0000","last_active":"2024-04-25T00:21:57.445520+0000","last_peered":"2024-04-25T00:21:57.445520+0000","last_clean":"2024-04-25T00:21:57.445520+0000","last_became_active":"2024-04-25T00:20:27.049595+0000","last_became_peered":"2024-04-25T00:20:27.049595+0000","last_unstale":"2024-04-25T00:21:57.445520+0000","last_undegraded":"2024-04-25T00:21:57.445520+0000","last_fullsized":"2024-04-25T00:21:57.445520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:05.358193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479893+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:21:57.479893+0000","last_peered":"2024-04-25T00:21:57.479893+0000","last_clean":"2024-04-25T00:21:57.479893+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:21:57.479893+0000","last_undegraded":"2024-04-25T00:21:57.479893+0000","last_fullsized":"2024-04-25T00:21:57.479893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569179+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:21:57.569179+0000","last_peered":"2024-04-25T00:21:57.569179+0000","last_clean":"2024-04-25T00:21:57.569179+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:21:57.569179+0000","last_undegraded":"2024-04-25T00:21:57.569179+0000","last_fullsized":"2024-04-25T00:21:57.569179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569684+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:21:57.569684+0000","last_peered":"2024-04-25T00:21:57.569684+0000","last_clean":"2024-04-25T00:21:57.569684+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:21:57.569684+0000","last_undegraded":"2024-04-25T00:21:57.569684+0000","last_fullsized":"2024-04-25T00:21:57.569684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367172+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:21:57.367172+0000","last_peered":"2024-04-25T00:21:57.367172+0000","last_clean":"2024-04-25T00:21:57.367172+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:21:57.367172+0000","last_undegraded":"2024-04-25T00:21:57.367172+0000","last_fullsized":"2024-04-25T00:21:57.367172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570117+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:21:57.570117+0000","last_peered":"2024-04-25T00:21:57.570117+0000","last_clean":"2024-04-25T00:21:57.570117+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:21:57.570117+0000","last_undegraded":"2024-04-25T00:21:57.570117+0000","last_fullsized":"2024-04-25T00:21:57.570117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479533+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:21:57.479533+0000","last_peered":"2024-04-25T00:21:57.479533+0000","last_clean":"2024-04-25T00:21:57.479533+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:21:57.479533+0000","last_undegraded":"2024-04-25T00:21:57.479533+0000","last_fullsized":"2024-04-25T00:21:57.479533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471265+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:21:57.471265+0000","last_peered":"2024-04-25T00:21:57.471265+0000","last_clean":"2024-04-25T00:21:57.471265+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:21:57.471265+0000","last_undegraded":"2024-04-25T00:21:57.471265+0000","last_fullsized":"2024-04-25T00:21:57.471265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367392+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:21:57.367392+0000","last_peered":"2024-04-25T00:21:57.367392+0000","last_clean":"2024-04-25T00:21:57.367392+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:21:57.367392+0000","last_undegraded":"2024-04-25T00:21:57.367392+0000","last_fullsized":"2024-04-25T00:21:57.367392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":274,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006037+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:21:58.006037+0000","last_peered":"2024-04-25T00:21:58.006037+0000","last_clean":"2024-04-25T00:21:58.006037+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:21:58.006037+0000","last_undegraded":"2024-04-25T00:21:58.006037+0000","last_fullsized":"2024-04-25T00:21:58.006037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556589+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:21:57.556589+0000","last_peered":"2024-04-25T00:21:57.556589+0000","last_clean":"2024-04-25T00:21:57.556589+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:21:57.556589+0000","last_undegraded":"2024-04-25T00:21:57.556589+0000","last_fullsized":"2024-04-25T00:21:57.556589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471195+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:21:57.471195+0000","last_peered":"2024-04-25T00:21:57.471195+0000","last_clean":"2024-04-25T00:21:57.471195+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:21:57.471195+0000","last_undegraded":"2024-04-25T00:21:57.471195+0000","last_fullsized":"2024-04-25T00:21:57.471195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569301+0000","last_change":"2024-04-25T00:20:27.176907+0000","last_active":"2024-04-25T00:21:57.569301+0000","last_peered":"2024-04-25T00:21:57.569301+0000","last_clean":"2024-04-25T00:21:57.569301+0000","last_became_active":"2024-04-25T00:20:27.153755+0000","last_became_peered":"2024-04-25T00:20:27.153755+0000","last_unstale":"2024-04-25T00:21:57.569301+0000","last_undegraded":"2024-04-25T00:21:57.569301+0000","last_fullsized":"2024-04-25T00:21:57.569301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:07.142383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570525+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:21:57.570525+0000","last_peered":"2024-04-25T00:21:57.570525+0000","last_clean":"2024-04-25T00:21:57.570525+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:21:57.570525+0000","last_undegraded":"2024-04-25T00:21:57.570525+0000","last_fullsized":"2024-04-25T00:21:57.570525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368645+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:21:57.368645+0000","last_peered":"2024-04-25T00:21:57.368645+0000","last_clean":"2024-04-25T00:21:57.368645+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:21:57.368645+0000","last_undegraded":"2024-04-25T00:21:57.368645+0000","last_fullsized":"2024-04-25T00:21:57.368645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479864+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:21:57.479864+0000","last_peered":"2024-04-25T00:21:57.479864+0000","last_clean":"2024-04-25T00:21:57.479864+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:21:57.479864+0000","last_undegraded":"2024-04-25T00:21:57.479864+0000","last_fullsized":"2024-04-25T00:21:57.479864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555798+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:21:57.555798+0000","last_peered":"2024-04-25T00:21:57.555798+0000","last_clean":"2024-04-25T00:21:57.555798+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:21:57.555798+0000","last_undegraded":"2024-04-25T00:21:57.555798+0000","last_fullsized":"2024-04-25T00:21:57.555798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471386+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:21:57.471386+0000","last_peered":"2024-04-25T00:21:57.471386+0000","last_clean":"2024-04-25T00:21:57.471386+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:21:57.471386+0000","last_undegraded":"2024-04-25T00:21:57.471386+0000","last_fullsized":"2024-04-25T00:21:57.471386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006697+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:21:58.006697+0000","last_peered":"2024-04-25T00:21:58.006697+0000","last_clean":"2024-04-25T00:21:58.006697+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:21:58.006697+0000","last_undegraded":"2024-04-25T00:21:58.006697+0000","last_fullsized":"2024-04-25T00:21:58.006697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474031+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:21:57.474031+0000","last_peered":"2024-04-25T00:21:57.474031+0000","last_clean":"2024-04-25T00:21:57.474031+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:21:57.474031+0000","last_undegraded":"2024-04-25T00:21:57.474031+0000","last_fullsized":"2024-04-25T00:21:57.474031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.480095+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:21:57.480095+0000","last_peered":"2024-04-25T00:21:57.480095+0000","last_clean":"2024-04-25T00:21:57.480095+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:21:57.480095+0000","last_undegraded":"2024-04-25T00:21:57.480095+0000","last_fullsized":"2024-04-25T00:21:57.480095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369252+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:21:57.369252+0000","last_peered":"2024-04-25T00:21:57.369252+0000","last_clean":"2024-04-25T00:21:57.369252+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:21:57.369252+0000","last_undegraded":"2024-04-25T00:21:57.369252+0000","last_fullsized":"2024-04-25T00:21:57.369252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570057+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:21:57.570057+0000","last_peered":"2024-04-25T00:21:57.570057+0000","last_clean":"2024-04-25T00:21:57.570057+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:21:57.570057+0000","last_undegraded":"2024-04-25T00:21:57.570057+0000","last_fullsized":"2024-04-25T00:21:57.570057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568987+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:21:57.568987+0000","last_peered":"2024-04-25T00:21:57.568987+0000","last_clean":"2024-04-25T00:21:57.568987+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:21:57.568987+0000","last_undegraded":"2024-04-25T00:21:57.568987+0000","last_fullsized":"2024-04-25T00:21:57.568987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479870+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:21:57.479870+0000","last_peered":"2024-04-25T00:21:57.479870+0000","last_clean":"2024-04-25T00:21:57.479870+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:21:57.479870+0000","last_undegraded":"2024-04-25T00:21:57.479870+0000","last_fullsized":"2024-04-25T00:21:57.479870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569260+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:21:57.569260+0000","last_peered":"2024-04-25T00:21:57.569260+0000","last_clean":"2024-04-25T00:21:57.569260+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:21:57.569260+0000","last_undegraded":"2024-04-25T00:21:57.569260+0000","last_fullsized":"2024-04-25T00:21:57.569260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474056+0000","last_change":"2024-04-25T00:20:27.153934+0000","last_active":"2024-04-25T00:21:57.474056+0000","last_peered":"2024-04-25T00:21:57.474056+0000","last_clean":"2024-04-25T00:21:57.474056+0000","last_became_active":"2024-04-25T00:20:27.152647+0000","last_became_peered":"2024-04-25T00:20:27.152647+0000","last_unstale":"2024-04-25T00:21:57.474056+0000","last_undegraded":"2024-04-25T00:21:57.474056+0000","last_fullsized":"2024-04-25T00:21:57.474056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.389722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.019845+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:21:58.019845+0000","last_peered":"2024-04-25T00:21:58.019845+0000","last_clean":"2024-04-25T00:21:58.019845+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:21:58.019845+0000","last_undegraded":"2024-04-25T00:21:58.019845+0000","last_fullsized":"2024-04-25T00:21:58.019845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569027+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:21:57.569027+0000","last_peered":"2024-04-25T00:21:57.569027+0000","last_clean":"2024-04-25T00:21:57.569027+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:21:57.569027+0000","last_undegraded":"2024-04-25T00:21:57.569027+0000","last_fullsized":"2024-04-25T00:21:57.569027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571825+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:21:57.571825+0000","last_peered":"2024-04-25T00:21:57.571825+0000","last_clean":"2024-04-25T00:21:57.571825+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:21:57.571825+0000","last_undegraded":"2024-04-25T00:21:57.571825+0000","last_fullsized":"2024-04-25T00:21:57.571825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473779+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:21:57.473779+0000","last_peered":"2024-04-25T00:21:57.473779+0000","last_clean":"2024-04-25T00:21:57.473779+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:21:57.473779+0000","last_undegraded":"2024-04-25T00:21:57.473779+0000","last_fullsized":"2024-04-25T00:21:57.473779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479596+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:21:57.479596+0000","last_peered":"2024-04-25T00:21:57.479596+0000","last_clean":"2024-04-25T00:21:57.479596+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:21:57.479596+0000","last_undegraded":"2024-04-25T00:21:57.479596+0000","last_fullsized":"2024-04-25T00:21:57.479596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479496+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:21:57.479496+0000","last_peered":"2024-04-25T00:21:57.479496+0000","last_clean":"2024-04-25T00:21:57.479496+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:21:57.479496+0000","last_undegraded":"2024-04-25T00:21:57.479496+0000","last_fullsized":"2024-04-25T00:21:57.479496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474293+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:21:57.474293+0000","last_peered":"2024-04-25T00:21:57.474293+0000","last_clean":"2024-04-25T00:21:57.474293+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:21:57.474293+0000","last_undegraded":"2024-04-25T00:21:57.474293+0000","last_fullsized":"2024-04-25T00:21:57.474293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569223+0000","last_change":"2024-04-25T00:21:27.923915+0000","last_active":"2024-04-25T00:21:57.569223+0000","last_peered":"2024-04-25T00:21:57.569223+0000","last_clean":"2024-04-25T00:21:57.569223+0000","last_became_active":"2024-04-25T00:21:27.923465+0000","last_became_peered":"2024-04-25T00:21:27.923465+0000","last_unstale":"2024-04-25T00:21:57.569223+0000","last_undegraded":"2024-04-25T00:21:57.569223+0000","last_fullsized":"2024-04-25T00:21:57.569223+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.804110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473722+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:21:57.473722+0000","last_peered":"2024-04-25T00:21:57.473722+0000","last_clean":"2024-04-25T00:21:57.473722+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:21:57.473722+0000","last_undegraded":"2024-04-25T00:21:57.473722+0000","last_fullsized":"2024-04-25T00:21:57.473722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471729+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:21:57.471729+0000","last_peered":"2024-04-25T00:21:57.471729+0000","last_clean":"2024-04-25T00:21:57.471729+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:21:57.471729+0000","last_undegraded":"2024-04-25T00:21:57.471729+0000","last_fullsized":"2024-04-25T00:21:57.471729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445825+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:21:57.445825+0000","last_peered":"2024-04-25T00:21:57.445825+0000","last_clean":"2024-04-25T00:21:57.445825+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:21:57.445825+0000","last_undegraded":"2024-04-25T00:21:57.445825+0000","last_fullsized":"2024-04-25T00:21:57.445825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571137+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:21:57.571137+0000","last_peered":"2024-04-25T00:21:57.571137+0000","last_clean":"2024-04-25T00:21:57.571137+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:21:57.571137+0000","last_undegraded":"2024-04-25T00:21:57.571137+0000","last_fullsized":"2024-04-25T00:21:57.571137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569036+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:21:57.569036+0000","last_peered":"2024-04-25T00:21:57.569036+0000","last_clean":"2024-04-25T00:21:57.569036+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:21:57.569036+0000","last_undegraded":"2024-04-25T00:21:57.569036+0000","last_fullsized":"2024-04-25T00:21:57.569036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445627+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:21:57.445627+0000","last_peered":"2024-04-25T00:21:57.445627+0000","last_clean":"2024-04-25T00:21:57.445627+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:21:57.445627+0000","last_undegraded":"2024-04-25T00:21:57.445627+0000","last_fullsized":"2024-04-25T00:21:57.445627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571805+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:21:57.571805+0000","last_peered":"2024-04-25T00:21:57.571805+0000","last_clean":"2024-04-25T00:21:57.571805+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:21:57.571805+0000","last_undegraded":"2024-04-25T00:21:57.571805+0000","last_fullsized":"2024-04-25T00:21:57.571805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479762+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:21:57.479762+0000","last_peered":"2024-04-25T00:21:57.479762+0000","last_clean":"2024-04-25T00:21:57.479762+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:21:57.479762+0000","last_undegraded":"2024-04-25T00:21:57.479762+0000","last_fullsized":"2024-04-25T00:21:57.479762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.707830+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:21:57.707830+0000","last_peered":"2024-04-25T00:21:57.707830+0000","last_clean":"2024-04-25T00:21:57.707830+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:21:57.707830+0000","last_undegraded":"2024-04-25T00:21:57.707830+0000","last_fullsized":"2024-04-25T00:21:57.707830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367200+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:21:57.367200+0000","last_peered":"2024-04-25T00:21:57.367200+0000","last_clean":"2024-04-25T00:21:57.367200+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:21:57.367200+0000","last_undegraded":"2024-04-25T00:21:57.367200+0000","last_fullsized":"2024-04-25T00:21:57.367200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368588+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:21:57.368588+0000","last_peered":"2024-04-25T00:21:57.368588+0000","last_clean":"2024-04-25T00:21:57.368588+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:21:57.368588+0000","last_undegraded":"2024-04-25T00:21:57.368588+0000","last_fullsized":"2024-04-25T00:21:57.368588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568856+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:21:57.568856+0000","last_peered":"2024-04-25T00:21:57.568856+0000","last_clean":"2024-04-25T00:21:57.568856+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:21:57.568856+0000","last_undegraded":"2024-04-25T00:21:57.568856+0000","last_fullsized":"2024-04-25T00:21:57.568856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474047+0000","last_change":"2024-04-25T00:20:27.049784+0000","last_active":"2024-04-25T00:21:57.474047+0000","last_peered":"2024-04-25T00:21:57.474047+0000","last_clean":"2024-04-25T00:21:57.474047+0000","last_became_active":"2024-04-25T00:20:27.049413+0000","last_became_peered":"2024-04-25T00:20:27.049413+0000","last_unstale":"2024-04-25T00:21:57.474047+0000","last_undegraded":"2024-04-25T00:21:57.474047+0000","last_fullsized":"2024-04-25T00:21:57.474047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:26.533987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569353+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:21:57.569353+0000","last_peered":"2024-04-25T00:21:57.569353+0000","last_clean":"2024-04-25T00:21:57.569353+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:21:57.569353+0000","last_undegraded":"2024-04-25T00:21:57.569353+0000","last_fullsized":"2024-04-25T00:21:57.569353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568945+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:21:57.568945+0000","last_peered":"2024-04-25T00:21:57.568945+0000","last_clean":"2024-04-25T00:21:57.568945+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:21:57.568945+0000","last_undegraded":"2024-04-25T00:21:57.568945+0000","last_fullsized":"2024-04-25T00:21:57.568945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474064+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:21:57.474064+0000","last_peered":"2024-04-25T00:21:57.474064+0000","last_clean":"2024-04-25T00:21:57.474064+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:21:57.474064+0000","last_undegraded":"2024-04-25T00:21:57.474064+0000","last_fullsized":"2024-04-25T00:21:57.474064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571331+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:21:57.571331+0000","last_peered":"2024-04-25T00:21:57.571331+0000","last_clean":"2024-04-25T00:21:57.571331+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:21:57.571331+0000","last_undegraded":"2024-04-25T00:21:57.571331+0000","last_fullsized":"2024-04-25T00:21:57.571331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367171+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:21:57.367171+0000","last_peered":"2024-04-25T00:21:57.367171+0000","last_clean":"2024-04-25T00:21:57.367171+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:21:57.367171+0000","last_undegraded":"2024-04-25T00:21:57.367171+0000","last_fullsized":"2024-04-25T00:21:57.367171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.598104+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:21:57.598104+0000","last_peered":"2024-04-25T00:21:57.598104+0000","last_clean":"2024-04-25T00:21:57.598104+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:21:57.598104+0000","last_undegraded":"2024-04-25T00:21:57.598104+0000","last_fullsized":"2024-04-25T00:21:57.598104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368050+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:21:57.368050+0000","last_peered":"2024-04-25T00:21:57.368050+0000","last_clean":"2024-04-25T00:21:57.368050+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:21:57.368050+0000","last_undegraded":"2024-04-25T00:21:57.368050+0000","last_fullsized":"2024-04-25T00:21:57.368050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368768+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:21:57.368768+0000","last_peered":"2024-04-25T00:21:57.368768+0000","last_clean":"2024-04-25T00:21:57.368768+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:21:57.368768+0000","last_undegraded":"2024-04-25T00:21:57.368768+0000","last_fullsized":"2024-04-25T00:21:57.368768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368990+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:21:57.368990+0000","last_peered":"2024-04-25T00:21:57.368990+0000","last_clean":"2024-04-25T00:21:57.368990+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:21:57.368990+0000","last_undegraded":"2024-04-25T00:21:57.368990+0000","last_fullsized":"2024-04-25T00:21:57.368990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570368+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:21:57.570368+0000","last_peered":"2024-04-25T00:21:57.570368+0000","last_clean":"2024-04-25T00:21:57.570368+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:21:57.570368+0000","last_undegraded":"2024-04-25T00:21:57.570368+0000","last_fullsized":"2024-04-25T00:21:57.570368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368871+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:21:57.368871+0000","last_peered":"2024-04-25T00:21:57.368871+0000","last_clean":"2024-04-25T00:21:57.368871+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:21:57.368871+0000","last_undegraded":"2024-04-25T00:21:57.368871+0000","last_fullsized":"2024-04-25T00:21:57.368871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555187+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:21:57.555187+0000","last_peered":"2024-04-25T00:21:57.555187+0000","last_clean":"2024-04-25T00:21:57.555187+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:21:57.555187+0000","last_undegraded":"2024-04-25T00:21:57.555187+0000","last_fullsized":"2024-04-25T00:21:57.555187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473693+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:21:57.473693+0000","last_peered":"2024-04-25T00:21:57.473693+0000","last_clean":"2024-04-25T00:21:57.473693+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:21:57.473693+0000","last_undegraded":"2024-04-25T00:21:57.473693+0000","last_fullsized":"2024-04-25T00:21:57.473693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474191+0000","last_change":"2024-04-25T00:20:27.153703+0000","last_active":"2024-04-25T00:21:57.474191+0000","last_peered":"2024-04-25T00:21:57.474191+0000","last_clean":"2024-04-25T00:21:57.474191+0000","last_became_active":"2024-04-25T00:20:27.152715+0000","last_became_peered":"2024-04-25T00:20:27.152715+0000","last_unstale":"2024-04-25T00:21:57.474191+0000","last_undegraded":"2024-04-25T00:21:57.474191+0000","last_fullsized":"2024-04-25T00:21:57.474191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:09.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367131+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:21:57.367131+0000","last_peered":"2024-04-25T00:21:57.367131+0000","last_clean":"2024-04-25T00:21:57.367131+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:21:57.367131+0000","last_undegraded":"2024-04-25T00:21:57.367131+0000","last_fullsized":"2024-04-25T00:21:57.367131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568847+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:21:57.568847+0000","last_peered":"2024-04-25T00:21:57.568847+0000","last_clean":"2024-04-25T00:21:57.568847+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:21:57.568847+0000","last_undegraded":"2024-04-25T00:21:57.568847+0000","last_fullsized":"2024-04-25T00:21:57.568847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368683+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:21:57.368683+0000","last_peered":"2024-04-25T00:21:57.368683+0000","last_clean":"2024-04-25T00:21:57.368683+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:21:57.368683+0000","last_undegraded":"2024-04-25T00:21:57.368683+0000","last_fullsized":"2024-04-25T00:21:57.368683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369170+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:21:57.369170+0000","last_peered":"2024-04-25T00:21:57.369170+0000","last_clean":"2024-04-25T00:21:57.369170+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:21:57.369170+0000","last_undegraded":"2024-04-25T00:21:57.369170+0000","last_fullsized":"2024-04-25T00:21:57.369170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367712+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:21:57.367712+0000","last_peered":"2024-04-25T00:21:57.367712+0000","last_clean":"2024-04-25T00:21:57.367712+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:21:57.367712+0000","last_undegraded":"2024-04-25T00:21:57.367712+0000","last_fullsized":"2024-04-25T00:21:57.367712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368859+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:21:57.368859+0000","last_peered":"2024-04-25T00:21:57.368859+0000","last_clean":"2024-04-25T00:21:57.368859+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:21:57.368859+0000","last_undegraded":"2024-04-25T00:21:57.368859+0000","last_fullsized":"2024-04-25T00:21:57.368859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473798+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:21:57.473798+0000","last_peered":"2024-04-25T00:21:57.473798+0000","last_clean":"2024-04-25T00:21:57.473798+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:21:57.473798+0000","last_undegraded":"2024-04-25T00:21:57.473798+0000","last_fullsized":"2024-04-25T00:21:57.473798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555469+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:21:57.555469+0000","last_peered":"2024-04-25T00:21:57.555469+0000","last_clean":"2024-04-25T00:21:57.555469+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:21:57.555469+0000","last_undegraded":"2024-04-25T00:21:57.555469+0000","last_fullsized":"2024-04-25T00:21:57.555469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369406+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:21:57.369406+0000","last_peered":"2024-04-25T00:21:57.369406+0000","last_clean":"2024-04-25T00:21:57.369406+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:21:57.369406+0000","last_undegraded":"2024-04-25T00:21:57.369406+0000","last_fullsized":"2024-04-25T00:21:57.369406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569340+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:21:57.569340+0000","last_peered":"2024-04-25T00:21:57.569340+0000","last_clean":"2024-04-25T00:21:57.569340+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:21:57.569340+0000","last_undegraded":"2024-04-25T00:21:57.569340+0000","last_fullsized":"2024-04-25T00:21:57.569340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570576+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:21:57.570576+0000","last_peered":"2024-04-25T00:21:57.570576+0000","last_clean":"2024-04-25T00:21:57.570576+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:21:57.570576+0000","last_undegraded":"2024-04-25T00:21:57.570576+0000","last_fullsized":"2024-04-25T00:21:57.570576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.599346+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:21:57.599346+0000","last_peered":"2024-04-25T00:21:57.599346+0000","last_clean":"2024-04-25T00:21:57.599346+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:21:57.599346+0000","last_undegraded":"2024-04-25T00:21:57.599346+0000","last_fullsized":"2024-04-25T00:21:57.599346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569431+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:21:57.569431+0000","last_peered":"2024-04-25T00:21:57.569431+0000","last_clean":"2024-04-25T00:21:57.569431+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:21:57.569431+0000","last_undegraded":"2024-04-25T00:21:57.569431+0000","last_fullsized":"2024-04-25T00:21:57.569431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479319+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:21:57.479319+0000","last_peered":"2024-04-25T00:21:57.479319+0000","last_clean":"2024-04-25T00:21:57.479319+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:21:57.479319+0000","last_undegraded":"2024-04-25T00:21:57.479319+0000","last_fullsized":"2024-04-25T00:21:57.479319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571948+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:21:57.571948+0000","last_peered":"2024-04-25T00:21:57.571948+0000","last_clean":"2024-04-25T00:21:57.571948+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:21:57.571948+0000","last_undegraded":"2024-04-25T00:21:57.571948+0000","last_fullsized":"2024-04-25T00:21:57.571948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569558+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:21:57.569558+0000","last_peered":"2024-04-25T00:21:57.569558+0000","last_clean":"2024-04-25T00:21:57.569558+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:21:57.569558+0000","last_undegraded":"2024-04-25T00:21:57.569558+0000","last_fullsized":"2024-04-25T00:21:57.569558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568933+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:21:57.568933+0000","last_peered":"2024-04-25T00:21:57.568933+0000","last_clean":"2024-04-25T00:21:57.568933+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:21:57.568933+0000","last_undegraded":"2024-04-25T00:21:57.568933+0000","last_fullsized":"2024-04-25T00:21:57.568933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369093+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:21:57.369093+0000","last_peered":"2024-04-25T00:21:57.369093+0000","last_clean":"2024-04-25T00:21:57.369093+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:21:57.369093+0000","last_undegraded":"2024-04-25T00:21:57.369093+0000","last_fullsized":"2024-04-25T00:21:57.369093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368834+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:21:57.368834+0000","last_peered":"2024-04-25T00:21:57.368834+0000","last_clean":"2024-04-25T00:21:57.368834+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:21:57.368834+0000","last_undegraded":"2024-04-25T00:21:57.368834+0000","last_fullsized":"2024-04-25T00:21:57.368834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570810+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:21:57.570810+0000","last_peered":"2024-04-25T00:21:57.570810+0000","last_clean":"2024-04-25T00:21:57.570810+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:21:57.570810+0000","last_undegraded":"2024-04-25T00:21:57.570810+0000","last_fullsized":"2024-04-25T00:21:57.570810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571459+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:21:57.571459+0000","last_peered":"2024-04-25T00:21:57.571459+0000","last_clean":"2024-04-25T00:21:57.571459+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:21:57.571459+0000","last_undegraded":"2024-04-25T00:21:57.571459+0000","last_fullsized":"2024-04-25T00:21:57.571459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569203+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:21:57.569203+0000","last_peered":"2024-04-25T00:21:57.569203+0000","last_clean":"2024-04-25T00:21:57.569203+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:21:57.569203+0000","last_undegraded":"2024-04-25T00:21:57.569203+0000","last_fullsized":"2024-04-25T00:21:57.569203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369599+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:21:57.369599+0000","last_peered":"2024-04-25T00:21:57.369599+0000","last_clean":"2024-04-25T00:21:57.369599+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:21:57.369599+0000","last_undegraded":"2024-04-25T00:21:57.369599+0000","last_fullsized":"2024-04-25T00:21:57.369599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368190+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:21:57.368190+0000","last_peered":"2024-04-25T00:21:57.368190+0000","last_clean":"2024-04-25T00:21:57.368190+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:21:57.368190+0000","last_undegraded":"2024-04-25T00:21:57.368190+0000","last_fullsized":"2024-04-25T00:21:57.368190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479506+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:21:57.479506+0000","last_peered":"2024-04-25T00:21:57.479506+0000","last_clean":"2024-04-25T00:21:57.479506+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:21:57.479506+0000","last_undegraded":"2024-04-25T00:21:57.479506+0000","last_fullsized":"2024-04-25T00:21:57.479506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367262+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:21:57.367262+0000","last_peered":"2024-04-25T00:21:57.367262+0000","last_clean":"2024-04-25T00:21:57.367262+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:21:57.367262+0000","last_undegraded":"2024-04-25T00:21:57.367262+0000","last_fullsized":"2024-04-25T00:21:57.367262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555764+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:21:57.555764+0000","last_peered":"2024-04-25T00:21:57.555764+0000","last_clean":"2024-04-25T00:21:57.555764+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:21:57.555764+0000","last_undegraded":"2024-04-25T00:21:57.555764+0000","last_fullsized":"2024-04-25T00:21:57.555764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570598+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:21:57.570598+0000","last_peered":"2024-04-25T00:21:57.570598+0000","last_clean":"2024-04-25T00:21:57.570598+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:21:57.570598+0000","last_undegraded":"2024-04-25T00:21:57.570598+0000","last_fullsized":"2024-04-25T00:21:57.570598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569041+0000","last_change":"2024-04-25T00:20:27.046816+0000","last_active":"2024-04-25T00:21:57.569041+0000","last_peered":"2024-04-25T00:21:57.569041+0000","last_clean":"2024-04-25T00:21:57.569041+0000","last_became_active":"2024-04-25T00:20:27.045845+0000","last_became_peered":"2024-04-25T00:20:27.045845+0000","last_unstale":"2024-04-25T00:21:57.569041+0000","last_undegraded":"2024-04-25T00:21:57.569041+0000","last_fullsized":"2024-04-25T00:21:57.569041+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:45.465308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108492,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599296,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1299,"num_read_kb":761,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1797888,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":530611,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28200,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716952,"statfs":{"total":95995035648,"available":95966158848,"internally_reserved":0,"allocated":1212416,"data_stored":1315182,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":927181,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":923463,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":1127835,"data_compressed":65691,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":525373,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":592307,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":526829,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":593758,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1049897,"data_compressed":54691,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":530370,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1254766,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":200163,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:21:59.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:22:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 -- 172.21.15.45:0/1690857801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc102220 msgr2=0x7f7abc104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/1690857801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc102220 0x7f7abc104610 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009900 tx=0x7f7aac02a410 comp rx=0 tx=0).stop 2024-04-25T00:22:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 -- 172.21.15.45:0/1690857801 shutdown_connections 2024-04-25T00:22:00.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/1690857801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc107480 0x7f7abc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/1690857801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/1690857801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc102220 0x7f7abc104610 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 -- 172.21.15.45:0/1690857801 >> 172.21.15.45:0/1690857801 conn(0x7f7abc0fbc30 msgr2=0x7f7abc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 -- 172.21.15.45:0/1690857801 shutdown_connections 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.025+0000 7f7ac22ae640 1 -- 172.21.15.45:0/1690857801 wait complete. 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 Processor -- start 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 -- start start 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc102220 0x7f7abc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc10d440 con 0x7f7abc107480 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc10d2c0 con 0x7f7abc104b50 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac22ae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc10d5c0 con 0x7f7abc102220 2024-04-25T00:22:00.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac084b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc102220 0x7f7abc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac084b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52816/0 (socket says 172.21.15.45:52816) 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7ac084b640 1 -- 172.21.15.45:0/386935990 learned_addr learned my addr 172.21.15.45:0/386935990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.026+0000 7f7abb7fe640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 -- 172.21.15.45:0/386935990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc102220 msgr2=0x7f7abc197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc102220 0x7f7abc197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 -- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 msgr2=0x7f7abc19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 -- 172.21.15.45:0/386935990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc19f700 con 0x7f7abc104b50 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ac084b640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:00.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7abb7fe640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc198020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000e9c0 tx=0x7f7ab000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000ccc0 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc19f9f0 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0010910 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0018880 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.027+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc19ff40 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.028+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc0fd720 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.029+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f7ab0010ac0 con 0x7f7abc104b50 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.030+0000 7f7ab97fa640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 0x7f7aa40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.030+0000 7f7abbfff640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 0x7f7aa40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.030+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7ab00cfdf0 con 0x7f7abc104b50 2024-04-25T00:22:00.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.031+0000 7f7abbfff640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 0x7f7aa40a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0314e0 tx=0x7f7aac000bc0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.033+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ab009c060 con 0x7f7abc104b50 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: [25/Apr/2024:00:21:58] ENGINE Bus STARTING 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: [25/Apr/2024:00:21:58] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: [25/Apr/2024:00:21:58] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: [25/Apr/2024:00:21:58] ENGINE Bus STARTED 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/238494546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/604158857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4055968370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:21:59 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.217+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7abc061b80 con 0x7f7aa40a2d00 2024-04-25T00:22:00.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.237+0000 7f7ab97fa640 1 -- 172.21.15.45:0/386935990 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339719 (secure 0 0 0) 0x7f7abc061b80 con 0x7f7aa40a2d00 2024-04-25T00:22:00.241 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:22:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 msgr2=0x7f7aa40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 0x7f7aa40a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0314e0 tx=0x7f7aac000bc0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 msgr2=0x7f7abc198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc198020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000e9c0 tx=0x7f7ab000ee80 comp rx=0 tx=0).stop 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 shutdown_connections 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7aa40a2d00 0x7f7aa40a51c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7abc107480 0x7f7abc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc104b50 0x7f7abc198020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 --2- 172.21.15.45:0/386935990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7abc102220 0x7f7abc197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 >> 172.21.15.45:0/386935990 conn(0x7f7abc0fbc30 msgr2=0x7f7abc102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 shutdown_connections 2024-04-25T00:22:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.241+0000 7f7ac22ae640 1 -- 172.21.15.45:0/386935990 wait complete. 2024-04-25T00:22:00.252 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-25T00:21:59.380607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189898,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1435,"num_read_kb":880,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391176,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550648,"statfs":{"total":1151940427776,"available":1151539863552,"internally_reserved":0,"allocated":13180928,"data_stored":9897572,"data_compressed":538584,"data_compressed_allocated":724992,"data_compressed_original":4300800,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.985744"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570237+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:21:57.570237+0000","last_peered":"2024-04-25T00:21:57.570237+0000","last_clean":"2024-04-25T00:21:57.570237+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:21:57.570237+0000","last_undegraded":"2024-04-25T00:21:57.570237+0000","last_fullsized":"2024-04-25T00:21:57.570237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569393+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:21:57.569393+0000","last_peered":"2024-04-25T00:21:57.569393+0000","last_clean":"2024-04-25T00:21:57.569393+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:21:57.569393+0000","last_undegraded":"2024-04-25T00:21:57.569393+0000","last_fullsized":"2024-04-25T00:21:57.569393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471340+0000","last_change":"2024-04-25T00:21:27.923300+0000","last_active":"2024-04-25T00:21:57.471340+0000","last_peered":"2024-04-25T00:21:57.471340+0000","last_clean":"2024-04-25T00:21:57.471340+0000","last_became_active":"2024-04-25T00:21:27.922951+0000","last_became_peered":"2024-04-25T00:21:27.922951+0000","last_unstale":"2024-04-25T00:21:57.471340+0000","last_undegraded":"2024-04-25T00:21:57.471340+0000","last_fullsized":"2024-04-25T00:21:57.471340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:50.728710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473760+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:21:57.473760+0000","last_peered":"2024-04-25T00:21:57.473760+0000","last_clean":"2024-04-25T00:21:57.473760+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:21:57.473760+0000","last_undegraded":"2024-04-25T00:21:57.473760+0000","last_fullsized":"2024-04-25T00:21:57.473760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445544+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:21:57.445544+0000","last_peered":"2024-04-25T00:21:57.445544+0000","last_clean":"2024-04-25T00:21:57.445544+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:21:57.445544+0000","last_undegraded":"2024-04-25T00:21:57.445544+0000","last_fullsized":"2024-04-25T00:21:57.445544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367303+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:21:57.367303+0000","last_peered":"2024-04-25T00:21:57.367303+0000","last_clean":"2024-04-25T00:21:57.367303+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:21:57.367303+0000","last_undegraded":"2024-04-25T00:21:57.367303+0000","last_fullsized":"2024-04-25T00:21:57.367303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556792+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:21:57.556792+0000","last_peered":"2024-04-25T00:21:57.556792+0000","last_clean":"2024-04-25T00:21:57.556792+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:21:57.556792+0000","last_undegraded":"2024-04-25T00:21:57.556792+0000","last_fullsized":"2024-04-25T00:21:57.556792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445954+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:21:57.445954+0000","last_peered":"2024-04-25T00:21:57.445954+0000","last_clean":"2024-04-25T00:21:57.445954+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:21:57.445954+0000","last_undegraded":"2024-04-25T00:21:57.445954+0000","last_fullsized":"2024-04-25T00:21:57.445954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368761+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:21:57.368761+0000","last_peered":"2024-04-25T00:21:57.368761+0000","last_clean":"2024-04-25T00:21:57.368761+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:21:57.368761+0000","last_undegraded":"2024-04-25T00:21:57.368761+0000","last_fullsized":"2024-04-25T00:21:57.368761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570829+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:21:57.570829+0000","last_peered":"2024-04-25T00:21:57.570829+0000","last_clean":"2024-04-25T00:21:57.570829+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:21:57.570829+0000","last_undegraded":"2024-04-25T00:21:57.570829+0000","last_fullsized":"2024-04-25T00:21:57.570829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571324+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:21:57.571324+0000","last_peered":"2024-04-25T00:21:57.571324+0000","last_clean":"2024-04-25T00:21:57.571324+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:21:57.571324+0000","last_undegraded":"2024-04-25T00:21:57.571324+0000","last_fullsized":"2024-04-25T00:21:57.571324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570961+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:21:57.570961+0000","last_peered":"2024-04-25T00:21:57.570961+0000","last_clean":"2024-04-25T00:21:57.570961+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:21:57.570961+0000","last_undegraded":"2024-04-25T00:21:57.570961+0000","last_fullsized":"2024-04-25T00:21:57.570961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555127+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:21:57.555127+0000","last_peered":"2024-04-25T00:21:57.555127+0000","last_clean":"2024-04-25T00:21:57.555127+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:21:57.555127+0000","last_undegraded":"2024-04-25T00:21:57.555127+0000","last_fullsized":"2024-04-25T00:21:57.555127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569250+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:21:57.569250+0000","last_peered":"2024-04-25T00:21:57.569250+0000","last_clean":"2024-04-25T00:21:57.569250+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:21:57.569250+0000","last_undegraded":"2024-04-25T00:21:57.569250+0000","last_fullsized":"2024-04-25T00:21:57.569250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471567+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:21:57.471567+0000","last_peered":"2024-04-25T00:21:57.471567+0000","last_clean":"2024-04-25T00:21:57.471567+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:21:57.471567+0000","last_undegraded":"2024-04-25T00:21:57.471567+0000","last_fullsized":"2024-04-25T00:21:57.471567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369493+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:21:57.369493+0000","last_peered":"2024-04-25T00:21:57.369493+0000","last_clean":"2024-04-25T00:21:57.369493+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:21:57.369493+0000","last_undegraded":"2024-04-25T00:21:57.369493+0000","last_fullsized":"2024-04-25T00:21:57.369493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368878+0000","last_change":"2024-04-25T00:20:27.153842+0000","last_active":"2024-04-25T00:21:57.368878+0000","last_peered":"2024-04-25T00:21:57.368878+0000","last_clean":"2024-04-25T00:21:57.368878+0000","last_became_active":"2024-04-25T00:20:27.152421+0000","last_became_peered":"2024-04-25T00:20:27.152421+0000","last_unstale":"2024-04-25T00:21:57.368878+0000","last_undegraded":"2024-04-25T00:21:57.368878+0000","last_fullsized":"2024-04-25T00:21:57.368878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:30.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":185,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471573+0000","last_change":"2024-04-25T00:20:27.056346+0000","last_active":"2024-04-25T00:21:57.471573+0000","last_peered":"2024-04-25T00:21:57.471573+0000","last_clean":"2024-04-25T00:21:57.471573+0000","last_became_active":"2024-04-25T00:20:27.046470+0000","last_became_peered":"2024-04-25T00:20:27.046470+0000","last_unstale":"2024-04-25T00:21:57.471573+0000","last_undegraded":"2024-04-25T00:21:57.471573+0000","last_fullsized":"2024-04-25T00:21:57.471573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:31.760140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367846+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:21:57.367846+0000","last_peered":"2024-04-25T00:21:57.367846+0000","last_clean":"2024-04-25T00:21:57.367846+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:21:57.367846+0000","last_undegraded":"2024-04-25T00:21:57.367846+0000","last_fullsized":"2024-04-25T00:21:57.367846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367941+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:21:57.367941+0000","last_peered":"2024-04-25T00:21:57.367941+0000","last_clean":"2024-04-25T00:21:57.367941+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:21:57.367941+0000","last_undegraded":"2024-04-25T00:21:57.367941+0000","last_fullsized":"2024-04-25T00:21:57.367941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570184+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:21:57.570184+0000","last_peered":"2024-04-25T00:21:57.570184+0000","last_clean":"2024-04-25T00:21:57.570184+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:21:57.570184+0000","last_undegraded":"2024-04-25T00:21:57.570184+0000","last_fullsized":"2024-04-25T00:21:57.570184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368450+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:21:57.368450+0000","last_peered":"2024-04-25T00:21:57.368450+0000","last_clean":"2024-04-25T00:21:57.368450+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:21:57.368450+0000","last_undegraded":"2024-04-25T00:21:57.368450+0000","last_fullsized":"2024-04-25T00:21:57.368450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569175+0000","last_change":"2024-04-25T00:20:27.173867+0000","last_active":"2024-04-25T00:21:57.569175+0000","last_peered":"2024-04-25T00:21:57.569175+0000","last_clean":"2024-04-25T00:21:57.569175+0000","last_became_active":"2024-04-25T00:20:27.154058+0000","last_became_peered":"2024-04-25T00:20:27.154058+0000","last_unstale":"2024-04-25T00:21:57.569175+0000","last_undegraded":"2024-04-25T00:21:57.569175+0000","last_fullsized":"2024-04-25T00:21:57.569175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:55.519210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555526+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:21:57.555526+0000","last_peered":"2024-04-25T00:21:57.555526+0000","last_clean":"2024-04-25T00:21:57.555526+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:21:57.555526+0000","last_undegraded":"2024-04-25T00:21:57.555526+0000","last_fullsized":"2024-04-25T00:21:57.555526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473816+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:21:57.473816+0000","last_peered":"2024-04-25T00:21:57.473816+0000","last_clean":"2024-04-25T00:21:57.473816+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:21:57.473816+0000","last_undegraded":"2024-04-25T00:21:57.473816+0000","last_fullsized":"2024-04-25T00:21:57.473816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479776+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:21:57.479776+0000","last_peered":"2024-04-25T00:21:57.479776+0000","last_clean":"2024-04-25T00:21:57.479776+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:21:57.479776+0000","last_undegraded":"2024-04-25T00:21:57.479776+0000","last_fullsized":"2024-04-25T00:21:57.479776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369325+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:21:57.369325+0000","last_peered":"2024-04-25T00:21:57.369325+0000","last_clean":"2024-04-25T00:21:57.369325+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:21:57.369325+0000","last_undegraded":"2024-04-25T00:21:57.369325+0000","last_fullsized":"2024-04-25T00:21:57.369325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006431+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:21:58.006431+0000","last_peered":"2024-04-25T00:21:58.006431+0000","last_clean":"2024-04-25T00:21:58.006431+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:21:58.006431+0000","last_undegraded":"2024-04-25T00:21:58.006431+0000","last_fullsized":"2024-04-25T00:21:58.006431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474170+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:21:57.474170+0000","last_peered":"2024-04-25T00:21:57.474170+0000","last_clean":"2024-04-25T00:21:57.474170+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:21:57.474170+0000","last_undegraded":"2024-04-25T00:21:57.474170+0000","last_fullsized":"2024-04-25T00:21:57.474170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369206+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:21:57.369206+0000","last_peered":"2024-04-25T00:21:57.369206+0000","last_clean":"2024-04-25T00:21:57.369206+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:21:57.369206+0000","last_undegraded":"2024-04-25T00:21:57.369206+0000","last_fullsized":"2024-04-25T00:21:57.369206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445520+0000","last_change":"2024-04-25T00:20:27.054883+0000","last_active":"2024-04-25T00:21:57.445520+0000","last_peered":"2024-04-25T00:21:57.445520+0000","last_clean":"2024-04-25T00:21:57.445520+0000","last_became_active":"2024-04-25T00:20:27.049595+0000","last_became_peered":"2024-04-25T00:20:27.049595+0000","last_unstale":"2024-04-25T00:21:57.445520+0000","last_undegraded":"2024-04-25T00:21:57.445520+0000","last_fullsized":"2024-04-25T00:21:57.445520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:05.358193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479893+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:21:57.479893+0000","last_peered":"2024-04-25T00:21:57.479893+0000","last_clean":"2024-04-25T00:21:57.479893+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:21:57.479893+0000","last_undegraded":"2024-04-25T00:21:57.479893+0000","last_fullsized":"2024-04-25T00:21:57.479893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569179+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:21:57.569179+0000","last_peered":"2024-04-25T00:21:57.569179+0000","last_clean":"2024-04-25T00:21:57.569179+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:21:57.569179+0000","last_undegraded":"2024-04-25T00:21:57.569179+0000","last_fullsized":"2024-04-25T00:21:57.569179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569684+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:21:57.569684+0000","last_peered":"2024-04-25T00:21:57.569684+0000","last_clean":"2024-04-25T00:21:57.569684+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:21:57.569684+0000","last_undegraded":"2024-04-25T00:21:57.569684+0000","last_fullsized":"2024-04-25T00:21:57.569684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367172+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:21:57.367172+0000","last_peered":"2024-04-25T00:21:57.367172+0000","last_clean":"2024-04-25T00:21:57.367172+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:21:57.367172+0000","last_undegraded":"2024-04-25T00:21:57.367172+0000","last_fullsized":"2024-04-25T00:21:57.367172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570117+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:21:57.570117+0000","last_peered":"2024-04-25T00:21:57.570117+0000","last_clean":"2024-04-25T00:21:57.570117+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:21:57.570117+0000","last_undegraded":"2024-04-25T00:21:57.570117+0000","last_fullsized":"2024-04-25T00:21:57.570117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479533+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:21:57.479533+0000","last_peered":"2024-04-25T00:21:57.479533+0000","last_clean":"2024-04-25T00:21:57.479533+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:21:57.479533+0000","last_undegraded":"2024-04-25T00:21:57.479533+0000","last_fullsized":"2024-04-25T00:21:57.479533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471265+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:21:57.471265+0000","last_peered":"2024-04-25T00:21:57.471265+0000","last_clean":"2024-04-25T00:21:57.471265+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:21:57.471265+0000","last_undegraded":"2024-04-25T00:21:57.471265+0000","last_fullsized":"2024-04-25T00:21:57.471265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367392+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:21:57.367392+0000","last_peered":"2024-04-25T00:21:57.367392+0000","last_clean":"2024-04-25T00:21:57.367392+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:21:57.367392+0000","last_undegraded":"2024-04-25T00:21:57.367392+0000","last_fullsized":"2024-04-25T00:21:57.367392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":274,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006037+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:21:58.006037+0000","last_peered":"2024-04-25T00:21:58.006037+0000","last_clean":"2024-04-25T00:21:58.006037+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:21:58.006037+0000","last_undegraded":"2024-04-25T00:21:58.006037+0000","last_fullsized":"2024-04-25T00:21:58.006037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556589+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:21:57.556589+0000","last_peered":"2024-04-25T00:21:57.556589+0000","last_clean":"2024-04-25T00:21:57.556589+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:21:57.556589+0000","last_undegraded":"2024-04-25T00:21:57.556589+0000","last_fullsized":"2024-04-25T00:21:57.556589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471195+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:21:57.471195+0000","last_peered":"2024-04-25T00:21:57.471195+0000","last_clean":"2024-04-25T00:21:57.471195+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:21:57.471195+0000","last_undegraded":"2024-04-25T00:21:57.471195+0000","last_fullsized":"2024-04-25T00:21:57.471195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569301+0000","last_change":"2024-04-25T00:20:27.176907+0000","last_active":"2024-04-25T00:21:57.569301+0000","last_peered":"2024-04-25T00:21:57.569301+0000","last_clean":"2024-04-25T00:21:57.569301+0000","last_became_active":"2024-04-25T00:20:27.153755+0000","last_became_peered":"2024-04-25T00:20:27.153755+0000","last_unstale":"2024-04-25T00:21:57.569301+0000","last_undegraded":"2024-04-25T00:21:57.569301+0000","last_fullsized":"2024-04-25T00:21:57.569301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:07.142383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570525+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:21:57.570525+0000","last_peered":"2024-04-25T00:21:57.570525+0000","last_clean":"2024-04-25T00:21:57.570525+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:21:57.570525+0000","last_undegraded":"2024-04-25T00:21:57.570525+0000","last_fullsized":"2024-04-25T00:21:57.570525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368645+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:21:57.368645+0000","last_peered":"2024-04-25T00:21:57.368645+0000","last_clean":"2024-04-25T00:21:57.368645+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:21:57.368645+0000","last_undegraded":"2024-04-25T00:21:57.368645+0000","last_fullsized":"2024-04-25T00:21:57.368645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479864+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:21:57.479864+0000","last_peered":"2024-04-25T00:21:57.479864+0000","last_clean":"2024-04-25T00:21:57.479864+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:21:57.479864+0000","last_undegraded":"2024-04-25T00:21:57.479864+0000","last_fullsized":"2024-04-25T00:21:57.479864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555798+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:21:57.555798+0000","last_peered":"2024-04-25T00:21:57.555798+0000","last_clean":"2024-04-25T00:21:57.555798+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:21:57.555798+0000","last_undegraded":"2024-04-25T00:21:57.555798+0000","last_fullsized":"2024-04-25T00:21:57.555798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471386+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:21:57.471386+0000","last_peered":"2024-04-25T00:21:57.471386+0000","last_clean":"2024-04-25T00:21:57.471386+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:21:57.471386+0000","last_undegraded":"2024-04-25T00:21:57.471386+0000","last_fullsized":"2024-04-25T00:21:57.471386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006697+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:21:58.006697+0000","last_peered":"2024-04-25T00:21:58.006697+0000","last_clean":"2024-04-25T00:21:58.006697+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:21:58.006697+0000","last_undegraded":"2024-04-25T00:21:58.006697+0000","last_fullsized":"2024-04-25T00:21:58.006697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474031+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:21:57.474031+0000","last_peered":"2024-04-25T00:21:57.474031+0000","last_clean":"2024-04-25T00:21:57.474031+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:21:57.474031+0000","last_undegraded":"2024-04-25T00:21:57.474031+0000","last_fullsized":"2024-04-25T00:21:57.474031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.480095+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:21:57.480095+0000","last_peered":"2024-04-25T00:21:57.480095+0000","last_clean":"2024-04-25T00:21:57.480095+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:21:57.480095+0000","last_undegraded":"2024-04-25T00:21:57.480095+0000","last_fullsized":"2024-04-25T00:21:57.480095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369252+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:21:57.369252+0000","last_peered":"2024-04-25T00:21:57.369252+0000","last_clean":"2024-04-25T00:21:57.369252+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:21:57.369252+0000","last_undegraded":"2024-04-25T00:21:57.369252+0000","last_fullsized":"2024-04-25T00:21:57.369252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570057+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:21:57.570057+0000","last_peered":"2024-04-25T00:21:57.570057+0000","last_clean":"2024-04-25T00:21:57.570057+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:21:57.570057+0000","last_undegraded":"2024-04-25T00:21:57.570057+0000","last_fullsized":"2024-04-25T00:21:57.570057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568987+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:21:57.568987+0000","last_peered":"2024-04-25T00:21:57.568987+0000","last_clean":"2024-04-25T00:21:57.568987+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:21:57.568987+0000","last_undegraded":"2024-04-25T00:21:57.568987+0000","last_fullsized":"2024-04-25T00:21:57.568987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479870+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:21:57.479870+0000","last_peered":"2024-04-25T00:21:57.479870+0000","last_clean":"2024-04-25T00:21:57.479870+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:21:57.479870+0000","last_undegraded":"2024-04-25T00:21:57.479870+0000","last_fullsized":"2024-04-25T00:21:57.479870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569260+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:21:57.569260+0000","last_peered":"2024-04-25T00:21:57.569260+0000","last_clean":"2024-04-25T00:21:57.569260+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:21:57.569260+0000","last_undegraded":"2024-04-25T00:21:57.569260+0000","last_fullsized":"2024-04-25T00:21:57.569260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474056+0000","last_change":"2024-04-25T00:20:27.153934+0000","last_active":"2024-04-25T00:21:57.474056+0000","last_peered":"2024-04-25T00:21:57.474056+0000","last_clean":"2024-04-25T00:21:57.474056+0000","last_became_active":"2024-04-25T00:20:27.152647+0000","last_became_peered":"2024-04-25T00:20:27.152647+0000","last_unstale":"2024-04-25T00:21:57.474056+0000","last_undegraded":"2024-04-25T00:21:57.474056+0000","last_fullsized":"2024-04-25T00:21:57.474056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.389722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.019845+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:21:58.019845+0000","last_peered":"2024-04-25T00:21:58.019845+0000","last_clean":"2024-04-25T00:21:58.019845+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:21:58.019845+0000","last_undegraded":"2024-04-25T00:21:58.019845+0000","last_fullsized":"2024-04-25T00:21:58.019845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569027+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:21:57.569027+0000","last_peered":"2024-04-25T00:21:57.569027+0000","last_clean":"2024-04-25T00:21:57.569027+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:21:57.569027+0000","last_undegraded":"2024-04-25T00:21:57.569027+0000","last_fullsized":"2024-04-25T00:21:57.569027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571825+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:21:57.571825+0000","last_peered":"2024-04-25T00:21:57.571825+0000","last_clean":"2024-04-25T00:21:57.571825+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:21:57.571825+0000","last_undegraded":"2024-04-25T00:21:57.571825+0000","last_fullsized":"2024-04-25T00:21:57.571825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473779+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:21:57.473779+0000","last_peered":"2024-04-25T00:21:57.473779+0000","last_clean":"2024-04-25T00:21:57.473779+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:21:57.473779+0000","last_undegraded":"2024-04-25T00:21:57.473779+0000","last_fullsized":"2024-04-25T00:21:57.473779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479596+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:21:57.479596+0000","last_peered":"2024-04-25T00:21:57.479596+0000","last_clean":"2024-04-25T00:21:57.479596+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:21:57.479596+0000","last_undegraded":"2024-04-25T00:21:57.479596+0000","last_fullsized":"2024-04-25T00:21:57.479596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479496+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:21:57.479496+0000","last_peered":"2024-04-25T00:21:57.479496+0000","last_clean":"2024-04-25T00:21:57.479496+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:21:57.479496+0000","last_undegraded":"2024-04-25T00:21:57.479496+0000","last_fullsized":"2024-04-25T00:21:57.479496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474293+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:21:57.474293+0000","last_peered":"2024-04-25T00:21:57.474293+0000","last_clean":"2024-04-25T00:21:57.474293+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:21:57.474293+0000","last_undegraded":"2024-04-25T00:21:57.474293+0000","last_fullsized":"2024-04-25T00:21:57.474293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569223+0000","last_change":"2024-04-25T00:21:27.923915+0000","last_active":"2024-04-25T00:21:57.569223+0000","last_peered":"2024-04-25T00:21:57.569223+0000","last_clean":"2024-04-25T00:21:57.569223+0000","last_became_active":"2024-04-25T00:21:27.923465+0000","last_became_peered":"2024-04-25T00:21:27.923465+0000","last_unstale":"2024-04-25T00:21:57.569223+0000","last_undegraded":"2024-04-25T00:21:57.569223+0000","last_fullsized":"2024-04-25T00:21:57.569223+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.804110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473722+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:21:57.473722+0000","last_peered":"2024-04-25T00:21:57.473722+0000","last_clean":"2024-04-25T00:21:57.473722+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:21:57.473722+0000","last_undegraded":"2024-04-25T00:21:57.473722+0000","last_fullsized":"2024-04-25T00:21:57.473722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471729+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:21:57.471729+0000","last_peered":"2024-04-25T00:21:57.471729+0000","last_clean":"2024-04-25T00:21:57.471729+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:21:57.471729+0000","last_undegraded":"2024-04-25T00:21:57.471729+0000","last_fullsized":"2024-04-25T00:21:57.471729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445825+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:21:57.445825+0000","last_peered":"2024-04-25T00:21:57.445825+0000","last_clean":"2024-04-25T00:21:57.445825+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:21:57.445825+0000","last_undegraded":"2024-04-25T00:21:57.445825+0000","last_fullsized":"2024-04-25T00:21:57.445825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571137+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:21:57.571137+0000","last_peered":"2024-04-25T00:21:57.571137+0000","last_clean":"2024-04-25T00:21:57.571137+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:21:57.571137+0000","last_undegraded":"2024-04-25T00:21:57.571137+0000","last_fullsized":"2024-04-25T00:21:57.571137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569036+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:21:57.569036+0000","last_peered":"2024-04-25T00:21:57.569036+0000","last_clean":"2024-04-25T00:21:57.569036+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:21:57.569036+0000","last_undegraded":"2024-04-25T00:21:57.569036+0000","last_fullsized":"2024-04-25T00:21:57.569036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445627+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:21:57.445627+0000","last_peered":"2024-04-25T00:21:57.445627+0000","last_clean":"2024-04-25T00:21:57.445627+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:21:57.445627+0000","last_undegraded":"2024-04-25T00:21:57.445627+0000","last_fullsized":"2024-04-25T00:21:57.445627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571805+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:21:57.571805+0000","last_peered":"2024-04-25T00:21:57.571805+0000","last_clean":"2024-04-25T00:21:57.571805+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:21:57.571805+0000","last_undegraded":"2024-04-25T00:21:57.571805+0000","last_fullsized":"2024-04-25T00:21:57.571805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479762+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:21:57.479762+0000","last_peered":"2024-04-25T00:21:57.479762+0000","last_clean":"2024-04-25T00:21:57.479762+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:21:57.479762+0000","last_undegraded":"2024-04-25T00:21:57.479762+0000","last_fullsized":"2024-04-25T00:21:57.479762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.707830+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:21:57.707830+0000","last_peered":"2024-04-25T00:21:57.707830+0000","last_clean":"2024-04-25T00:21:57.707830+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:21:57.707830+0000","last_undegraded":"2024-04-25T00:21:57.707830+0000","last_fullsized":"2024-04-25T00:21:57.707830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367200+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:21:57.367200+0000","last_peered":"2024-04-25T00:21:57.367200+0000","last_clean":"2024-04-25T00:21:57.367200+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:21:57.367200+0000","last_undegraded":"2024-04-25T00:21:57.367200+0000","last_fullsized":"2024-04-25T00:21:57.367200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368588+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:21:57.368588+0000","last_peered":"2024-04-25T00:21:57.368588+0000","last_clean":"2024-04-25T00:21:57.368588+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:21:57.368588+0000","last_undegraded":"2024-04-25T00:21:57.368588+0000","last_fullsized":"2024-04-25T00:21:57.368588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568856+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:21:57.568856+0000","last_peered":"2024-04-25T00:21:57.568856+0000","last_clean":"2024-04-25T00:21:57.568856+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:21:57.568856+0000","last_undegraded":"2024-04-25T00:21:57.568856+0000","last_fullsized":"2024-04-25T00:21:57.568856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474047+0000","last_change":"2024-04-25T00:20:27.049784+0000","last_active":"2024-04-25T00:21:57.474047+0000","last_peered":"2024-04-25T00:21:57.474047+0000","last_clean":"2024-04-25T00:21:57.474047+0000","last_became_active":"2024-04-25T00:20:27.049413+0000","last_became_peered":"2024-04-25T00:20:27.049413+0000","last_unstale":"2024-04-25T00:21:57.474047+0000","last_undegraded":"2024-04-25T00:21:57.474047+0000","last_fullsized":"2024-04-25T00:21:57.474047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:26.533987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569353+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:21:57.569353+0000","last_peered":"2024-04-25T00:21:57.569353+0000","last_clean":"2024-04-25T00:21:57.569353+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:21:57.569353+0000","last_undegraded":"2024-04-25T00:21:57.569353+0000","last_fullsized":"2024-04-25T00:21:57.569353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568945+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:21:57.568945+0000","last_peered":"2024-04-25T00:21:57.568945+0000","last_clean":"2024-04-25T00:21:57.568945+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:21:57.568945+0000","last_undegraded":"2024-04-25T00:21:57.568945+0000","last_fullsized":"2024-04-25T00:21:57.568945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474064+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:21:57.474064+0000","last_peered":"2024-04-25T00:21:57.474064+0000","last_clean":"2024-04-25T00:21:57.474064+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:21:57.474064+0000","last_undegraded":"2024-04-25T00:21:57.474064+0000","last_fullsized":"2024-04-25T00:21:57.474064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571331+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:21:57.571331+0000","last_peered":"2024-04-25T00:21:57.571331+0000","last_clean":"2024-04-25T00:21:57.571331+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:21:57.571331+0000","last_undegraded":"2024-04-25T00:21:57.571331+0000","last_fullsized":"2024-04-25T00:21:57.571331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367171+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:21:57.367171+0000","last_peered":"2024-04-25T00:21:57.367171+0000","last_clean":"2024-04-25T00:21:57.367171+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:21:57.367171+0000","last_undegraded":"2024-04-25T00:21:57.367171+0000","last_fullsized":"2024-04-25T00:21:57.367171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.598104+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:21:57.598104+0000","last_peered":"2024-04-25T00:21:57.598104+0000","last_clean":"2024-04-25T00:21:57.598104+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:21:57.598104+0000","last_undegraded":"2024-04-25T00:21:57.598104+0000","last_fullsized":"2024-04-25T00:21:57.598104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368050+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:21:57.368050+0000","last_peered":"2024-04-25T00:21:57.368050+0000","last_clean":"2024-04-25T00:21:57.368050+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:21:57.368050+0000","last_undegraded":"2024-04-25T00:21:57.368050+0000","last_fullsized":"2024-04-25T00:21:57.368050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368768+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:21:57.368768+0000","last_peered":"2024-04-25T00:21:57.368768+0000","last_clean":"2024-04-25T00:21:57.368768+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:21:57.368768+0000","last_undegraded":"2024-04-25T00:21:57.368768+0000","last_fullsized":"2024-04-25T00:21:57.368768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368990+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:21:57.368990+0000","last_peered":"2024-04-25T00:21:57.368990+0000","last_clean":"2024-04-25T00:21:57.368990+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:21:57.368990+0000","last_undegraded":"2024-04-25T00:21:57.368990+0000","last_fullsized":"2024-04-25T00:21:57.368990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570368+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:21:57.570368+0000","last_peered":"2024-04-25T00:21:57.570368+0000","last_clean":"2024-04-25T00:21:57.570368+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:21:57.570368+0000","last_undegraded":"2024-04-25T00:21:57.570368+0000","last_fullsized":"2024-04-25T00:21:57.570368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368871+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:21:57.368871+0000","last_peered":"2024-04-25T00:21:57.368871+0000","last_clean":"2024-04-25T00:21:57.368871+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:21:57.368871+0000","last_undegraded":"2024-04-25T00:21:57.368871+0000","last_fullsized":"2024-04-25T00:21:57.368871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555187+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:21:57.555187+0000","last_peered":"2024-04-25T00:21:57.555187+0000","last_clean":"2024-04-25T00:21:57.555187+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:21:57.555187+0000","last_undegraded":"2024-04-25T00:21:57.555187+0000","last_fullsized":"2024-04-25T00:21:57.555187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473693+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:21:57.473693+0000","last_peered":"2024-04-25T00:21:57.473693+0000","last_clean":"2024-04-25T00:21:57.473693+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:21:57.473693+0000","last_undegraded":"2024-04-25T00:21:57.473693+0000","last_fullsized":"2024-04-25T00:21:57.473693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474191+0000","last_change":"2024-04-25T00:20:27.153703+0000","last_active":"2024-04-25T00:21:57.474191+0000","last_peered":"2024-04-25T00:21:57.474191+0000","last_clean":"2024-04-25T00:21:57.474191+0000","last_became_active":"2024-04-25T00:20:27.152715+0000","last_became_peered":"2024-04-25T00:20:27.152715+0000","last_unstale":"2024-04-25T00:21:57.474191+0000","last_undegraded":"2024-04-25T00:21:57.474191+0000","last_fullsized":"2024-04-25T00:21:57.474191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:09.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367131+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:21:57.367131+0000","last_peered":"2024-04-25T00:21:57.367131+0000","last_clean":"2024-04-25T00:21:57.367131+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:21:57.367131+0000","last_undegraded":"2024-04-25T00:21:57.367131+0000","last_fullsized":"2024-04-25T00:21:57.367131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568847+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:21:57.568847+0000","last_peered":"2024-04-25T00:21:57.568847+0000","last_clean":"2024-04-25T00:21:57.568847+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:21:57.568847+0000","last_undegraded":"2024-04-25T00:21:57.568847+0000","last_fullsized":"2024-04-25T00:21:57.568847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368683+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:21:57.368683+0000","last_peered":"2024-04-25T00:21:57.368683+0000","last_clean":"2024-04-25T00:21:57.368683+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:21:57.368683+0000","last_undegraded":"2024-04-25T00:21:57.368683+0000","last_fullsized":"2024-04-25T00:21:57.368683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369170+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:21:57.369170+0000","last_peered":"2024-04-25T00:21:57.369170+0000","last_clean":"2024-04-25T00:21:57.369170+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:21:57.369170+0000","last_undegraded":"2024-04-25T00:21:57.369170+0000","last_fullsized":"2024-04-25T00:21:57.369170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367712+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:21:57.367712+0000","last_peered":"2024-04-25T00:21:57.367712+0000","last_clean":"2024-04-25T00:21:57.367712+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:21:57.367712+0000","last_undegraded":"2024-04-25T00:21:57.367712+0000","last_fullsized":"2024-04-25T00:21:57.367712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368859+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:21:57.368859+0000","last_peered":"2024-04-25T00:21:57.368859+0000","last_clean":"2024-04-25T00:21:57.368859+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:21:57.368859+0000","last_undegraded":"2024-04-25T00:21:57.368859+0000","last_fullsized":"2024-04-25T00:21:57.368859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473798+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:21:57.473798+0000","last_peered":"2024-04-25T00:21:57.473798+0000","last_clean":"2024-04-25T00:21:57.473798+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:21:57.473798+0000","last_undegraded":"2024-04-25T00:21:57.473798+0000","last_fullsized":"2024-04-25T00:21:57.473798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555469+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:21:57.555469+0000","last_peered":"2024-04-25T00:21:57.555469+0000","last_clean":"2024-04-25T00:21:57.555469+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:21:57.555469+0000","last_undegraded":"2024-04-25T00:21:57.555469+0000","last_fullsized":"2024-04-25T00:21:57.555469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369406+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:21:57.369406+0000","last_peered":"2024-04-25T00:21:57.369406+0000","last_clean":"2024-04-25T00:21:57.369406+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:21:57.369406+0000","last_undegraded":"2024-04-25T00:21:57.369406+0000","last_fullsized":"2024-04-25T00:21:57.369406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569340+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:21:57.569340+0000","last_peered":"2024-04-25T00:21:57.569340+0000","last_clean":"2024-04-25T00:21:57.569340+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:21:57.569340+0000","last_undegraded":"2024-04-25T00:21:57.569340+0000","last_fullsized":"2024-04-25T00:21:57.569340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570576+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:21:57.570576+0000","last_peered":"2024-04-25T00:21:57.570576+0000","last_clean":"2024-04-25T00:21:57.570576+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:21:57.570576+0000","last_undegraded":"2024-04-25T00:21:57.570576+0000","last_fullsized":"2024-04-25T00:21:57.570576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.599346+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:21:57.599346+0000","last_peered":"2024-04-25T00:21:57.599346+0000","last_clean":"2024-04-25T00:21:57.599346+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:21:57.599346+0000","last_undegraded":"2024-04-25T00:21:57.599346+0000","last_fullsized":"2024-04-25T00:21:57.599346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569431+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:21:57.569431+0000","last_peered":"2024-04-25T00:21:57.569431+0000","last_clean":"2024-04-25T00:21:57.569431+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:21:57.569431+0000","last_undegraded":"2024-04-25T00:21:57.569431+0000","last_fullsized":"2024-04-25T00:21:57.569431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479319+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:21:57.479319+0000","last_peered":"2024-04-25T00:21:57.479319+0000","last_clean":"2024-04-25T00:21:57.479319+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:21:57.479319+0000","last_undegraded":"2024-04-25T00:21:57.479319+0000","last_fullsized":"2024-04-25T00:21:57.479319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571948+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:21:57.571948+0000","last_peered":"2024-04-25T00:21:57.571948+0000","last_clean":"2024-04-25T00:21:57.571948+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:21:57.571948+0000","last_undegraded":"2024-04-25T00:21:57.571948+0000","last_fullsized":"2024-04-25T00:21:57.571948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569558+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:21:57.569558+0000","last_peered":"2024-04-25T00:21:57.569558+0000","last_clean":"2024-04-25T00:21:57.569558+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:21:57.569558+0000","last_undegraded":"2024-04-25T00:21:57.569558+0000","last_fullsized":"2024-04-25T00:21:57.569558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568933+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:21:57.568933+0000","last_peered":"2024-04-25T00:21:57.568933+0000","last_clean":"2024-04-25T00:21:57.568933+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:21:57.568933+0000","last_undegraded":"2024-04-25T00:21:57.568933+0000","last_fullsized":"2024-04-25T00:21:57.568933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369093+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:21:57.369093+0000","last_peered":"2024-04-25T00:21:57.369093+0000","last_clean":"2024-04-25T00:21:57.369093+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:21:57.369093+0000","last_undegraded":"2024-04-25T00:21:57.369093+0000","last_fullsized":"2024-04-25T00:21:57.369093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368834+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:21:57.368834+0000","last_peered":"2024-04-25T00:21:57.368834+0000","last_clean":"2024-04-25T00:21:57.368834+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:21:57.368834+0000","last_undegraded":"2024-04-25T00:21:57.368834+0000","last_fullsized":"2024-04-25T00:21:57.368834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570810+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:21:57.570810+0000","last_peered":"2024-04-25T00:21:57.570810+0000","last_clean":"2024-04-25T00:21:57.570810+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:21:57.570810+0000","last_undegraded":"2024-04-25T00:21:57.570810+0000","last_fullsized":"2024-04-25T00:21:57.570810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571459+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:21:57.571459+0000","last_peered":"2024-04-25T00:21:57.571459+0000","last_clean":"2024-04-25T00:21:57.571459+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:21:57.571459+0000","last_undegraded":"2024-04-25T00:21:57.571459+0000","last_fullsized":"2024-04-25T00:21:57.571459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569203+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:21:57.569203+0000","last_peered":"2024-04-25T00:21:57.569203+0000","last_clean":"2024-04-25T00:21:57.569203+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:21:57.569203+0000","last_undegraded":"2024-04-25T00:21:57.569203+0000","last_fullsized":"2024-04-25T00:21:57.569203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369599+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:21:57.369599+0000","last_peered":"2024-04-25T00:21:57.369599+0000","last_clean":"2024-04-25T00:21:57.369599+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:21:57.369599+0000","last_undegraded":"2024-04-25T00:21:57.369599+0000","last_fullsized":"2024-04-25T00:21:57.369599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368190+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:21:57.368190+0000","last_peered":"2024-04-25T00:21:57.368190+0000","last_clean":"2024-04-25T00:21:57.368190+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:21:57.368190+0000","last_undegraded":"2024-04-25T00:21:57.368190+0000","last_fullsized":"2024-04-25T00:21:57.368190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479506+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:21:57.479506+0000","last_peered":"2024-04-25T00:21:57.479506+0000","last_clean":"2024-04-25T00:21:57.479506+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:21:57.479506+0000","last_undegraded":"2024-04-25T00:21:57.479506+0000","last_fullsized":"2024-04-25T00:21:57.479506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367262+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:21:57.367262+0000","last_peered":"2024-04-25T00:21:57.367262+0000","last_clean":"2024-04-25T00:21:57.367262+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:21:57.367262+0000","last_undegraded":"2024-04-25T00:21:57.367262+0000","last_fullsized":"2024-04-25T00:21:57.367262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555764+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:21:57.555764+0000","last_peered":"2024-04-25T00:21:57.555764+0000","last_clean":"2024-04-25T00:21:57.555764+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:21:57.555764+0000","last_undegraded":"2024-04-25T00:21:57.555764+0000","last_fullsized":"2024-04-25T00:21:57.555764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570598+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:21:57.570598+0000","last_peered":"2024-04-25T00:21:57.570598+0000","last_clean":"2024-04-25T00:21:57.570598+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:21:57.570598+0000","last_undegraded":"2024-04-25T00:21:57.570598+0000","last_fullsized":"2024-04-25T00:21:57.570598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569041+0000","last_change":"2024-04-25T00:20:27.046816+0000","last_active":"2024-04-25T00:21:57.569041+0000","last_peered":"2024-04-25T00:21:57.569041+0000","last_clean":"2024-04-25T00:21:57.569041+0000","last_became_active":"2024-04-25T00:20:27.045845+0000","last_became_peered":"2024-04-25T00:20:27.045845+0000","last_unstale":"2024-04-25T00:21:57.569041+0000","last_undegraded":"2024-04-25T00:21:57.569041+0000","last_fullsized":"2024-04-25T00:21:57.569041+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:45.465308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108492,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599296,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1299,"num_read_kb":761,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1797888,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":530611,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28200,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716952,"statfs":{"total":95995035648,"available":95966158848,"internally_reserved":0,"allocated":1212416,"data_stored":1315182,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":927181,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":923463,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":1127835,"data_compressed":65691,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":525373,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":592307,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":526829,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":593758,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1049897,"data_compressed":54691,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":530370,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1254766,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":200163,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:22:00.257 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T00:22:00.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- 172.21.15.45:0/1148771057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 msgr2=0x7f74ec100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1148771057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec100aa0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009900 tx=0x7f74e002a410 comp rx=0 tx=0).stop 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- 172.21.15.45:0/1148771057 shutdown_connections 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1148771057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 0x7f74ec108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1148771057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1148771057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec100aa0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- 172.21.15.45:0/1148771057 >> 172.21.15.45:0/1148771057 conn(0x7f74ec0fbbf0 msgr2=0x7f74ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- 172.21.15.45:0/1148771057 shutdown_connections 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- 172.21.15.45:0/1148771057 wait complete. 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 Processor -- start 2024-04-25T00:22:00.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.399+0000 7f74f2e53640 1 -- start start 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 0x7f74ec19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10c300 con 0x7f74ec100680 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10c180 con 0x7f74ec102230 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f2e53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10c480 con 0x7f74ec101870 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f0bef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f0bef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52820/0 (socket says 172.21.15.45:52820) 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f0bef640 1 -- 172.21.15.45:0/1703193752 learned_addr learned my addr 172.21.15.45:0/1703193752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f13f0640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 0x7f74ec19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 -- 172.21.15.45:0/1703193752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 msgr2=0x7f74ec19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 0x7f74ec19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 -- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 msgr2=0x7f74ec197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 -- 172.21.15.45:0/1703193752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec19f720 con 0x7f74ec101870 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74f0bef640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.400+0000 7f74ebfff640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec198040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.401+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d800ccc0 con 0x7f74ec101870 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.401+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d8005020 con 0x7f74ec101870 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.401+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec19fa10 con 0x7f74ec101870 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.401+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004a00 con 0x7f74ec101870 2024-04-25T00:22:00.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.401+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec19ff60 con 0x7f74ec101870 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.402+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ec1a01f0 con 0x7f74ec101870 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.403+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f74d8004ba0 con 0x7f74ec101870 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.404+0000 7f74e9ffb640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 0x7f74dc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.404+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f74d80cf6d0 con 0x7f74ec101870 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.404+0000 7f74f0bef640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 0x7f74dc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.404+0000 7f74f0bef640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 0x7f74dc0a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00046c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.408+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f74d809bf90 con 0x7f74ec101870 2024-04-25T00:22:00.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.614+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74ec1a03c0 con 0x7f74dc0a2d80 2024-04-25T00:22:00.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.633+0000 7f74e9ffb640 1 -- 172.21.15.45:0/1703193752 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339719 (secure 0 0 0) 0x7f74ec1a03c0 con 0x7f74dc0a2d80 2024-04-25T00:22:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:00.639 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 msgr2=0x7f74dc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 0x7f74dc0a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00046c0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 msgr2=0x7f74ec198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec198040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800edf0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 shutdown_connections 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f74dc0a2d80 0x7f74dc0a5240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74ec102230 0x7f74ec19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74ec101870 0x7f74ec198040 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 --2- 172.21.15.45:0/1703193752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74ec100680 0x7f74ec197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 >> 172.21.15.45:0/1703193752 conn(0x7f74ec0fbbf0 msgr2=0x7f74ec0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:00.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 shutdown_connections 2024-04-25T00:22:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.637+0000 7f74f2e53640 1 -- 172.21.15.45:0/1703193752 wait complete. 2024-04-25T00:22:00.649 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-25T00:21:59.380607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189898,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1435,"num_read_kb":880,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1181408,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":940,"ondisk_log_size":940,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391176,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550648,"statfs":{"total":1151940427776,"available":1151539863552,"internally_reserved":0,"allocated":13180928,"data_stored":9897572,"data_compressed":538584,"data_compressed_allocated":724992,"data_compressed_original":4300800,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.985744"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.880965+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.880449+0000","last_became_peered":"2024-04-25T00:19:33.880449+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.380053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570237+0000","last_change":"2024-04-25T00:19:33.886049+0000","last_active":"2024-04-25T00:21:57.570237+0000","last_peered":"2024-04-25T00:21:57.570237+0000","last_clean":"2024-04-25T00:21:57.570237+0000","last_became_active":"2024-04-25T00:19:33.885247+0000","last_became_peered":"2024-04-25T00:19:33.885247+0000","last_unstale":"2024-04-25T00:21:57.570237+0000","last_undegraded":"2024-04-25T00:21:57.570237+0000","last_fullsized":"2024-04-25T00:21:57.570237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:54:29.324260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569393+0000","last_change":"2024-04-25T00:19:33.892642+0000","last_active":"2024-04-25T00:21:57.569393+0000","last_peered":"2024-04-25T00:21:57.569393+0000","last_clean":"2024-04-25T00:21:57.569393+0000","last_became_active":"2024-04-25T00:19:33.891168+0000","last_became_peered":"2024-04-25T00:19:33.891168+0000","last_unstale":"2024-04-25T00:21:57.569393+0000","last_undegraded":"2024-04-25T00:21:57.569393+0000","last_fullsized":"2024-04-25T00:21:57.569393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:42.256461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471340+0000","last_change":"2024-04-25T00:21:27.923300+0000","last_active":"2024-04-25T00:21:57.471340+0000","last_peered":"2024-04-25T00:21:57.471340+0000","last_clean":"2024-04-25T00:21:57.471340+0000","last_became_active":"2024-04-25T00:21:27.922951+0000","last_became_peered":"2024-04-25T00:21:27.922951+0000","last_unstale":"2024-04-25T00:21:57.471340+0000","last_undegraded":"2024-04-25T00:21:57.471340+0000","last_fullsized":"2024-04-25T00:21:57.471340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:50.728710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473760+0000","last_change":"2024-04-25T00:19:33.882322+0000","last_active":"2024-04-25T00:21:57.473760+0000","last_peered":"2024-04-25T00:21:57.473760+0000","last_clean":"2024-04-25T00:21:57.473760+0000","last_became_active":"2024-04-25T00:19:33.881958+0000","last_became_peered":"2024-04-25T00:19:33.881958+0000","last_unstale":"2024-04-25T00:21:57.473760+0000","last_undegraded":"2024-04-25T00:21:57.473760+0000","last_fullsized":"2024-04-25T00:21:57.473760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:08.925716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445544+0000","last_change":"2024-04-25T00:19:33.880625+0000","last_active":"2024-04-25T00:21:57.445544+0000","last_peered":"2024-04-25T00:21:57.445544+0000","last_clean":"2024-04-25T00:21:57.445544+0000","last_became_active":"2024-04-25T00:19:33.880237+0000","last_became_peered":"2024-04-25T00:19:33.880237+0000","last_unstale":"2024-04-25T00:21:57.445544+0000","last_undegraded":"2024-04-25T00:21:57.445544+0000","last_fullsized":"2024-04-25T00:21:57.445544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:29.356493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:33.880335+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:33.874686+0000","last_became_peered":"2024-04-25T00:19:33.874686+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:03.433091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367303+0000","last_change":"2024-04-25T00:19:33.889965+0000","last_active":"2024-04-25T00:21:57.367303+0000","last_peered":"2024-04-25T00:21:57.367303+0000","last_clean":"2024-04-25T00:21:57.367303+0000","last_became_active":"2024-04-25T00:19:33.889576+0000","last_became_peered":"2024-04-25T00:19:33.889576+0000","last_unstale":"2024-04-25T00:21:57.367303+0000","last_undegraded":"2024-04-25T00:21:57.367303+0000","last_fullsized":"2024-04-25T00:21:57.367303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.887113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556792+0000","last_change":"2024-04-25T00:19:33.877243+0000","last_active":"2024-04-25T00:21:57.556792+0000","last_peered":"2024-04-25T00:21:57.556792+0000","last_clean":"2024-04-25T00:21:57.556792+0000","last_became_active":"2024-04-25T00:19:33.876891+0000","last_became_peered":"2024-04-25T00:19:33.876891+0000","last_unstale":"2024-04-25T00:21:57.556792+0000","last_undegraded":"2024-04-25T00:21:57.556792+0000","last_fullsized":"2024-04-25T00:21:57.556792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:00:32.215434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445954+0000","last_change":"2024-04-25T00:19:33.879402+0000","last_active":"2024-04-25T00:21:57.445954+0000","last_peered":"2024-04-25T00:21:57.445954+0000","last_clean":"2024-04-25T00:21:57.445954+0000","last_became_active":"2024-04-25T00:19:33.879057+0000","last_became_peered":"2024-04-25T00:19:33.879057+0000","last_unstale":"2024-04-25T00:21:57.445954+0000","last_undegraded":"2024-04-25T00:21:57.445954+0000","last_fullsized":"2024-04-25T00:21:57.445954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:23.283323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569360+0000","last_change":"2024-04-25T00:19:33.893844+0000","last_active":"2024-04-25T00:21:57.569360+0000","last_peered":"2024-04-25T00:21:57.569360+0000","last_clean":"2024-04-25T00:21:57.569360+0000","last_became_active":"2024-04-25T00:19:33.893509+0000","last_became_peered":"2024-04-25T00:19:33.893509+0000","last_unstale":"2024-04-25T00:21:57.569360+0000","last_undegraded":"2024-04-25T00:21:57.569360+0000","last_fullsized":"2024-04-25T00:21:57.569360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.190079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368761+0000","last_change":"2024-04-25T00:19:33.880244+0000","last_active":"2024-04-25T00:21:57.368761+0000","last_peered":"2024-04-25T00:21:57.368761+0000","last_clean":"2024-04-25T00:21:57.368761+0000","last_became_active":"2024-04-25T00:19:33.879368+0000","last_became_peered":"2024-04-25T00:19:33.879368+0000","last_unstale":"2024-04-25T00:21:57.368761+0000","last_undegraded":"2024-04-25T00:21:57.368761+0000","last_fullsized":"2024-04-25T00:21:57.368761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:41.187248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570829+0000","last_change":"2024-04-25T00:19:33.878305+0000","last_active":"2024-04-25T00:21:57.570829+0000","last_peered":"2024-04-25T00:21:57.570829+0000","last_clean":"2024-04-25T00:21:57.570829+0000","last_became_active":"2024-04-25T00:19:33.877800+0000","last_became_peered":"2024-04-25T00:19:33.877800+0000","last_unstale":"2024-04-25T00:21:57.570829+0000","last_undegraded":"2024-04-25T00:21:57.570829+0000","last_fullsized":"2024-04-25T00:21:57.570829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:50.558253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571324+0000","last_change":"2024-04-25T00:19:33.881238+0000","last_active":"2024-04-25T00:21:57.571324+0000","last_peered":"2024-04-25T00:21:57.571324+0000","last_clean":"2024-04-25T00:21:57.571324+0000","last_became_active":"2024-04-25T00:19:33.880835+0000","last_became_peered":"2024-04-25T00:19:33.880835+0000","last_unstale":"2024-04-25T00:21:57.571324+0000","last_undegraded":"2024-04-25T00:21:57.571324+0000","last_fullsized":"2024-04-25T00:21:57.571324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:04.006417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570961+0000","last_change":"2024-04-25T00:19:33.883456+0000","last_active":"2024-04-25T00:21:57.570961+0000","last_peered":"2024-04-25T00:21:57.570961+0000","last_clean":"2024-04-25T00:21:57.570961+0000","last_became_active":"2024-04-25T00:19:33.883037+0000","last_became_peered":"2024-04-25T00:19:33.883037+0000","last_unstale":"2024-04-25T00:21:57.570961+0000","last_undegraded":"2024-04-25T00:21:57.570961+0000","last_fullsized":"2024-04-25T00:21:57.570961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:59.219372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555127+0000","last_change":"2024-04-25T00:19:33.876251+0000","last_active":"2024-04-25T00:21:57.555127+0000","last_peered":"2024-04-25T00:21:57.555127+0000","last_clean":"2024-04-25T00:21:57.555127+0000","last_became_active":"2024-04-25T00:19:33.875933+0000","last_became_peered":"2024-04-25T00:19:33.875933+0000","last_unstale":"2024-04-25T00:21:57.555127+0000","last_undegraded":"2024-04-25T00:21:57.555127+0000","last_fullsized":"2024-04-25T00:21:57.555127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:35.120086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569250+0000","last_change":"2024-04-25T00:19:33.881598+0000","last_active":"2024-04-25T00:21:57.569250+0000","last_peered":"2024-04-25T00:21:57.569250+0000","last_clean":"2024-04-25T00:21:57.569250+0000","last_became_active":"2024-04-25T00:19:33.881235+0000","last_became_peered":"2024-04-25T00:19:33.881235+0000","last_unstale":"2024-04-25T00:21:57.569250+0000","last_undegraded":"2024-04-25T00:21:57.569250+0000","last_fullsized":"2024-04-25T00:21:57.569250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:58.948905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471567+0000","last_change":"2024-04-25T00:19:33.877029+0000","last_active":"2024-04-25T00:21:57.471567+0000","last_peered":"2024-04-25T00:21:57.471567+0000","last_clean":"2024-04-25T00:21:57.471567+0000","last_became_active":"2024-04-25T00:19:33.876663+0000","last_became_peered":"2024-04-25T00:19:33.876663+0000","last_unstale":"2024-04-25T00:21:57.471567+0000","last_undegraded":"2024-04-25T00:21:57.471567+0000","last_fullsized":"2024-04-25T00:21:57.471567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:10.459253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369493+0000","last_change":"2024-04-25T00:19:33.883203+0000","last_active":"2024-04-25T00:21:57.369493+0000","last_peered":"2024-04-25T00:21:57.369493+0000","last_clean":"2024-04-25T00:21:57.369493+0000","last_became_active":"2024-04-25T00:19:33.882827+0000","last_became_peered":"2024-04-25T00:19:33.882827+0000","last_unstale":"2024-04-25T00:21:57.369493+0000","last_undegraded":"2024-04-25T00:21:57.369493+0000","last_fullsized":"2024-04-25T00:21:57.369493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:32.236187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368878+0000","last_change":"2024-04-25T00:20:27.153842+0000","last_active":"2024-04-25T00:21:57.368878+0000","last_peered":"2024-04-25T00:21:57.368878+0000","last_clean":"2024-04-25T00:21:57.368878+0000","last_became_active":"2024-04-25T00:20:27.152421+0000","last_became_peered":"2024-04-25T00:20:27.152421+0000","last_unstale":"2024-04-25T00:21:57.368878+0000","last_undegraded":"2024-04-25T00:21:57.368878+0000","last_fullsized":"2024-04-25T00:21:57.368878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:30.036670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,10,0],"acting":[2,3,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":185,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471573+0000","last_change":"2024-04-25T00:20:27.056346+0000","last_active":"2024-04-25T00:21:57.471573+0000","last_peered":"2024-04-25T00:21:57.471573+0000","last_clean":"2024-04-25T00:21:57.471573+0000","last_became_active":"2024-04-25T00:20:27.046470+0000","last_became_peered":"2024-04-25T00:20:27.046470+0000","last_unstale":"2024-04-25T00:21:57.471573+0000","last_undegraded":"2024-04-25T00:21:57.471573+0000","last_fullsized":"2024-04-25T00:21:57.471573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:31.760140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367846+0000","last_change":"2024-04-25T00:19:33.890708+0000","last_active":"2024-04-25T00:21:57.367846+0000","last_peered":"2024-04-25T00:21:57.367846+0000","last_clean":"2024-04-25T00:21:57.367846+0000","last_became_active":"2024-04-25T00:19:33.888981+0000","last_became_peered":"2024-04-25T00:19:33.888981+0000","last_unstale":"2024-04-25T00:21:57.367846+0000","last_undegraded":"2024-04-25T00:21:57.367846+0000","last_fullsized":"2024-04-25T00:21:57.367846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:29.118969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.917566+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.917254+0000","last_became_peered":"2024-04-25T00:19:37.917254+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:13.999423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367941+0000","last_change":"2024-04-25T00:19:32.877568+0000","last_active":"2024-04-25T00:21:57.367941+0000","last_peered":"2024-04-25T00:21:57.367941+0000","last_clean":"2024-04-25T00:21:57.367941+0000","last_became_active":"2024-04-25T00:19:32.877203+0000","last_became_peered":"2024-04-25T00:19:32.877203+0000","last_unstale":"2024-04-25T00:21:57.367941+0000","last_undegraded":"2024-04-25T00:21:57.367941+0000","last_fullsized":"2024-04-25T00:21:57.367941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:22.963920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570184+0000","last_change":"2024-04-25T00:19:33.890412+0000","last_active":"2024-04-25T00:21:57.570184+0000","last_peered":"2024-04-25T00:21:57.570184+0000","last_clean":"2024-04-25T00:21:57.570184+0000","last_became_active":"2024-04-25T00:19:33.889684+0000","last_became_peered":"2024-04-25T00:19:33.889684+0000","last_unstale":"2024-04-25T00:21:57.570184+0000","last_undegraded":"2024-04-25T00:21:57.570184+0000","last_fullsized":"2024-04-25T00:21:57.570184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:57:45.686873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368450+0000","last_change":"2024-04-25T00:19:37.914047+0000","last_active":"2024-04-25T00:21:57.368450+0000","last_peered":"2024-04-25T00:21:57.368450+0000","last_clean":"2024-04-25T00:21:57.368450+0000","last_became_active":"2024-04-25T00:19:37.913598+0000","last_became_peered":"2024-04-25T00:19:37.913598+0000","last_unstale":"2024-04-25T00:21:57.368450+0000","last_undegraded":"2024-04-25T00:21:57.368450+0000","last_fullsized":"2024-04-25T00:21:57.368450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:57:17.169055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569175+0000","last_change":"2024-04-25T00:20:27.173867+0000","last_active":"2024-04-25T00:21:57.569175+0000","last_peered":"2024-04-25T00:21:57.569175+0000","last_clean":"2024-04-25T00:21:57.569175+0000","last_became_active":"2024-04-25T00:20:27.154058+0000","last_became_peered":"2024-04-25T00:20:27.154058+0000","last_unstale":"2024-04-25T00:21:57.569175+0000","last_undegraded":"2024-04-25T00:21:57.569175+0000","last_fullsized":"2024-04-25T00:21:57.569175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:55.519210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555526+0000","last_change":"2024-04-25T00:19:33.880978+0000","last_active":"2024-04-25T00:21:57.555526+0000","last_peered":"2024-04-25T00:21:57.555526+0000","last_clean":"2024-04-25T00:21:57.555526+0000","last_became_active":"2024-04-25T00:19:33.879699+0000","last_became_peered":"2024-04-25T00:19:33.879699+0000","last_unstale":"2024-04-25T00:21:57.555526+0000","last_undegraded":"2024-04-25T00:21:57.555526+0000","last_fullsized":"2024-04-25T00:21:57.555526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:55.707273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368407+0000","last_change":"2024-04-25T00:19:37.912222+0000","last_active":"2024-04-25T00:21:57.368407+0000","last_peered":"2024-04-25T00:21:57.368407+0000","last_clean":"2024-04-25T00:21:57.368407+0000","last_became_active":"2024-04-25T00:19:37.911752+0000","last_became_peered":"2024-04-25T00:19:37.911752+0000","last_unstale":"2024-04-25T00:21:57.368407+0000","last_undegraded":"2024-04-25T00:21:57.368407+0000","last_fullsized":"2024-04-25T00:21:57.368407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:34.171437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473816+0000","last_change":"2024-04-25T00:19:32.875207+0000","last_active":"2024-04-25T00:21:57.473816+0000","last_peered":"2024-04-25T00:21:57.473816+0000","last_clean":"2024-04-25T00:21:57.473816+0000","last_became_active":"2024-04-25T00:19:32.874896+0000","last_became_peered":"2024-04-25T00:19:32.874896+0000","last_unstale":"2024-04-25T00:21:57.473816+0000","last_undegraded":"2024-04-25T00:21:57.473816+0000","last_fullsized":"2024-04-25T00:21:57.473816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:45.463451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479776+0000","last_change":"2024-04-25T00:19:33.871768+0000","last_active":"2024-04-25T00:21:57.479776+0000","last_peered":"2024-04-25T00:21:57.479776+0000","last_clean":"2024-04-25T00:21:57.479776+0000","last_became_active":"2024-04-25T00:19:33.871084+0000","last_became_peered":"2024-04-25T00:19:33.871084+0000","last_unstale":"2024-04-25T00:21:57.479776+0000","last_undegraded":"2024-04-25T00:21:57.479776+0000","last_fullsized":"2024-04-25T00:21:57.479776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:47.581958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369325+0000","last_change":"2024-04-25T00:19:37.905019+0000","last_active":"2024-04-25T00:21:57.369325+0000","last_peered":"2024-04-25T00:21:57.369325+0000","last_clean":"2024-04-25T00:21:57.369325+0000","last_became_active":"2024-04-25T00:19:37.904276+0000","last_became_peered":"2024-04-25T00:19:37.904276+0000","last_unstale":"2024-04-25T00:21:57.369325+0000","last_undegraded":"2024-04-25T00:21:57.369325+0000","last_fullsized":"2024-04-25T00:21:57.369325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.197199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006431+0000","last_change":"2024-04-25T00:19:32.875990+0000","last_active":"2024-04-25T00:21:58.006431+0000","last_peered":"2024-04-25T00:21:58.006431+0000","last_clean":"2024-04-25T00:21:58.006431+0000","last_became_active":"2024-04-25T00:19:32.875664+0000","last_became_peered":"2024-04-25T00:19:32.875664+0000","last_unstale":"2024-04-25T00:21:58.006431+0000","last_undegraded":"2024-04-25T00:21:58.006431+0000","last_fullsized":"2024-04-25T00:21:58.006431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:31.214014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474170+0000","last_change":"2024-04-25T00:19:33.870448+0000","last_active":"2024-04-25T00:21:57.474170+0000","last_peered":"2024-04-25T00:21:57.474170+0000","last_clean":"2024-04-25T00:21:57.474170+0000","last_became_active":"2024-04-25T00:19:33.870087+0000","last_became_peered":"2024-04-25T00:19:33.870087+0000","last_unstale":"2024-04-25T00:21:57.474170+0000","last_undegraded":"2024-04-25T00:21:57.474170+0000","last_fullsized":"2024-04-25T00:21:57.474170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:19.497360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369206+0000","last_change":"2024-04-25T00:19:37.917699+0000","last_active":"2024-04-25T00:21:57.369206+0000","last_peered":"2024-04-25T00:21:57.369206+0000","last_clean":"2024-04-25T00:21:57.369206+0000","last_became_active":"2024-04-25T00:19:37.917289+0000","last_became_peered":"2024-04-25T00:19:37.917289+0000","last_unstale":"2024-04-25T00:21:57.369206+0000","last_undegraded":"2024-04-25T00:21:57.369206+0000","last_fullsized":"2024-04-25T00:21:57.369206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:42:24.633754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445520+0000","last_change":"2024-04-25T00:20:27.054883+0000","last_active":"2024-04-25T00:21:57.445520+0000","last_peered":"2024-04-25T00:21:57.445520+0000","last_clean":"2024-04-25T00:21:57.445520+0000","last_became_active":"2024-04-25T00:20:27.049595+0000","last_became_peered":"2024-04-25T00:20:27.049595+0000","last_unstale":"2024-04-25T00:21:57.445520+0000","last_undegraded":"2024-04-25T00:21:57.445520+0000","last_fullsized":"2024-04-25T00:21:57.445520+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:05.358193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479893+0000","last_change":"2024-04-25T00:19:33.884421+0000","last_active":"2024-04-25T00:21:57.479893+0000","last_peered":"2024-04-25T00:21:57.479893+0000","last_clean":"2024-04-25T00:21:57.479893+0000","last_became_active":"2024-04-25T00:19:33.884116+0000","last_became_peered":"2024-04-25T00:19:33.884116+0000","last_unstale":"2024-04-25T00:21:57.479893+0000","last_undegraded":"2024-04-25T00:21:57.479893+0000","last_fullsized":"2024-04-25T00:21:57.479893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:30.110401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569179+0000","last_change":"2024-04-25T00:19:37.907785+0000","last_active":"2024-04-25T00:21:57.569179+0000","last_peered":"2024-04-25T00:21:57.569179+0000","last_clean":"2024-04-25T00:21:57.569179+0000","last_became_active":"2024-04-25T00:19:37.907428+0000","last_became_peered":"2024-04-25T00:19:37.907428+0000","last_unstale":"2024-04-25T00:21:57.569179+0000","last_undegraded":"2024-04-25T00:21:57.569179+0000","last_fullsized":"2024-04-25T00:21:57.569179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:46.997401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569684+0000","last_change":"2024-04-25T00:19:32.868720+0000","last_active":"2024-04-25T00:21:57.569684+0000","last_peered":"2024-04-25T00:21:57.569684+0000","last_clean":"2024-04-25T00:21:57.569684+0000","last_became_active":"2024-04-25T00:19:32.868374+0000","last_became_peered":"2024-04-25T00:19:32.868374+0000","last_unstale":"2024-04-25T00:21:57.569684+0000","last_undegraded":"2024-04-25T00:21:57.569684+0000","last_fullsized":"2024-04-25T00:21:57.569684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:06.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367172+0000","last_change":"2024-04-25T00:19:33.882200+0000","last_active":"2024-04-25T00:21:57.367172+0000","last_peered":"2024-04-25T00:21:57.367172+0000","last_clean":"2024-04-25T00:21:57.367172+0000","last_became_active":"2024-04-25T00:19:33.881819+0000","last_became_peered":"2024-04-25T00:19:33.881819+0000","last_unstale":"2024-04-25T00:21:57.367172+0000","last_undegraded":"2024-04-25T00:21:57.367172+0000","last_fullsized":"2024-04-25T00:21:57.367172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:41.826431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570117+0000","last_change":"2024-04-25T00:19:37.911407+0000","last_active":"2024-04-25T00:21:57.570117+0000","last_peered":"2024-04-25T00:21:57.570117+0000","last_clean":"2024-04-25T00:21:57.570117+0000","last_became_active":"2024-04-25T00:19:37.911001+0000","last_became_peered":"2024-04-25T00:19:37.911001+0000","last_unstale":"2024-04-25T00:21:57.570117+0000","last_undegraded":"2024-04-25T00:21:57.570117+0000","last_fullsized":"2024-04-25T00:21:57.570117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:40.294397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479533+0000","last_change":"2024-04-25T00:19:32.878990+0000","last_active":"2024-04-25T00:21:57.479533+0000","last_peered":"2024-04-25T00:21:57.479533+0000","last_clean":"2024-04-25T00:21:57.479533+0000","last_became_active":"2024-04-25T00:19:32.878673+0000","last_became_peered":"2024-04-25T00:19:32.878673+0000","last_unstale":"2024-04-25T00:21:57.479533+0000","last_undegraded":"2024-04-25T00:21:57.479533+0000","last_fullsized":"2024-04-25T00:21:57.479533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:21.458819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471265+0000","last_change":"2024-04-25T00:19:33.877100+0000","last_active":"2024-04-25T00:21:57.471265+0000","last_peered":"2024-04-25T00:21:57.471265+0000","last_clean":"2024-04-25T00:21:57.471265+0000","last_became_active":"2024-04-25T00:19:33.876770+0000","last_became_peered":"2024-04-25T00:19:33.876770+0000","last_unstale":"2024-04-25T00:21:57.471265+0000","last_undegraded":"2024-04-25T00:21:57.471265+0000","last_fullsized":"2024-04-25T00:21:57.471265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:58.703290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367392+0000","last_change":"2024-04-25T00:19:37.911732+0000","last_active":"2024-04-25T00:21:57.367392+0000","last_peered":"2024-04-25T00:21:57.367392+0000","last_clean":"2024-04-25T00:21:57.367392+0000","last_became_active":"2024-04-25T00:19:37.911319+0000","last_became_peered":"2024-04-25T00:19:37.911319+0000","last_unstale":"2024-04-25T00:21:57.367392+0000","last_undegraded":"2024-04-25T00:21:57.367392+0000","last_fullsized":"2024-04-25T00:21:57.367392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:26.644085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":274,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006037+0000","last_change":"2024-04-25T00:19:32.875437+0000","last_active":"2024-04-25T00:21:58.006037+0000","last_peered":"2024-04-25T00:21:58.006037+0000","last_clean":"2024-04-25T00:21:58.006037+0000","last_became_active":"2024-04-25T00:19:32.875076+0000","last_became_peered":"2024-04-25T00:19:32.875076+0000","last_unstale":"2024-04-25T00:21:58.006037+0000","last_undegraded":"2024-04-25T00:21:58.006037+0000","last_fullsized":"2024-04-25T00:21:58.006037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:02.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.556589+0000","last_change":"2024-04-25T00:19:33.878799+0000","last_active":"2024-04-25T00:21:57.556589+0000","last_peered":"2024-04-25T00:21:57.556589+0000","last_clean":"2024-04-25T00:21:57.556589+0000","last_became_active":"2024-04-25T00:19:33.878454+0000","last_became_peered":"2024-04-25T00:19:33.878454+0000","last_unstale":"2024-04-25T00:21:57.556589+0000","last_undegraded":"2024-04-25T00:21:57.556589+0000","last_fullsized":"2024-04-25T00:21:57.556589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:47.486041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471195+0000","last_change":"2024-04-25T00:19:37.912054+0000","last_active":"2024-04-25T00:21:57.471195+0000","last_peered":"2024-04-25T00:21:57.471195+0000","last_clean":"2024-04-25T00:21:57.471195+0000","last_became_active":"2024-04-25T00:19:37.911684+0000","last_became_peered":"2024-04-25T00:19:37.911684+0000","last_unstale":"2024-04-25T00:21:57.471195+0000","last_undegraded":"2024-04-25T00:21:57.471195+0000","last_fullsized":"2024-04-25T00:21:57.471195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:28:34.034253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569301+0000","last_change":"2024-04-25T00:20:27.176907+0000","last_active":"2024-04-25T00:21:57.569301+0000","last_peered":"2024-04-25T00:21:57.569301+0000","last_clean":"2024-04-25T00:21:57.569301+0000","last_became_active":"2024-04-25T00:20:27.153755+0000","last_became_peered":"2024-04-25T00:20:27.153755+0000","last_unstale":"2024-04-25T00:21:57.569301+0000","last_undegraded":"2024-04-25T00:21:57.569301+0000","last_fullsized":"2024-04-25T00:21:57.569301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:07.142383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570525+0000","last_change":"2024-04-25T00:19:33.891682+0000","last_active":"2024-04-25T00:21:57.570525+0000","last_peered":"2024-04-25T00:21:57.570525+0000","last_clean":"2024-04-25T00:21:57.570525+0000","last_became_active":"2024-04-25T00:19:33.891346+0000","last_became_peered":"2024-04-25T00:19:33.891346+0000","last_unstale":"2024-04-25T00:21:57.570525+0000","last_undegraded":"2024-04-25T00:21:57.570525+0000","last_fullsized":"2024-04-25T00:21:57.570525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:13.149265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368645+0000","last_change":"2024-04-25T00:19:37.904336+0000","last_active":"2024-04-25T00:21:57.368645+0000","last_peered":"2024-04-25T00:21:57.368645+0000","last_clean":"2024-04-25T00:21:57.368645+0000","last_became_active":"2024-04-25T00:19:37.903938+0000","last_became_peered":"2024-04-25T00:19:37.903938+0000","last_unstale":"2024-04-25T00:21:57.368645+0000","last_undegraded":"2024-04-25T00:21:57.368645+0000","last_fullsized":"2024-04-25T00:21:57.368645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:06.643985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479864+0000","last_change":"2024-04-25T00:19:32.882527+0000","last_active":"2024-04-25T00:21:57.479864+0000","last_peered":"2024-04-25T00:21:57.479864+0000","last_clean":"2024-04-25T00:21:57.479864+0000","last_became_active":"2024-04-25T00:19:32.882134+0000","last_became_peered":"2024-04-25T00:19:32.882134+0000","last_unstale":"2024-04-25T00:21:57.479864+0000","last_undegraded":"2024-04-25T00:21:57.479864+0000","last_fullsized":"2024-04-25T00:21:57.479864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:42.918006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555798+0000","last_change":"2024-04-25T00:19:33.873847+0000","last_active":"2024-04-25T00:21:57.555798+0000","last_peered":"2024-04-25T00:21:57.555798+0000","last_clean":"2024-04-25T00:21:57.555798+0000","last_became_active":"2024-04-25T00:19:33.873510+0000","last_became_peered":"2024-04-25T00:19:33.873510+0000","last_unstale":"2024-04-25T00:21:57.555798+0000","last_undegraded":"2024-04-25T00:21:57.555798+0000","last_fullsized":"2024-04-25T00:21:57.555798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:36:47.177417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471386+0000","last_change":"2024-04-25T00:19:37.905701+0000","last_active":"2024-04-25T00:21:57.471386+0000","last_peered":"2024-04-25T00:21:57.471386+0000","last_clean":"2024-04-25T00:21:57.471386+0000","last_became_active":"2024-04-25T00:19:37.905275+0000","last_became_peered":"2024-04-25T00:19:37.905275+0000","last_unstale":"2024-04-25T00:21:57.471386+0000","last_undegraded":"2024-04-25T00:21:57.471386+0000","last_fullsized":"2024-04-25T00:21:57.471386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:26.663140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.006697+0000","last_change":"2024-04-25T00:19:32.874645+0000","last_active":"2024-04-25T00:21:58.006697+0000","last_peered":"2024-04-25T00:21:58.006697+0000","last_clean":"2024-04-25T00:21:58.006697+0000","last_became_active":"2024-04-25T00:19:32.874335+0000","last_became_peered":"2024-04-25T00:19:32.874335+0000","last_unstale":"2024-04-25T00:21:58.006697+0000","last_undegraded":"2024-04-25T00:21:58.006697+0000","last_fullsized":"2024-04-25T00:21:58.006697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:12.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474031+0000","last_change":"2024-04-25T00:19:33.872108+0000","last_active":"2024-04-25T00:21:57.474031+0000","last_peered":"2024-04-25T00:21:57.474031+0000","last_clean":"2024-04-25T00:21:57.474031+0000","last_became_active":"2024-04-25T00:19:33.871711+0000","last_became_peered":"2024-04-25T00:19:33.871711+0000","last_unstale":"2024-04-25T00:21:57.474031+0000","last_undegraded":"2024-04-25T00:21:57.474031+0000","last_fullsized":"2024-04-25T00:21:57.474031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:21.556176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.480095+0000","last_change":"2024-04-25T00:19:37.911497+0000","last_active":"2024-04-25T00:21:57.480095+0000","last_peered":"2024-04-25T00:21:57.480095+0000","last_clean":"2024-04-25T00:21:57.480095+0000","last_became_active":"2024-04-25T00:19:37.911126+0000","last_became_peered":"2024-04-25T00:19:37.911126+0000","last_unstale":"2024-04-25T00:21:57.480095+0000","last_undegraded":"2024-04-25T00:21:57.480095+0000","last_fullsized":"2024-04-25T00:21:57.480095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:42.594332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369252+0000","last_change":"2024-04-25T00:19:32.880365+0000","last_active":"2024-04-25T00:21:57.369252+0000","last_peered":"2024-04-25T00:21:57.369252+0000","last_clean":"2024-04-25T00:21:57.369252+0000","last_became_active":"2024-04-25T00:19:32.880050+0000","last_became_peered":"2024-04-25T00:19:32.880050+0000","last_unstale":"2024-04-25T00:21:57.369252+0000","last_undegraded":"2024-04-25T00:21:57.369252+0000","last_fullsized":"2024-04-25T00:21:57.369252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.626134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570057+0000","last_change":"2024-04-25T00:19:33.878515+0000","last_active":"2024-04-25T00:21:57.570057+0000","last_peered":"2024-04-25T00:21:57.570057+0000","last_clean":"2024-04-25T00:21:57.570057+0000","last_became_active":"2024-04-25T00:19:33.877695+0000","last_became_peered":"2024-04-25T00:19:33.877695+0000","last_unstale":"2024-04-25T00:21:57.570057+0000","last_undegraded":"2024-04-25T00:21:57.570057+0000","last_fullsized":"2024-04-25T00:21:57.570057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:21.922416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568987+0000","last_change":"2024-04-25T00:19:37.910108+0000","last_active":"2024-04-25T00:21:57.568987+0000","last_peered":"2024-04-25T00:21:57.568987+0000","last_clean":"2024-04-25T00:21:57.568987+0000","last_became_active":"2024-04-25T00:19:37.909718+0000","last_became_peered":"2024-04-25T00:19:37.909718+0000","last_unstale":"2024-04-25T00:21:57.568987+0000","last_undegraded":"2024-04-25T00:21:57.568987+0000","last_fullsized":"2024-04-25T00:21:57.568987+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:52.187480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479870+0000","last_change":"2024-04-25T00:19:32.878812+0000","last_active":"2024-04-25T00:21:57.479870+0000","last_peered":"2024-04-25T00:21:57.479870+0000","last_clean":"2024-04-25T00:21:57.479870+0000","last_became_active":"2024-04-25T00:19:32.878334+0000","last_became_peered":"2024-04-25T00:19:32.878334+0000","last_unstale":"2024-04-25T00:21:57.479870+0000","last_undegraded":"2024-04-25T00:21:57.479870+0000","last_fullsized":"2024-04-25T00:21:57.479870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:46.419254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569260+0000","last_change":"2024-04-25T00:19:33.884997+0000","last_active":"2024-04-25T00:21:57.569260+0000","last_peered":"2024-04-25T00:21:57.569260+0000","last_clean":"2024-04-25T00:21:57.569260+0000","last_became_active":"2024-04-25T00:19:33.884528+0000","last_became_peered":"2024-04-25T00:19:33.884528+0000","last_unstale":"2024-04-25T00:21:57.569260+0000","last_undegraded":"2024-04-25T00:21:57.569260+0000","last_fullsized":"2024-04-25T00:21:57.569260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:23.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474056+0000","last_change":"2024-04-25T00:20:27.153934+0000","last_active":"2024-04-25T00:21:57.474056+0000","last_peered":"2024-04-25T00:21:57.474056+0000","last_clean":"2024-04-25T00:21:57.474056+0000","last_became_active":"2024-04-25T00:20:27.152647+0000","last_became_peered":"2024-04-25T00:20:27.152647+0000","last_unstale":"2024-04-25T00:21:57.474056+0000","last_undegraded":"2024-04-25T00:21:57.474056+0000","last_fullsized":"2024-04-25T00:21:57.474056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.389722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:58.019845+0000","last_change":"2024-04-25T00:19:32.871120+0000","last_active":"2024-04-25T00:21:58.019845+0000","last_peered":"2024-04-25T00:21:58.019845+0000","last_clean":"2024-04-25T00:21:58.019845+0000","last_became_active":"2024-04-25T00:19:32.870786+0000","last_became_peered":"2024-04-25T00:19:32.870786+0000","last_unstale":"2024-04-25T00:21:58.019845+0000","last_undegraded":"2024-04-25T00:21:58.019845+0000","last_fullsized":"2024-04-25T00:21:58.019845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:25.293359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569027+0000","last_change":"2024-04-25T00:19:33.884029+0000","last_active":"2024-04-25T00:21:57.569027+0000","last_peered":"2024-04-25T00:21:57.569027+0000","last_clean":"2024-04-25T00:21:57.569027+0000","last_became_active":"2024-04-25T00:19:33.883698+0000","last_became_peered":"2024-04-25T00:19:33.883698+0000","last_unstale":"2024-04-25T00:21:57.569027+0000","last_undegraded":"2024-04-25T00:21:57.569027+0000","last_fullsized":"2024-04-25T00:21:57.569027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:51:54.786536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571825+0000","last_change":"2024-04-25T00:19:37.912622+0000","last_active":"2024-04-25T00:21:57.571825+0000","last_peered":"2024-04-25T00:21:57.571825+0000","last_clean":"2024-04-25T00:21:57.571825+0000","last_became_active":"2024-04-25T00:19:37.911895+0000","last_became_peered":"2024-04-25T00:19:37.911895+0000","last_unstale":"2024-04-25T00:21:57.571825+0000","last_undegraded":"2024-04-25T00:21:57.571825+0000","last_fullsized":"2024-04-25T00:21:57.571825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:10.185382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473779+0000","last_change":"2024-04-25T00:19:32.879237+0000","last_active":"2024-04-25T00:21:57.473779+0000","last_peered":"2024-04-25T00:21:57.473779+0000","last_clean":"2024-04-25T00:21:57.473779+0000","last_became_active":"2024-04-25T00:19:32.878918+0000","last_became_peered":"2024-04-25T00:19:32.878918+0000","last_unstale":"2024-04-25T00:21:57.473779+0000","last_undegraded":"2024-04-25T00:21:57.473779+0000","last_fullsized":"2024-04-25T00:21:57.473779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:26:11.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479596+0000","last_change":"2024-04-25T00:19:33.868864+0000","last_active":"2024-04-25T00:21:57.479596+0000","last_peered":"2024-04-25T00:21:57.479596+0000","last_clean":"2024-04-25T00:21:57.479596+0000","last_became_active":"2024-04-25T00:19:33.868494+0000","last_became_peered":"2024-04-25T00:19:33.868494+0000","last_unstale":"2024-04-25T00:21:57.479596+0000","last_undegraded":"2024-04-25T00:21:57.479596+0000","last_fullsized":"2024-04-25T00:21:57.479596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:01.650437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479496+0000","last_change":"2024-04-25T00:19:37.913103+0000","last_active":"2024-04-25T00:21:57.479496+0000","last_peered":"2024-04-25T00:21:57.479496+0000","last_clean":"2024-04-25T00:21:57.479496+0000","last_became_active":"2024-04-25T00:19:37.912658+0000","last_became_peered":"2024-04-25T00:19:37.912658+0000","last_unstale":"2024-04-25T00:21:57.479496+0000","last_undegraded":"2024-04-25T00:21:57.479496+0000","last_fullsized":"2024-04-25T00:21:57.479496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:03.213223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474293+0000","last_change":"2024-04-25T00:19:32.879142+0000","last_active":"2024-04-25T00:21:57.474293+0000","last_peered":"2024-04-25T00:21:57.474293+0000","last_clean":"2024-04-25T00:21:57.474293+0000","last_became_active":"2024-04-25T00:19:32.878783+0000","last_became_peered":"2024-04-25T00:19:32.878783+0000","last_unstale":"2024-04-25T00:21:57.474293+0000","last_undegraded":"2024-04-25T00:21:57.474293+0000","last_fullsized":"2024-04-25T00:21:57.474293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:39.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569223+0000","last_change":"2024-04-25T00:21:27.923915+0000","last_active":"2024-04-25T00:21:57.569223+0000","last_peered":"2024-04-25T00:21:57.569223+0000","last_clean":"2024-04-25T00:21:57.569223+0000","last_became_active":"2024-04-25T00:21:27.923465+0000","last_became_peered":"2024-04-25T00:21:27.923465+0000","last_unstale":"2024-04-25T00:21:57.569223+0000","last_undegraded":"2024-04-25T00:21:57.569223+0000","last_fullsized":"2024-04-25T00:21:57.569223+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.804110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473722+0000","last_change":"2024-04-25T00:19:37.966599+0000","last_active":"2024-04-25T00:21:57.473722+0000","last_peered":"2024-04-25T00:21:57.473722+0000","last_clean":"2024-04-25T00:21:57.473722+0000","last_became_active":"2024-04-25T00:19:37.966234+0000","last_became_peered":"2024-04-25T00:19:37.966234+0000","last_unstale":"2024-04-25T00:21:57.473722+0000","last_undegraded":"2024-04-25T00:21:57.473722+0000","last_fullsized":"2024-04-25T00:21:57.473722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:45.572495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.471729+0000","last_change":"2024-04-25T00:19:32.877596+0000","last_active":"2024-04-25T00:21:57.471729+0000","last_peered":"2024-04-25T00:21:57.471729+0000","last_clean":"2024-04-25T00:21:57.471729+0000","last_became_active":"2024-04-25T00:19:32.877266+0000","last_became_peered":"2024-04-25T00:19:32.877266+0000","last_unstale":"2024-04-25T00:21:57.471729+0000","last_undegraded":"2024-04-25T00:21:57.471729+0000","last_fullsized":"2024-04-25T00:21:57.471729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:51.052149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445825+0000","last_change":"2024-04-25T00:19:33.888061+0000","last_active":"2024-04-25T00:21:57.445825+0000","last_peered":"2024-04-25T00:21:57.445825+0000","last_clean":"2024-04-25T00:21:57.445825+0000","last_became_active":"2024-04-25T00:19:33.887687+0000","last_became_peered":"2024-04-25T00:19:33.887687+0000","last_unstale":"2024-04-25T00:21:57.445825+0000","last_undegraded":"2024-04-25T00:21:57.445825+0000","last_fullsized":"2024-04-25T00:21:57.445825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:47.515924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571137+0000","last_change":"2024-04-25T00:19:37.909452+0000","last_active":"2024-04-25T00:21:57.571137+0000","last_peered":"2024-04-25T00:21:57.571137+0000","last_clean":"2024-04-25T00:21:57.571137+0000","last_became_active":"2024-04-25T00:19:37.909030+0000","last_became_peered":"2024-04-25T00:19:37.909030+0000","last_unstale":"2024-04-25T00:21:57.571137+0000","last_undegraded":"2024-04-25T00:21:57.571137+0000","last_fullsized":"2024-04-25T00:21:57.571137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:44.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569036+0000","last_change":"2024-04-25T00:19:32.873149+0000","last_active":"2024-04-25T00:21:57.569036+0000","last_peered":"2024-04-25T00:21:57.569036+0000","last_clean":"2024-04-25T00:21:57.569036+0000","last_became_active":"2024-04-25T00:19:32.872743+0000","last_became_peered":"2024-04-25T00:19:32.872743+0000","last_unstale":"2024-04-25T00:21:57.569036+0000","last_undegraded":"2024-04-25T00:21:57.569036+0000","last_fullsized":"2024-04-25T00:21:57.569036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:19.449810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.445627+0000","last_change":"2024-04-25T00:19:33.887921+0000","last_active":"2024-04-25T00:21:57.445627+0000","last_peered":"2024-04-25T00:21:57.445627+0000","last_clean":"2024-04-25T00:21:57.445627+0000","last_became_active":"2024-04-25T00:19:33.887545+0000","last_became_peered":"2024-04-25T00:19:33.887545+0000","last_unstale":"2024-04-25T00:21:57.445627+0000","last_undegraded":"2024-04-25T00:21:57.445627+0000","last_fullsized":"2024-04-25T00:21:57.445627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:44.788920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571805+0000","last_change":"2024-04-25T00:19:37.966792+0000","last_active":"2024-04-25T00:21:57.571805+0000","last_peered":"2024-04-25T00:21:57.571805+0000","last_clean":"2024-04-25T00:21:57.571805+0000","last_became_active":"2024-04-25T00:19:37.966350+0000","last_became_peered":"2024-04-25T00:19:37.966350+0000","last_unstale":"2024-04-25T00:21:57.571805+0000","last_undegraded":"2024-04-25T00:21:57.571805+0000","last_fullsized":"2024-04-25T00:21:57.571805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:02.280594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479762+0000","last_change":"2024-04-25T00:19:33.877371+0000","last_active":"2024-04-25T00:21:57.479762+0000","last_peered":"2024-04-25T00:21:57.479762+0000","last_clean":"2024-04-25T00:21:57.479762+0000","last_became_active":"2024-04-25T00:19:33.872122+0000","last_became_peered":"2024-04-25T00:19:33.872122+0000","last_unstale":"2024-04-25T00:21:57.479762+0000","last_undegraded":"2024-04-25T00:21:57.479762+0000","last_fullsized":"2024-04-25T00:21:57.479762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:56.765503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.707830+0000","last_change":"2024-04-25T00:19:32.879773+0000","last_active":"2024-04-25T00:21:57.707830+0000","last_peered":"2024-04-25T00:21:57.707830+0000","last_clean":"2024-04-25T00:21:57.707830+0000","last_became_active":"2024-04-25T00:19:32.879404+0000","last_became_peered":"2024-04-25T00:19:32.879404+0000","last_unstale":"2024-04-25T00:21:57.707830+0000","last_undegraded":"2024-04-25T00:21:57.707830+0000","last_fullsized":"2024-04-25T00:21:57.707830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:22.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:37.904861+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:37.904407+0000","last_became_peered":"2024-04-25T00:19:37.904407+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:09.954500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367200+0000","last_change":"2024-04-25T00:19:33.886415+0000","last_active":"2024-04-25T00:21:57.367200+0000","last_peered":"2024-04-25T00:21:57.367200+0000","last_clean":"2024-04-25T00:21:57.367200+0000","last_became_active":"2024-04-25T00:19:33.885140+0000","last_became_peered":"2024-04-25T00:19:33.885140+0000","last_unstale":"2024-04-25T00:21:57.367200+0000","last_undegraded":"2024-04-25T00:21:57.367200+0000","last_fullsized":"2024-04-25T00:21:57.367200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:33.893814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368588+0000","last_change":"2024-04-25T00:19:32.866916+0000","last_active":"2024-04-25T00:21:57.368588+0000","last_peered":"2024-04-25T00:21:57.368588+0000","last_clean":"2024-04-25T00:21:57.368588+0000","last_became_active":"2024-04-25T00:19:32.866564+0000","last_became_peered":"2024-04-25T00:19:32.866564+0000","last_unstale":"2024-04-25T00:21:57.368588+0000","last_undegraded":"2024-04-25T00:21:57.368588+0000","last_fullsized":"2024-04-25T00:21:57.368588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:53:58.428919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568856+0000","last_change":"2024-04-25T00:18:17.353618+0000","last_active":"2024-04-25T00:21:57.568856+0000","last_peered":"2024-04-25T00:21:57.568856+0000","last_clean":"2024-04-25T00:21:57.568856+0000","last_became_active":"2024-04-25T00:18:17.353247+0000","last_became_peered":"2024-04-25T00:18:17.353247+0000","last_unstale":"2024-04-25T00:21:57.568856+0000","last_undegraded":"2024-04-25T00:21:57.568856+0000","last_fullsized":"2024-04-25T00:21:57.568856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:51.894242+0000","last_clean_scrub_stamp":"2024-04-25T00:14:51.894242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:59.160841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474047+0000","last_change":"2024-04-25T00:20:27.049784+0000","last_active":"2024-04-25T00:21:57.474047+0000","last_peered":"2024-04-25T00:21:57.474047+0000","last_clean":"2024-04-25T00:21:57.474047+0000","last_became_active":"2024-04-25T00:20:27.049413+0000","last_became_peered":"2024-04-25T00:20:27.049413+0000","last_unstale":"2024-04-25T00:21:57.474047+0000","last_undegraded":"2024-04-25T00:21:57.474047+0000","last_fullsized":"2024-04-25T00:21:57.474047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:28:26.533987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569353+0000","last_change":"2024-04-25T00:19:33.875824+0000","last_active":"2024-04-25T00:21:57.569353+0000","last_peered":"2024-04-25T00:21:57.569353+0000","last_clean":"2024-04-25T00:21:57.569353+0000","last_became_active":"2024-04-25T00:19:33.875439+0000","last_became_peered":"2024-04-25T00:19:33.875439+0000","last_unstale":"2024-04-25T00:21:57.569353+0000","last_undegraded":"2024-04-25T00:21:57.569353+0000","last_fullsized":"2024-04-25T00:21:57.569353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:15.461031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568947+0000","last_change":"2024-04-25T00:19:32.872172+0000","last_active":"2024-04-25T00:21:57.568947+0000","last_peered":"2024-04-25T00:21:57.568947+0000","last_clean":"2024-04-25T00:21:57.568947+0000","last_became_active":"2024-04-25T00:19:32.871199+0000","last_became_peered":"2024-04-25T00:19:32.871199+0000","last_unstale":"2024-04-25T00:21:57.568947+0000","last_undegraded":"2024-04-25T00:21:57.568947+0000","last_fullsized":"2024-04-25T00:21:57.568947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:34.722015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568945+0000","last_change":"2024-04-25T00:19:37.910128+0000","last_active":"2024-04-25T00:21:57.568945+0000","last_peered":"2024-04-25T00:21:57.568945+0000","last_clean":"2024-04-25T00:21:57.568945+0000","last_became_active":"2024-04-25T00:19:37.909699+0000","last_became_peered":"2024-04-25T00:19:37.909699+0000","last_unstale":"2024-04-25T00:21:57.568945+0000","last_undegraded":"2024-04-25T00:21:57.568945+0000","last_fullsized":"2024-04-25T00:21:57.568945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:38.108835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474064+0000","last_change":"2024-04-25T00:19:32.877945+0000","last_active":"2024-04-25T00:21:57.474064+0000","last_peered":"2024-04-25T00:21:57.474064+0000","last_clean":"2024-04-25T00:21:57.474064+0000","last_became_active":"2024-04-25T00:19:32.877557+0000","last_became_peered":"2024-04-25T00:19:32.877557+0000","last_unstale":"2024-04-25T00:21:57.474064+0000","last_undegraded":"2024-04-25T00:21:57.474064+0000","last_fullsized":"2024-04-25T00:21:57.474064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:40:59.180770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571331+0000","last_change":"2024-04-25T00:19:33.876964+0000","last_active":"2024-04-25T00:21:57.571331+0000","last_peered":"2024-04-25T00:21:57.571331+0000","last_clean":"2024-04-25T00:21:57.571331+0000","last_became_active":"2024-04-25T00:19:33.876609+0000","last_became_peered":"2024-04-25T00:19:33.876609+0000","last_unstale":"2024-04-25T00:21:57.571331+0000","last_undegraded":"2024-04-25T00:21:57.571331+0000","last_fullsized":"2024-04-25T00:21:57.571331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.688160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367171+0000","last_change":"2024-04-25T00:19:37.967239+0000","last_active":"2024-04-25T00:21:57.367171+0000","last_peered":"2024-04-25T00:21:57.367171+0000","last_clean":"2024-04-25T00:21:57.367171+0000","last_became_active":"2024-04-25T00:19:37.966487+0000","last_became_peered":"2024-04-25T00:19:37.966487+0000","last_unstale":"2024-04-25T00:21:57.367171+0000","last_undegraded":"2024-04-25T00:21:57.367171+0000","last_fullsized":"2024-04-25T00:21:57.367171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:53.942262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'47","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.598104+0000","last_change":"2024-04-25T00:19:32.870258+0000","last_active":"2024-04-25T00:21:57.598104+0000","last_peered":"2024-04-25T00:21:57.598104+0000","last_clean":"2024-04-25T00:21:57.598104+0000","last_became_active":"2024-04-25T00:19:32.869916+0000","last_became_peered":"2024-04-25T00:19:32.869916+0000","last_unstale":"2024-04-25T00:21:57.598104+0000","last_undegraded":"2024-04-25T00:21:57.598104+0000","last_fullsized":"2024-04-25T00:21:57.598104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:29.617892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368050+0000","last_change":"2024-04-25T00:19:33.894784+0000","last_active":"2024-04-25T00:21:57.368050+0000","last_peered":"2024-04-25T00:21:57.368050+0000","last_clean":"2024-04-25T00:21:57.368050+0000","last_became_active":"2024-04-25T00:19:33.894413+0000","last_became_peered":"2024-04-25T00:19:33.894413+0000","last_unstale":"2024-04-25T00:21:57.368050+0000","last_undegraded":"2024-04-25T00:21:57.368050+0000","last_fullsized":"2024-04-25T00:21:57.368050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:29.891736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368768+0000","last_change":"2024-04-25T00:19:37.901168+0000","last_active":"2024-04-25T00:21:57.368768+0000","last_peered":"2024-04-25T00:21:57.368768+0000","last_clean":"2024-04-25T00:21:57.368768+0000","last_became_active":"2024-04-25T00:19:37.900836+0000","last_became_peered":"2024-04-25T00:19:37.900836+0000","last_unstale":"2024-04-25T00:21:57.368768+0000","last_undegraded":"2024-04-25T00:21:57.368768+0000","last_fullsized":"2024-04-25T00:21:57.368768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.989937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368990+0000","last_change":"2024-04-25T00:19:32.868901+0000","last_active":"2024-04-25T00:21:57.368990+0000","last_peered":"2024-04-25T00:21:57.368990+0000","last_clean":"2024-04-25T00:21:57.368990+0000","last_became_active":"2024-04-25T00:19:32.868506+0000","last_became_peered":"2024-04-25T00:19:32.868506+0000","last_unstale":"2024-04-25T00:21:57.368990+0000","last_undegraded":"2024-04-25T00:21:57.368990+0000","last_fullsized":"2024-04-25T00:21:57.368990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.290236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570368+0000","last_change":"2024-04-25T00:19:33.886344+0000","last_active":"2024-04-25T00:21:57.570368+0000","last_peered":"2024-04-25T00:21:57.570368+0000","last_clean":"2024-04-25T00:21:57.570368+0000","last_became_active":"2024-04-25T00:19:33.885693+0000","last_became_peered":"2024-04-25T00:19:33.885693+0000","last_unstale":"2024-04-25T00:21:57.570368+0000","last_undegraded":"2024-04-25T00:21:57.570368+0000","last_fullsized":"2024-04-25T00:21:57.570368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:29:01.747854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368871+0000","last_change":"2024-04-25T00:19:37.967007+0000","last_active":"2024-04-25T00:21:57.368871+0000","last_peered":"2024-04-25T00:21:57.368871+0000","last_clean":"2024-04-25T00:21:57.368871+0000","last_became_active":"2024-04-25T00:19:37.966600+0000","last_became_peered":"2024-04-25T00:19:37.966600+0000","last_unstale":"2024-04-25T00:21:57.368871+0000","last_undegraded":"2024-04-25T00:21:57.368871+0000","last_fullsized":"2024-04-25T00:21:57.368871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:39.521231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555187+0000","last_change":"2024-04-25T00:19:32.877751+0000","last_active":"2024-04-25T00:21:57.555187+0000","last_peered":"2024-04-25T00:21:57.555187+0000","last_clean":"2024-04-25T00:21:57.555187+0000","last_became_active":"2024-04-25T00:19:32.877482+0000","last_became_peered":"2024-04-25T00:19:32.877482+0000","last_unstale":"2024-04-25T00:21:57.555187+0000","last_undegraded":"2024-04-25T00:21:57.555187+0000","last_fullsized":"2024-04-25T00:21:57.555187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:03.541329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473693+0000","last_change":"2024-04-25T00:19:33.874590+0000","last_active":"2024-04-25T00:21:57.473693+0000","last_peered":"2024-04-25T00:21:57.473693+0000","last_clean":"2024-04-25T00:21:57.473693+0000","last_became_active":"2024-04-25T00:19:33.874226+0000","last_became_peered":"2024-04-25T00:19:33.874226+0000","last_unstale":"2024-04-25T00:21:57.473693+0000","last_undegraded":"2024-04-25T00:21:57.473693+0000","last_fullsized":"2024-04-25T00:21:57.473693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:01.090865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.474191+0000","last_change":"2024-04-25T00:20:27.153703+0000","last_active":"2024-04-25T00:21:57.474191+0000","last_peered":"2024-04-25T00:21:57.474191+0000","last_clean":"2024-04-25T00:21:57.474191+0000","last_became_active":"2024-04-25T00:20:27.152715+0000","last_became_peered":"2024-04-25T00:20:27.152715+0000","last_unstale":"2024-04-25T00:21:57.474191+0000","last_undegraded":"2024-04-25T00:21:57.474191+0000","last_fullsized":"2024-04-25T00:21:57.474191+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:09.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367131+0000","last_change":"2024-04-25T00:19:32.871721+0000","last_active":"2024-04-25T00:21:57.367131+0000","last_peered":"2024-04-25T00:21:57.367131+0000","last_clean":"2024-04-25T00:21:57.367131+0000","last_became_active":"2024-04-25T00:19:32.870979+0000","last_became_peered":"2024-04-25T00:19:32.870979+0000","last_unstale":"2024-04-25T00:21:57.367131+0000","last_undegraded":"2024-04-25T00:21:57.367131+0000","last_fullsized":"2024-04-25T00:21:57.367131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:06.355230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568847+0000","last_change":"2024-04-25T00:19:33.878779+0000","last_active":"2024-04-25T00:21:57.568847+0000","last_peered":"2024-04-25T00:21:57.568847+0000","last_clean":"2024-04-25T00:21:57.568847+0000","last_became_active":"2024-04-25T00:19:33.878394+0000","last_became_peered":"2024-04-25T00:19:33.878394+0000","last_unstale":"2024-04-25T00:21:57.568847+0000","last_undegraded":"2024-04-25T00:21:57.568847+0000","last_fullsized":"2024-04-25T00:21:57.568847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:30.862712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368683+0000","last_change":"2024-04-25T00:19:37.914316+0000","last_active":"2024-04-25T00:21:57.368683+0000","last_peered":"2024-04-25T00:21:57.368683+0000","last_clean":"2024-04-25T00:21:57.368683+0000","last_became_active":"2024-04-25T00:19:37.913802+0000","last_became_peered":"2024-04-25T00:19:37.913802+0000","last_unstale":"2024-04-25T00:21:57.368683+0000","last_undegraded":"2024-04-25T00:21:57.368683+0000","last_fullsized":"2024-04-25T00:21:57.368683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:53:45.872050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369170+0000","last_change":"2024-04-25T00:19:32.881280+0000","last_active":"2024-04-25T00:21:57.369170+0000","last_peered":"2024-04-25T00:21:57.369170+0000","last_clean":"2024-04-25T00:21:57.369170+0000","last_became_active":"2024-04-25T00:19:32.880883+0000","last_became_peered":"2024-04-25T00:19:32.880883+0000","last_unstale":"2024-04-25T00:21:57.369170+0000","last_undegraded":"2024-04-25T00:21:57.369170+0000","last_fullsized":"2024-04-25T00:21:57.369170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:34.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367712+0000","last_change":"2024-04-25T00:19:33.896472+0000","last_active":"2024-04-25T00:21:57.367712+0000","last_peered":"2024-04-25T00:21:57.367712+0000","last_clean":"2024-04-25T00:21:57.367712+0000","last_became_active":"2024-04-25T00:19:33.895356+0000","last_became_peered":"2024-04-25T00:19:33.895356+0000","last_unstale":"2024-04-25T00:21:57.367712+0000","last_undegraded":"2024-04-25T00:21:57.367712+0000","last_fullsized":"2024-04-25T00:21:57.367712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:21.523684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368859+0000","last_change":"2024-04-25T00:19:37.966785+0000","last_active":"2024-04-25T00:21:57.368859+0000","last_peered":"2024-04-25T00:21:57.368859+0000","last_clean":"2024-04-25T00:21:57.368859+0000","last_became_active":"2024-04-25T00:19:37.966382+0000","last_became_peered":"2024-04-25T00:19:37.966382+0000","last_unstale":"2024-04-25T00:21:57.368859+0000","last_undegraded":"2024-04-25T00:21:57.368859+0000","last_fullsized":"2024-04-25T00:21:57.368859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:52.240951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.473798+0000","last_change":"2024-04-25T00:19:32.871146+0000","last_active":"2024-04-25T00:21:57.473798+0000","last_peered":"2024-04-25T00:21:57.473798+0000","last_clean":"2024-04-25T00:21:57.473798+0000","last_became_active":"2024-04-25T00:19:32.870817+0000","last_became_peered":"2024-04-25T00:19:32.870817+0000","last_unstale":"2024-04-25T00:21:57.473798+0000","last_undegraded":"2024-04-25T00:21:57.473798+0000","last_fullsized":"2024-04-25T00:21:57.473798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:21.713617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569147+0000","last_change":"2024-04-25T00:19:33.891169+0000","last_active":"2024-04-25T00:21:57.569147+0000","last_peered":"2024-04-25T00:21:57.569147+0000","last_clean":"2024-04-25T00:21:57.569147+0000","last_became_active":"2024-04-25T00:19:33.890491+0000","last_became_peered":"2024-04-25T00:19:33.890491+0000","last_unstale":"2024-04-25T00:21:57.569147+0000","last_undegraded":"2024-04-25T00:21:57.569147+0000","last_fullsized":"2024-04-25T00:21:57.569147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:57.721638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555469+0000","last_change":"2024-04-25T00:19:37.904723+0000","last_active":"2024-04-25T00:21:57.555469+0000","last_peered":"2024-04-25T00:21:57.555469+0000","last_clean":"2024-04-25T00:21:57.555469+0000","last_became_active":"2024-04-25T00:19:37.904318+0000","last_became_peered":"2024-04-25T00:19:37.904318+0000","last_unstale":"2024-04-25T00:21:57.555469+0000","last_undegraded":"2024-04-25T00:21:57.555469+0000","last_fullsized":"2024-04-25T00:21:57.555469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:40.433487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369406+0000","last_change":"2024-04-25T00:19:32.874376+0000","last_active":"2024-04-25T00:21:57.369406+0000","last_peered":"2024-04-25T00:21:57.369406+0000","last_clean":"2024-04-25T00:21:57.369406+0000","last_became_active":"2024-04-25T00:19:32.874028+0000","last_became_peered":"2024-04-25T00:19:32.874028+0000","last_unstale":"2024-04-25T00:21:57.369406+0000","last_undegraded":"2024-04-25T00:21:57.369406+0000","last_fullsized":"2024-04-25T00:21:57.369406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:05.947095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569340+0000","last_change":"2024-04-25T00:19:33.888615+0000","last_active":"2024-04-25T00:21:57.569340+0000","last_peered":"2024-04-25T00:21:57.569340+0000","last_clean":"2024-04-25T00:21:57.569340+0000","last_became_active":"2024-04-25T00:19:33.888259+0000","last_became_peered":"2024-04-25T00:19:33.888259+0000","last_unstale":"2024-04-25T00:21:57.569340+0000","last_undegraded":"2024-04-25T00:21:57.569340+0000","last_fullsized":"2024-04-25T00:21:57.569340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:48:14.347178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570576+0000","last_change":"2024-04-25T00:19:37.902634+0000","last_active":"2024-04-25T00:21:57.570576+0000","last_peered":"2024-04-25T00:21:57.570576+0000","last_clean":"2024-04-25T00:21:57.570576+0000","last_became_active":"2024-04-25T00:19:37.902206+0000","last_became_peered":"2024-04-25T00:19:37.902206+0000","last_unstale":"2024-04-25T00:21:57.570576+0000","last_undegraded":"2024-04-25T00:21:57.570576+0000","last_fullsized":"2024-04-25T00:21:57.570576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:34.952433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.599346+0000","last_change":"2024-04-25T00:19:32.880844+0000","last_active":"2024-04-25T00:21:57.599346+0000","last_peered":"2024-04-25T00:21:57.599346+0000","last_clean":"2024-04-25T00:21:57.599346+0000","last_became_active":"2024-04-25T00:19:32.879574+0000","last_became_peered":"2024-04-25T00:19:32.879574+0000","last_unstale":"2024-04-25T00:21:57.599346+0000","last_undegraded":"2024-04-25T00:21:57.599346+0000","last_fullsized":"2024-04-25T00:21:57.599346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:32.381679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569431+0000","last_change":"2024-04-25T00:19:33.893064+0000","last_active":"2024-04-25T00:21:57.569431+0000","last_peered":"2024-04-25T00:21:57.569431+0000","last_clean":"2024-04-25T00:21:57.569431+0000","last_became_active":"2024-04-25T00:19:33.892717+0000","last_became_peered":"2024-04-25T00:19:33.892717+0000","last_unstale":"2024-04-25T00:21:57.569431+0000","last_undegraded":"2024-04-25T00:21:57.569431+0000","last_fullsized":"2024-04-25T00:21:57.569431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:20:47.703224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479319+0000","last_change":"2024-04-25T00:19:37.904749+0000","last_active":"2024-04-25T00:21:57.479319+0000","last_peered":"2024-04-25T00:21:57.479319+0000","last_clean":"2024-04-25T00:21:57.479319+0000","last_became_active":"2024-04-25T00:19:37.904273+0000","last_became_peered":"2024-04-25T00:19:37.904273+0000","last_unstale":"2024-04-25T00:21:57.479319+0000","last_undegraded":"2024-04-25T00:21:57.479319+0000","last_fullsized":"2024-04-25T00:21:57.479319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:36.880568+0000","last_clean_scrub_stamp":"2024-04-25T00:19:36.880568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:30.596420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571948+0000","last_change":"2024-04-25T00:19:32.872207+0000","last_active":"2024-04-25T00:21:57.571948+0000","last_peered":"2024-04-25T00:21:57.571948+0000","last_clean":"2024-04-25T00:21:57.571948+0000","last_became_active":"2024-04-25T00:19:32.871857+0000","last_became_peered":"2024-04-25T00:19:32.871857+0000","last_unstale":"2024-04-25T00:21:57.571948+0000","last_undegraded":"2024-04-25T00:21:57.571948+0000","last_fullsized":"2024-04-25T00:21:57.571948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:31.851304+0000","last_clean_scrub_stamp":"2024-04-25T00:19:31.851304+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:13.900431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569558+0000","last_change":"2024-04-25T00:19:33.880864+0000","last_active":"2024-04-25T00:21:57.569558+0000","last_peered":"2024-04-25T00:21:57.569558+0000","last_clean":"2024-04-25T00:21:57.569558+0000","last_became_active":"2024-04-25T00:19:33.880519+0000","last_became_peered":"2024-04-25T00:19:33.880519+0000","last_unstale":"2024-04-25T00:21:57.569558+0000","last_undegraded":"2024-04-25T00:21:57.569558+0000","last_fullsized":"2024-04-25T00:21:57.569558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:08.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.568933+0000","last_change":"2024-04-25T00:19:33.877906+0000","last_active":"2024-04-25T00:21:57.568933+0000","last_peered":"2024-04-25T00:21:57.568933+0000","last_clean":"2024-04-25T00:21:57.568933+0000","last_became_active":"2024-04-25T00:19:33.877543+0000","last_became_peered":"2024-04-25T00:19:33.877543+0000","last_unstale":"2024-04-25T00:21:57.568933+0000","last_undegraded":"2024-04-25T00:21:57.568933+0000","last_fullsized":"2024-04-25T00:21:57.568933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:49.380313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369093+0000","last_change":"2024-04-25T00:19:33.879112+0000","last_active":"2024-04-25T00:21:57.369093+0000","last_peered":"2024-04-25T00:21:57.369093+0000","last_clean":"2024-04-25T00:21:57.369093+0000","last_became_active":"2024-04-25T00:19:33.878715+0000","last_became_peered":"2024-04-25T00:19:33.878715+0000","last_unstale":"2024-04-25T00:21:57.369093+0000","last_undegraded":"2024-04-25T00:21:57.369093+0000","last_fullsized":"2024-04-25T00:21:57.369093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:28:29.402698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368834+0000","last_change":"2024-04-25T00:19:33.892796+0000","last_active":"2024-04-25T00:21:57.368834+0000","last_peered":"2024-04-25T00:21:57.368834+0000","last_clean":"2024-04-25T00:21:57.368834+0000","last_became_active":"2024-04-25T00:19:33.892403+0000","last_became_peered":"2024-04-25T00:19:33.892403+0000","last_unstale":"2024-04-25T00:21:57.368834+0000","last_undegraded":"2024-04-25T00:21:57.368834+0000","last_fullsized":"2024-04-25T00:21:57.368834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:08.378132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570810+0000","last_change":"2024-04-25T00:19:33.884347+0000","last_active":"2024-04-25T00:21:57.570810+0000","last_peered":"2024-04-25T00:21:57.570810+0000","last_clean":"2024-04-25T00:21:57.570810+0000","last_became_active":"2024-04-25T00:19:33.883977+0000","last_became_peered":"2024-04-25T00:19:33.883977+0000","last_unstale":"2024-04-25T00:21:57.570810+0000","last_undegraded":"2024-04-25T00:21:57.570810+0000","last_fullsized":"2024-04-25T00:21:57.570810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:59.844715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.571459+0000","last_change":"2024-04-25T00:19:33.885150+0000","last_active":"2024-04-25T00:21:57.571459+0000","last_peered":"2024-04-25T00:21:57.571459+0000","last_clean":"2024-04-25T00:21:57.571459+0000","last_became_active":"2024-04-25T00:19:33.884430+0000","last_became_peered":"2024-04-25T00:19:33.884430+0000","last_unstale":"2024-04-25T00:21:57.571459+0000","last_undegraded":"2024-04-25T00:21:57.571459+0000","last_fullsized":"2024-04-25T00:21:57.571459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:07.080792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569203+0000","last_change":"2024-04-25T00:19:33.873722+0000","last_active":"2024-04-25T00:21:57.569203+0000","last_peered":"2024-04-25T00:21:57.569203+0000","last_clean":"2024-04-25T00:21:57.569203+0000","last_became_active":"2024-04-25T00:19:33.873219+0000","last_became_peered":"2024-04-25T00:19:33.873219+0000","last_unstale":"2024-04-25T00:21:57.569203+0000","last_undegraded":"2024-04-25T00:21:57.569203+0000","last_fullsized":"2024-04-25T00:21:57.569203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:37.777495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.369599+0000","last_change":"2024-04-25T00:19:33.884468+0000","last_active":"2024-04-25T00:21:57.369599+0000","last_peered":"2024-04-25T00:21:57.369599+0000","last_clean":"2024-04-25T00:21:57.369599+0000","last_became_active":"2024-04-25T00:19:33.884007+0000","last_became_peered":"2024-04-25T00:19:33.884007+0000","last_unstale":"2024-04-25T00:21:57.369599+0000","last_undegraded":"2024-04-25T00:21:57.369599+0000","last_fullsized":"2024-04-25T00:21:57.369599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.368190+0000","last_change":"2024-04-25T00:19:33.888637+0000","last_active":"2024-04-25T00:21:57.368190+0000","last_peered":"2024-04-25T00:21:57.368190+0000","last_clean":"2024-04-25T00:21:57.368190+0000","last_became_active":"2024-04-25T00:19:33.888009+0000","last_became_peered":"2024-04-25T00:19:33.888009+0000","last_unstale":"2024-04-25T00:21:57.368190+0000","last_undegraded":"2024-04-25T00:21:57.368190+0000","last_fullsized":"2024-04-25T00:21:57.368190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:49.724059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.479506+0000","last_change":"2024-04-25T00:19:33.878918+0000","last_active":"2024-04-25T00:21:57.479506+0000","last_peered":"2024-04-25T00:21:57.479506+0000","last_clean":"2024-04-25T00:21:57.479506+0000","last_became_active":"2024-04-25T00:19:33.878496+0000","last_became_peered":"2024-04-25T00:19:33.878496+0000","last_unstale":"2024-04-25T00:21:57.479506+0000","last_undegraded":"2024-04-25T00:21:57.479506+0000","last_fullsized":"2024-04-25T00:21:57.479506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.326493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.367262+0000","last_change":"2024-04-25T00:19:33.880210+0000","last_active":"2024-04-25T00:21:57.367262+0000","last_peered":"2024-04-25T00:21:57.367262+0000","last_clean":"2024-04-25T00:21:57.367262+0000","last_became_active":"2024-04-25T00:19:33.879030+0000","last_became_peered":"2024-04-25T00:19:33.879030+0000","last_unstale":"2024-04-25T00:21:57.367262+0000","last_undegraded":"2024-04-25T00:21:57.367262+0000","last_fullsized":"2024-04-25T00:21:57.367262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:57.587429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.555764+0000","last_change":"2024-04-25T00:19:33.876332+0000","last_active":"2024-04-25T00:21:57.555764+0000","last_peered":"2024-04-25T00:21:57.555764+0000","last_clean":"2024-04-25T00:21:57.555764+0000","last_became_active":"2024-04-25T00:19:33.876010+0000","last_became_peered":"2024-04-25T00:19:33.876010+0000","last_unstale":"2024-04-25T00:21:57.555764+0000","last_undegraded":"2024-04-25T00:21:57.555764+0000","last_fullsized":"2024-04-25T00:21:57.555764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:56:54.830540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.570598+0000","last_change":"2024-04-25T00:19:33.877024+0000","last_active":"2024-04-25T00:21:57.570598+0000","last_peered":"2024-04-25T00:21:57.570598+0000","last_clean":"2024-04-25T00:21:57.570598+0000","last_became_active":"2024-04-25T00:19:33.876635+0000","last_became_peered":"2024-04-25T00:19:33.876635+0000","last_unstale":"2024-04-25T00:21:57.570598+0000","last_undegraded":"2024-04-25T00:21:57.570598+0000","last_fullsized":"2024-04-25T00:21:57.570598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:07.817758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:21:57.569041+0000","last_change":"2024-04-25T00:20:27.046816+0000","last_active":"2024-04-25T00:21:57.569041+0000","last_peered":"2024-04-25T00:21:57.569041+0000","last_clean":"2024-04-25T00:21:57.569041+0000","last_became_active":"2024-04-25T00:20:27.045845+0000","last_became_peered":"2024-04-25T00:20:27.045845+0000","last_unstale":"2024-04-25T00:21:57.569041+0000","last_undegraded":"2024-04-25T00:21:57.569041+0000","last_fullsized":"2024-04-25T00:21:57.569041+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:19:32.852934+0000","last_clean_scrub_stamp":"2024-04-25T00:19:32.852934+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:45.465308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108492,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599296,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1299,"num_read_kb":761,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":672,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1797888,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":530611,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28200,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716952,"statfs":{"total":95995035648,"available":95966158848,"internally_reserved":0,"allocated":1212416,"data_stored":1315182,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":927181,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":923463,"data_compressed":48673,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":1127835,"data_compressed":65691,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":525373,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":592307,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":526829,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1130496,"data_stored":593758,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1049897,"data_compressed":54691,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":530370,"data_compressed":29527,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1142784,"data_stored":1254766,"data_compressed":71847,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36164,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":200163,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:22:00.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:22:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- 172.21.15.45:0/1522903976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278102230 msgr2=0x7f4278108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1522903976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278102230 0x7f4278108ac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4270009ca0 tx=0x7f427002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- 172.21.15.45:0/1522903976 shutdown_connections 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1522903976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278102230 0x7f4278108ac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1522903976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278101870 0x7f4278101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1522903976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278100680 0x7f4278100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- 172.21.15.45:0/1522903976 >> 172.21.15.45:0/1522903976 conn(0x7f42780fbbf0 msgr2=0x7f42780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- 172.21.15.45:0/1522903976 shutdown_connections 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- 172.21.15.45:0/1522903976 wait complete. 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 Processor -- start 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.793+0000 7f427d7a9640 1 -- start start 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278100680 0x7f4278104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278102230 0x7f4278103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f427810c300 con 0x7f4278101870 2024-04-25T00:22:00.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f427810c180 con 0x7f4278100680 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f427810c480 con 0x7f4278102230 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52834/0 (socket says 172.21.15.45:52834) 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 -- 172.21.15.45:0/1887858551 learned_addr learned my addr 172.21.15.45:0/1887858551 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f4276ffd640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278100680 0x7f4278104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f42777fe640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278102230 0x7f4278103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 -- 172.21.15.45:0/1887858551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278102230 msgr2=0x7f4278103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278102230 0x7f4278103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 -- 172.21.15.45:0/1887858551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278100680 msgr2=0x7f4278104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278100680 0x7f4278104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 -- 172.21.15.45:0/1887858551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4278103a20 con 0x7f4278101870 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f426ffff640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260017030 con 0x7f4278101870 2024-04-25T00:22:00.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781a1b90 con 0x7f4278101870 2024-04-25T00:22:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4260004020 con 0x7f4278101870 2024-04-25T00:22:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.794+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42600047a0 con 0x7f4278101870 2024-04-25T00:22:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.795+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781a20e0 con 0x7f4278101870 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.795+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42781a2310 con 0x7f4278101870 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.797+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f42600041d0 con 0x7f4278101870 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.797+0000 7f4274ff9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 0x7f42640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.798+0000 7f4276ffd640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 0x7f42640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.798+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f42600cf8b0 con 0x7f4278101870 2024-04-25T00:22:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.798+0000 7f4276ffd640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 0x7f42640a51c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4268005fd0 tx=0x7f426800b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:00.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.805+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f426009c170 con 0x7f4278101870 2024-04-25T00:22:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:00 smithi045 ceph-mon[29194]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:00 smithi045 ceph-mon[29194]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-25T00:22:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:00 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:00 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:00 smithi045 ceph-mon[29194]: from='client.25295 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:00.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.995+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42781a26a0 con 0x7f4278101870 2024-04-25T00:22:00.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.997+0000 7f4274ff9640 1 -- 172.21.15.45:0/1887858551 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25086 (secure 0 0 0) 0x7f426009ba70 con 0x7f4278101870 2024-04-25T00:22:00.998 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:00.998 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:57.361695+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 msgr2=0x7f42640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 0x7f42640a51c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4268005fd0 tx=0x7f426800b390 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 msgr2=0x7f4278102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000de10 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 shutdown_connections 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f42640a2d00 0x7f42640a51c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4278102230 0x7f4278103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4278101870 0x7f4278102c50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 --2- 172.21.15.45:0/1887858551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4278100680 0x7f4278104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 >> 172.21.15.45:0/1887858551 conn(0x7f42780fbbf0 msgr2=0x7f42780fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 shutdown_connections 2024-04-25T00:22:01.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:00.999+0000 7f427d7a9640 1 -- 172.21.15.45:0/1887858551 wait complete. 2024-04-25T00:22:01.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:22:01.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:22:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:00 smithi122 ceph-mon[32959]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:00 smithi122 ceph-mon[32959]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-25T00:22:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:00 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:00 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:00 smithi122 ceph-mon[32959]: from='client.25295 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:00 smithi071 ceph-mon[32667]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:00 smithi071 ceph-mon[32667]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-04-25T00:22:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:00 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:00 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:01.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:00 smithi071 ceph-mon[32667]: from='client.25295 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 -- 172.21.15.45:0/3885691153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c01074c0 msgr2=0x7f19c01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 --2- 172.21.15.45:0/3885691153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c01074c0 0x7f19c01098b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009ca0 tx=0x7f19b802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 -- 172.21.15.45:0/3885691153 shutdown_connections 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 --2- 172.21.15.45:0/3885691153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c01074c0 0x7f19c01098b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 --2- 172.21.15.45:0/3885691153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 0x7f19c0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 --2- 172.21.15.45:0/3885691153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c0102260 0x7f19c0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 -- 172.21.15.45:0/3885691153 >> 172.21.15.45:0/3885691153 conn(0x7f19c00fbbf0 msgr2=0x7f19c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 -- 172.21.15.45:0/3885691153 shutdown_connections 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 -- 172.21.15.45:0/3885691153 wait complete. 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.135+0000 7f19c5bce640 1 Processor -- start 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 -- start start 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 0x7f19c0100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 0x7f19c0100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010d420 con 0x7f19c01074c0 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010d2a0 con 0x7f19c0104b90 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19c5bce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010d5a0 con 0x7f19c0102260 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 0x7f19c0100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:36218/0 (socket says 172.21.15.45:36218) 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 -- 172.21.15.45:0/2842662539 learned_addr learned my addr 172.21.15.45:0/2842662539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19beffd640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 0x7f19c0100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 -- 172.21.15.45:0/2842662539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 msgr2=0x7f19c0100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 0x7f19c0100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 -- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 msgr2=0x7f19c0100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 0x7f19c0100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bf7fe640 1 -- 172.21.15.45:0/2842662539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c0100fe0 con 0x7f19c0102260 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.136+0000 7f19bffff640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 0x7f19c0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19bf7fe640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19b400ea60 tx=0x7f19b400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b400cde0 con 0x7f19c0102260 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b40035f0 con 0x7f19c0102260 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4010670 con 0x7f19c0102260 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c01a1bf0 con 0x7f19c0102260 2024-04-25T00:22:01.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.137+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c01a20f0 con 0x7f19c0102260 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.138+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c00fd6e0 con 0x7f19c0102260 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.139+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f19b40037a0 con 0x7f19c0102260 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.140+0000 7f19bcff9640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 0x7f19a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.140+0000 7f19beffd640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 0x7f19a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.140+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f19b40cfb90 con 0x7f19c0102260 2024-04-25T00:22:01.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.140+0000 7f19beffd640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 0x7f19a80a51c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f19c0101940 tx=0x7f19b0005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.143+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f19b409c4e0 con 0x7f19c0102260 2024-04-25T00:22:01.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.336+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19c002d3e0 con 0x7f19c0102260 2024-04-25T00:22:01.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.337+0000 7f19bcff9640 1 -- 172.21.15.45:0/2842662539 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f19b409bde0 con 0x7f19c0102260 2024-04-25T00:22:01.337 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 msgr2=0x7f19a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 0x7f19a80a51c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f19c0101940 tx=0x7f19b0005f90 comp rx=0 tx=0).stop 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 msgr2=0x7f19c0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19b400ea60 tx=0x7f19b400ef20 comp rx=0 tx=0).stop 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 shutdown_connections 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f19a80a2d00 0x7f19a80a51c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c01074c0 0x7f19c0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c0104b90 0x7f19c0100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 --2- 172.21.15.45:0/2842662539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19c0102260 0x7f19c0101bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 >> 172.21.15.45:0/2842662539 conn(0x7f19c00fbbf0 msgr2=0x7f19c0102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 shutdown_connections 2024-04-25T00:22:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.340+0000 7f19c5bce640 1 -- 172.21.15.45:0/2842662539 wait complete. 2024-04-25T00:22:01.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 -- 172.21.15.45:0/103251980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134102220 msgr2=0x7fd134104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/103251980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134102220 0x7fd134104610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd130009900 tx=0x7fd13002a410 comp rx=0 tx=0).stop 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 -- 172.21.15.45:0/103251980 shutdown_connections 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/103251980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134107480 0x7fd1341098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/103251980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134104b50 0x7fd134106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/103251980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134102220 0x7fd134104610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 -- 172.21.15.45:0/103251980 >> 172.21.15.45:0/103251980 conn(0x7fd1340fbc30 msgr2=0x7fd1340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 -- 172.21.15.45:0/103251980 shutdown_connections 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.461+0000 7fd13c2b3640 1 -- 172.21.15.45:0/103251980 wait complete. 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 Processor -- start 2024-04-25T00:22:01.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 -- start start 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134107480 0x7fd134100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410d450 con 0x7fd134102220 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410d2d0 con 0x7fd134104b50 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13c2b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410d5d0 con 0x7fd134107480 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13a04f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13984e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13984e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:51014/0 (socket says 172.21.15.45:51014) 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13a04f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52876/0 (socket says 172.21.15.45:52876) 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13984e640 1 -- 172.21.15.45:0/1845912105 learned_addr learned my addr 172.21.15.45:0/1845912105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.462+0000 7fd13a850640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134107480 0x7fd134100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 -- 172.21.15.45:0/1845912105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134107480 msgr2=0x7fd134100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134107480 0x7fd134100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 -- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 msgr2=0x7fd134101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 -- 172.21.15.45:0/1845912105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd134100fa0 con 0x7fd134104b50 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13a04f640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13984e640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd12400e9f0 tx=0x7fd12400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12400ccc0 con 0x7fd134104b50 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1341a1c60 con 0x7fd134104b50 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd124005500 con 0x7fd134104b50 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124010690 con 0x7fd134104b50 2024-04-25T00:22:01.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.463+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1341a21b0 con 0x7fd134104b50 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.464+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1340fd720 con 0x7fd134104b50 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.465+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fd1240046c0 con 0x7fd134104b50 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.466+0000 7fd12b7fe640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 0x7fd1200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.466+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd1240cfb30 con 0x7fd134104b50 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.466+0000 7fd13a04f640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 0x7fd1200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.466+0000 7fd13a04f640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 0x7fd1200a51c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1300314e0 tx=0x7fd130000bc0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.470+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd12409c3f0 con 0x7fd134104b50 2024-04-25T00:22:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.657+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd13402d320 con 0x7fd134104b50 2024-04-25T00:22:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.658+0000 7fd12b7fe640 1 -- 172.21.15.45:0/1845912105 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fd12409bcf0 con 0x7fd134104b50 2024-04-25T00:22:01.658 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:22:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 msgr2=0x7fd1200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 0x7fd1200a51c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1300314e0 tx=0x7fd130000bc0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 msgr2=0x7fd1341001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd12400e9f0 tx=0x7fd12400edf0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 shutdown_connections 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1200a2d00 0x7fd1200a51c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd134107480 0x7fd134100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd134104b50 0x7fd1341001d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 --2- 172.21.15.45:0/1845912105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd134102220 0x7fd134101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 >> 172.21.15.45:0/1845912105 conn(0x7fd1340fbc30 msgr2=0x7fd134102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 shutdown_connections 2024-04-25T00:22:01.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.660+0000 7fd13c2b3640 1 -- 172.21.15.45:0/1845912105 wait complete. 2024-04-25T00:22:01.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 -- 172.21.15.45:0/190360641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 msgr2=0x7fd7d8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/190360641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8101bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009900 tx=0x7fd7d002a410 comp rx=0 tx=0).stop 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 -- 172.21.15.45:0/190360641 shutdown_connections 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/190360641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/190360641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/190360641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8101bb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 -- 172.21.15.45:0/190360641 >> 172.21.15.45:0/190360641 conn(0x7fd7d80fcd20 msgr2=0x7fd7d80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 -- 172.21.15.45:0/190360641 shutdown_connections 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.789+0000 7fd7dcad6640 1 -- 172.21.15.45:0/190360641 wait complete. 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 Processor -- start 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 -- start start 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810d400 con 0x7fd7d8101790 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810d280 con 0x7fd7d8103340 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7dcad6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d810d580 con 0x7fd7d8102980 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52898/0 (socket says 172.21.15.45:52898) 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 -- 172.21.15.45:0/4075029627 learned_addr learned my addr 172.21.15.45:0/4075029627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d5d9b640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d6d9d640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 -- 172.21.15.45:0/4075029627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 msgr2=0x7fd7d8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 -- 172.21.15.45:0/4075029627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 msgr2=0x7fd7d819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d659c640 1 -- 172.21.15.45:0/4075029627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d819f720 con 0x7fd7d8101790 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d5d9b640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.790+0000 7fd7d6d9d640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7d659c640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0031610 tx=0x7fd7d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d003f030 con 0x7fd7d8101790 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d819f9b0 con 0x7fd7d8101790 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d0003ad0 con 0x7fd7d8101790 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0035070 con 0x7fd7d8101790 2024-04-25T00:22:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.791+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d819fea0 con 0x7fd7d8101790 2024-04-25T00:22:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.792+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d8108b80 con 0x7fd7d8101790 2024-04-25T00:22:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.794+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fd7d0042030 con 0x7fd7d8101790 2024-04-25T00:22:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.794+0000 7fd7c77fe640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 0x7fd7c00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.795+0000 7fd7d5d9b640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 0x7fd7c00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.795+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd7d00efea0 con 0x7fd7d8101790 2024-04-25T00:22:01.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.795+0000 7fd7d5d9b640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 0x7fd7c00a51c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8199010 tx=0x7fd7cc009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:01.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.799+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd7d00bc760 con 0x7fd7d8101790 2024-04-25T00:22:02.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:01.999+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7d802d3b0 con 0x7fd7d8101790 2024-04-25T00:22:02.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.000+0000 7fd7c77fe640 1 -- 172.21.15.45:0/4075029627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fd7d00bc060 con 0x7fd7d8101790 2024-04-25T00:22:02.000 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 msgr2=0x7fd7c00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 0x7fd7c00a51c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8199010 tx=0x7fd7cc009500 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 msgr2=0x7fd7d8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0031610 tx=0x7fd7d0004270 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 shutdown_connections 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7c00a2d00 0x7fd7c00a51c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d8103340 0x7fd7d819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8102980 0x7fd7d8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 --2- 172.21.15.45:0/4075029627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7d8101790 0x7fd7d8197b00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 >> 172.21.15.45:0/4075029627 conn(0x7fd7d80fcd20 msgr2=0x7fd7d8105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 shutdown_connections 2024-04-25T00:22:02.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.002+0000 7fd7dcad6640 1 -- 172.21.15.45:0/4075029627 wait complete. 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1887858551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:02.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:22:02.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:22:02.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:22:02.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:01 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2842662539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:22:02.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:22:02.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1887858551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:22:02.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:22:02.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:01 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2842662539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='client.25291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1887858551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:22:02.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-25T00:22:02.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:01 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2842662539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 -- 172.21.15.45:0/1179751305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 msgr2=0x7f73d4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 --2- 172.21.15.45:0/1179751305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4100a90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009930 tx=0x7f73c802a420 comp rx=0 tx=0).stop 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 -- 172.21.15.45:0/1179751305 shutdown_connections 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 --2- 172.21.15.45:0/1179751305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 0x7f73d410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 --2- 172.21.15.45:0/1179751305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4100a90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 --2- 172.21.15.45:0/1179751305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 -- 172.21.15.45:0/1179751305 >> 172.21.15.45:0/1179751305 conn(0x7f73d4077740 msgr2=0x7f73d4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 -- 172.21.15.45:0/1179751305 shutdown_connections 2024-04-25T00:22:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.133+0000 7f73d9c12640 1 -- 172.21.15.45:0/1179751305 wait complete. 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 Processor -- start 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 -- start start 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 0x7f73d41971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73d41109f0 con 0x7f73d4100610 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4110870 con 0x7f73d41066b0 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d9c12640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4110b70 con 0x7f73d4100fd0 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52914/0 (socket says 172.21.15.45:52914) 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d37fe640 1 -- 172.21.15.45:0/887609264 learned_addr learned my addr 172.21.15.45:0/887609264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d3fff640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.134+0000 7f73d2ffd640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 0x7f73d41971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 -- 172.21.15.45:0/887609264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 msgr2=0x7f73d41971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 0x7f73d41971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 -- 172.21.15.45:0/887609264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 msgr2=0x7f73d41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 -- 172.21.15.45:0/887609264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d41a64d0 con 0x7f73d4100610 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d3fff640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d37fe640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f73c400d770 tx=0x7f73c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4017070 con 0x7f73d4100610 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d41a67c0 con 0x7f73d4100610 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a6d30 con 0x7f73d4100610 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c4009c30 con 0x7f73d4100610 2024-04-25T00:22:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.135+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4020060 con 0x7f73d4100610 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.136+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c0005290 con 0x7f73d4100610 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.137+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f73c4004020 con 0x7f73d4100610 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.138+0000 7f73d0ff9640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 0x7f73bc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.138+0000 7f73d2ffd640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 0x7f73bc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.138+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f73c40cf090 con 0x7f73d4100610 2024-04-25T00:22:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.139+0000 7f73d2ffd640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 0x7f73bc0a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009900 tx=0x7f73c80058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.143+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73c409b130 con 0x7f73d4100610 2024-04-25T00:22:02.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.350+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f73c0005850 con 0x7f73d4100610 2024-04-25T00:22:02.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.350+0000 7f73d0ff9640 1 -- 172.21.15.45:0/887609264 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f73c409f430 con 0x7f73d4100610 2024-04-25T00:22:02.351 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:22:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 msgr2=0x7f73bc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 0x7f73bc0a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009900 tx=0x7f73c80058d0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 msgr2=0x7f73d4196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f73c400d770 tx=0x7f73c400dc30 comp rx=0 tx=0).stop 2024-04-25T00:22:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 shutdown_connections 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73bc0a2e10 0x7f73bc0a52d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73d41066b0 0x7f73d41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73d4100fd0 0x7f73d41971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 --2- 172.21.15.45:0/887609264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73d4100610 0x7f73d4196c60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 >> 172.21.15.45:0/887609264 conn(0x7f73d4077740 msgr2=0x7f73d40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 shutdown_connections 2024-04-25T00:22:02.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.353+0000 7f73d9c12640 1 -- 172.21.15.45:0/887609264 wait complete. 2024-04-25T00:22:02.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:22:02.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- 172.21.15.45:0/2994907836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 msgr2=0x7f41dc102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 --2- 172.21.15.45:0/2994907836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc102e00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009900 tx=0x7f41d802a480 comp rx=0 tx=0).stop 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- 172.21.15.45:0/2994907836 shutdown_connections 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 --2- 172.21.15.45:0/2994907836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 0x7f41dc10e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 --2- 172.21.15.45:0/2994907836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 0x7f41dc10b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 --2- 172.21.15.45:0/2994907836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc102e00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- 172.21.15.45:0/2994907836 >> 172.21.15.45:0/2994907836 conn(0x7f41dc0777a0 msgr2=0x7f41dc077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- 172.21.15.45:0/2994907836 shutdown_connections 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- 172.21.15.45:0/2994907836 wait complete. 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 Processor -- start 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.472+0000 7f41e3b16640 1 -- start start 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 0x7f41dc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 0x7f41dc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111b30 con 0x7f41dc102a20 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1119b0 con 0x7f41dc103340 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111cb0 con 0x7f41dc10bd60 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52924/0 (socket says 172.21.15.45:52924) 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 -- 172.21.15.45:0/1737650388 learned_addr learned my addr 172.21.15.45:0/1737650388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e10b1640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 0x7f41dc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e20b3640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 0x7f41dc19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 -- 172.21.15.45:0/1737650388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 msgr2=0x7f41dc19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 0x7f41dc19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 -- 172.21.15.45:0/1737650388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 msgr2=0x7f41dc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 0x7f41dc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 -- 172.21.15.45:0/1737650388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc19cfd0 con 0x7f41dc102a20 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e18b2640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006010 tx=0x7f41d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d803f030 con 0x7f41dc102a20 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d8004440 con 0x7f41dc102a20 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc19d260 con 0x7f41dc102a20 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8035070 con 0x7f41dc102a20 2024-04-25T00:22:02.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.473+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a4b50 con 0x7f41dc102a20 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.475+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc0689b0 con 0x7f41dc102a20 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.476+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f41d8042030 con 0x7f41dc102a20 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.476+0000 7f41d2ffd640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 0x7f41c80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.476+0000 7f41e10b1640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 0x7f41c80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.476+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f41d80efc90 con 0x7f41dc102a20 2024-04-25T00:22:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.477+0000 7f41e10b1640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 0x7f41c80a5250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41dc10daa0 tx=0x7f41cc005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.481+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f41d80bbd90 con 0x7f41dc102a20 2024-04-25T00:22:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.705+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41dc106ee0 con 0x7f41dc102a20 2024-04-25T00:22:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.706+0000 7f41d2ffd640 1 -- 172.21.15.45:0/1737650388 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f41d80c0090 con 0x7f41dc102a20 2024-04-25T00:22:02.707 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:02.707 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:22:02.707 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 msgr2=0x7f41c80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 0x7f41c80a5250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41dc10daa0 tx=0x7f41cc005f90 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 msgr2=0x7f41dc198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006010 tx=0x7f41d8004040 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 shutdown_connections 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f41c80a2d90 0x7f41c80a5250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41dc10bd60 0x7f41dc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41dc103340 0x7f41dc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 --2- 172.21.15.45:0/1737650388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc102a20 0x7f41dc198030 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 >> 172.21.15.45:0/1737650388 conn(0x7f41dc0777a0 msgr2=0x7f41dc0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 shutdown_connections 2024-04-25T00:22:02.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.708+0000 7f41e3b16640 1 -- 172.21.15.45:0/1737650388 wait complete. 2024-04-25T00:22:02.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:22:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 -- 172.21.15.45:0/945091376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625010bd60 msgr2=0x7f625010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 --2- 172.21.15.45:0/945091376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625010bd60 0x7f625010e1d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6248009ca0 tx=0x7f624802a950 comp rx=0 tx=0).stop 2024-04-25T00:22:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 -- 172.21.15.45:0/945091376 shutdown_connections 2024-04-25T00:22:02.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 --2- 172.21.15.45:0/945091376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f625010bd60 0x7f625010e1d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 --2- 172.21.15.45:0/945091376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 0x7f625010b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 --2- 172.21.15.45:0/945091376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6250102a20 0x7f6250102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 -- 172.21.15.45:0/945091376 >> 172.21.15.45:0/945091376 conn(0x7f62500777a0 msgr2=0x7f6250077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 -- 172.21.15.45:0/945091376 shutdown_connections 2024-04-25T00:22:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.822+0000 7f6255a9a640 1 -- 172.21.15.45:0/945091376 wait complete. 2024-04-25T00:22:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 Processor -- start 2024-04-25T00:22:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 -- start start 2024-04-25T00:22:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 0x7f6250198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 0x7f625019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6250111b30 con 0x7f6250102a20 2024-04-25T00:22:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62501119b0 con 0x7f625010bd60 2024-04-25T00:22:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f6255a9a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6250111cb0 con 0x7f6250103340 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52954/0 (socket says 172.21.15.45:52954) 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 -- 172.21.15.45:0/332895719 learned_addr learned my addr 172.21.15.45:0/332895719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624effd640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 0x7f6250198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624ffff640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 0x7f625019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 -- 172.21.15.45:0/332895719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 msgr2=0x7f6250198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 0x7f6250198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 -- 172.21.15.45:0/332895719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 msgr2=0x7f625019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 0x7f625019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624f7fe640 1 -- 172.21.15.45:0/332895719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625019cfd0 con 0x7f6250102a20 2024-04-25T00:22:02.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.823+0000 7f624ffff640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 0x7f625019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f624f7fe640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f624000e9f0 tx=0x7f624000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000cd80 con 0x7f6250102a20 2024-04-25T00:22:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6240010910 con 0x7f6250102a20 2024-04-25T00:22:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625019d2c0 con 0x7f6250102a20 2024-04-25T00:22:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240018990 con 0x7f6250102a20 2024-04-25T00:22:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.824+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62501a4b50 con 0x7f6250102a20 2024-04-25T00:22:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.825+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62500689b0 con 0x7f6250102a20 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.826+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6240010ac0 con 0x7f6250102a20 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.826+0000 7f624cff9640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 0x7f62380a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.827+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f62400cfcf0 con 0x7f6250102a20 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.827+0000 7f624effd640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 0x7f62380a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.827+0000 7f624effd640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 0x7f62380a5250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f625010d9f0 tx=0x7f6244005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:02.831+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62400d4070 con 0x7f6250102a20 2024-04-25T00:22:03.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.017+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62501a4d50 con 0x7f6250102a20 2024-04-25T00:22:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.019+0000 7f624cff9640 1 -- 172.21.15.45:0/332895719 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25086 (secure 0 0 0) 0x7f624009c440 con 0x7f6250102a20 2024-04-25T00:22:03.019 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:03.020 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:21:57.361695+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:22:03.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 msgr2=0x7f62380a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 0x7f62380a5250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f625010d9f0 tx=0x7f6244005f90 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 msgr2=0x7f6250198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f624000e9f0 tx=0x7f624000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 shutdown_connections 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62380a2d90 0x7f62380a5250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f625010bd60 0x7f625019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6250103340 0x7f6250198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 --2- 172.21.15.45:0/332895719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6250102a20 0x7f6250198030 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 >> 172.21.15.45:0/332895719 conn(0x7f62500777a0 msgr2=0x7f62500fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 shutdown_connections 2024-04-25T00:22:03.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.021+0000 7f6255a9a640 1 -- 172.21.15.45:0/332895719 wait complete. 2024-04-25T00:22:03.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1845912105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4075029627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.032 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/887609264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.035 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T00:22:03.035 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:22:03.047 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:22:03.048 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:22:03.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T00:22:03.048 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T00:22:03.049 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:22:03.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:22:03.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1845912105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4075029627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:03.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:03.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:03.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/887609264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1845912105' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: Updating smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:22:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4075029627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:22:03.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/887609264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:22:03.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 -- 172.21.15.45:0/2213562229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0106630 msgr2=0x7ffbb0106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 --2- 172.21.15.45:0/2213562229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0106630 0x7ffbb0106a10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009930 tx=0x7ffba002a420 comp rx=0 tx=0).stop 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 -- 172.21.15.45:0/2213562229 shutdown_connections 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 --2- 172.21.15.45:0/2213562229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 --2- 172.21.15.45:0/2213562229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0100670 0x7ffbb0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 --2- 172.21.15.45:0/2213562229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0106630 0x7ffbb0106a10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 -- 172.21.15.45:0/2213562229 >> 172.21.15.45:0/2213562229 conn(0x7ffbb00777c0 msgr2=0x7ffbb0077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 -- 172.21.15.45:0/2213562229 shutdown_connections 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.154+0000 7ffbb5977640 1 -- 172.21.15.45:0/2213562229 wait complete. 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.155+0000 7ffbb5977640 1 Processor -- start 2024-04-25T00:22:03.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.155+0000 7ffbb5977640 1 -- start start 2024-04-25T00:22:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.155+0000 7f2e217e9640 1 -- 172.21.15.45:0/2329477821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c1066b0 msgr2=0x7f2e1c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.155+0000 7f2e217e9640 1 --2- 172.21.15.45:0/2329477821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c106a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009900 tx=0x7f2e0c02a410 comp rx=0 tx=0).stop 2024-04-25T00:22:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0106630 0x7ffbb019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0111b40 con 0x7ffbb0100670 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb01119c0 con 0x7ffbb0106630 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb5977640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0111cc0 con 0x7ffbb0101030 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 -- 172.21.15.45:0/2329477821 shutdown_connections 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 --2- 172.21.15.45:0/2329477821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c100fd0 0x7f2e1c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 --2- 172.21.15.45:0/2329477821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100610 0x7f2e1c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 --2- 172.21.15.45:0/2329477821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c106a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 -- 172.21.15.45:0/2329477821 >> 172.21.15.45:0/2329477821 conn(0x7f2e1c077740 msgr2=0x7f2e1c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 -- 172.21.15.45:0/2329477821 shutdown_connections 2024-04-25T00:22:03.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 -- 172.21.15.45:0/2329477821 wait complete. 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53002/0 (socket says 172.21.15.45:53002) 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbaeffd640 1 -- 172.21.15.45:0/1835846911 learned_addr learned my addr 172.21.15.45:0/1835846911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbaf7fe640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0106630 0x7ffbb019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 -- 172.21.15.45:0/1835846911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0106630 msgr2=0x7ffbb019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0106630 0x7ffbb019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 -- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 msgr2=0x7ffbb0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 -- 172.21.15.45:0/1835846911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb019cfa0 con 0x7ffbb0101030 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbaeffd640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:03.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbae7fc640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb0198540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d6d0 tx=0x7ffba400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7f2e217e9640 1 Processor -- start 2024-04-25T00:22:03.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.156+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4017070 con 0x7ffbb0101030 2024-04-25T00:22:03.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb019d290 con 0x7ffbb0101030 2024-04-25T00:22:03.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba4009820 con 0x7ffbb0101030 2024-04-25T00:22:03.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4004d00 con 0x7ffbb0101030 2024-04-25T00:22:03.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb01a4b20 con 0x7ffbb0101030 2024-04-25T00:22:03.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 -- start start 2024-04-25T00:22:03.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 0x7f2e1c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100fd0 0x7f2e1c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c110a30 con 0x7f2e1c1066b0 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1108b0 con 0x7f2e1c100610 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e217e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c110bb0 con 0x7f2e1c100fd0 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 0x7f2e1c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53006/0 (socket says 172.21.15.45:53006) 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100fd0 0x7f2e1c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/4185997555 learned_addr learned my addr 172.21.15.45:0/4185997555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/4185997555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100fd0 msgr2=0x7f2e1c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100fd0 0x7f2e1c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/4185997555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 msgr2=0x7f2e1c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 0x7f2e1c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/4185997555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c19cfb0 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.157+0000 7f2e1affd640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 0x7f2e1c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400eb10 tx=0x7f2e1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e03fff640 1 -- 172.21.15.45:0/4185997555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14004b20 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e03fff640 1 -- 172.21.15.45:0/4185997555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e14004cc0 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c19d2a0 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e03fff640 1 -- 172.21.15.45:0/4185997555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14010760 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c1a4b30 con 0x7f2e1c1066b0 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.158+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb004f500 con 0x7ffbb0101030 2024-04-25T00:22:03.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.160+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ffba40099d0 con 0x7ffbb0101030 2024-04-25T00:22:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.160+0000 7f2e03fff640 1 -- 172.21.15.45:0/4185997555 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f2e14005020 con 0x7f2e1c1066b0 2024-04-25T00:22:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.160+0000 7ffbb4975640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 0x7ffb980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.160+0000 7ffbaeffd640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 0x7ffb980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.160+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ffba40cf420 con 0x7ffbb0101030 2024-04-25T00:22:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e03fff640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 0x7f2e040a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1affd640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 0x7f2e040a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e03fff640 1 -- 172.21.15.45:0/4185997555 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f2e140cfe40 con 0x7f2e1c1066b0 2024-04-25T00:22:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 Processor -- start 2024-04-25T00:22:03.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7ffbaeffd640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 0x7ffb980a5300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000c00 tx=0x7ffba0009560 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 -- start start 2024-04-25T00:22:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b78d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1affd640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 0x7f2e040a5250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c004770 tx=0x7f2e0c0058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b798010 0x56283b79a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56283b6f12c0 con 0x56283b78e980 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56283b699910 con 0x56283b798010 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e245cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56283b6ed280 con 0x56283b78d290 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53008/0 (socket says 172.21.15.45:53008) 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1affd640 1 -- 172.21.15.45:0/3728483015 learned_addr learned my addr 172.21.15.45:0/3728483015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.161+0000 7f2e1a7fc640 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b78d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b798010 0x56283b79a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 -- 172.21.15.45:0/3728483015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 msgr2=0x56283b78d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b78d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 -- 172.21.15.45:0/3728483015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b798010 msgr2=0x56283b79a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b798010 0x56283b79a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 -- 172.21.15.45:0/3728483015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283b6aafb0 con 0x56283b78e980 2024-04-25T00:22:03.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bb96a25a5119a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e017fa640 1 -- 172.21.15.45:0/3728483015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c04d070 con 0x56283b78e980 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e017fa640 1 -- 172.21.15.45:0/3728483015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c003ca0 con 0x56283b78e980 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.162+0000 7f2e017fa640 1 -- 172.21.15.45:0/3728483015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c035050 con 0x56283b78e980 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- 172.21.15.45:0/3728483015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 msgr2=0x56283b78ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c03e000 comp rx=0 tx=0).stop 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- 172.21.15.45:0/3728483015 shutdown_connections 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b798010 0x56283b79a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b78d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3728483015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b78e980 0x56283b78ccc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- 172.21.15.45:0/3728483015 >> 172.21.15.45:0/3728483015 conn(0x56283b755950 msgr2=0x56283b7897f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- 172.21.15.45:0/3728483015 shutdown_connections 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- 172.21.15.45:0/3728483015 wait complete. 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 -- start start 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.163+0000 7f2e245cc740 1 Processor -- start 2024-04-25T00:22:03.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b794fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b7954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56283b79ae40 con 0x56283b798010 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56283b6ace90 con 0x56283b78d290 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56283b79afc0 con 0x56283b78e980 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 10 client.0 _ll_register_callbacks cb 0x56283b6e8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53018/0 (socket says 172.21.15.45:53018) 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3298804108 learned_addr learned my addr 172.21.15.45:0/3298804108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b794fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1a7fc640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b7954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3298804108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 msgr2=0x56283b7954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b7954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 msgr2=0x56283b794fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b794fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3298804108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283b4cd7e0 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004af0 tx=0x7f2e1409ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68919c88bd32cc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2dfeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140a4870 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c003ca0 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f2e14005310 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.164+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e14082050 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 10 client.15753 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56283b52df70 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c003ca0 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e1409b2e0 con 0x56283b798010 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 10 client.15753 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 20 client.15753 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b7aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 4 client.15753 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283b6c55a0 con 0x56283b7a81c0 2024-04-25T00:22:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b7aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.165+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffba409b520 con 0x7ffbb0101030 2024-04-25T00:22:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.166+0000 7f2dfeff5640 10 client.15753 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.166+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b7aa620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ead9595b1f082f server_cookie=8b421c1e958216b4 in_seq=0 out_seq=0 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.166+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3298804108 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56283b6c55a0 con 0x56283b7a81c0 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.166+0000 7f2dfeff5640 10 client.15753 handle_command_reply: tid=0 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.166+0000 7f2e245cc740 1 client.15753 shutdown 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 msgr2=0x56283b795a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004af0 tx=0x7f2e1409ba90 comp rx=0 tx=0).stop 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 shutdown_connections 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b7aa620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2dfeff5640 0 client.15753 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b795a20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b7954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3298804108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b794fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 >> 172.21.15.45:0/3298804108 conn(0x56283b755950 msgr2=0x56283b78f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 shutdown_connections 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 1 -- 172.21.15.45:0/3298804108 wait complete. 2024-04-25T00:22:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.167+0000 7f2e245cc740 20 client.15753 trim_cache size 0 max 16384 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.184+0000 7f2e245cc740 1 Processor -- start 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 -- start start 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b7a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b794100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1aeae0 con 0x56283b798010 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b0620 con 0x56283b78d290 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e245cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1af1b0 con 0x56283b78e980 2024-04-25T00:22:03.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53026/0 (socket says 172.21.15.45:53026) 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3852486962 learned_addr learned my addr 172.21.15.45:0/3852486962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b7a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1a7fc640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b794100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 -- 172.21.15.45:0/3852486962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 msgr2=0x56283b794100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b794100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 -- 172.21.15.45:0/3852486962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 msgr2=0x56283b7887d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1affd640 1 -- 172.21.15.45:0/3852486962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283b789a00 con 0x56283b78d290 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.185+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b7a3d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0314e0 tx=0x7f2e0c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c1e82a61dd4405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e00ff9640 1 -- 172.21.15.45:0/3852486962 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c04d040 con 0x56283b78d290 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e00ff9640 1 -- 172.21.15.45:0/3852486962 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c062020 con 0x56283b78d290 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e00ff9640 1 -- 172.21.15.45:0/3852486962 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c035040 con 0x56283b78d290 2024-04-25T00:22:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e245cc740 1 -- 172.21.15.45:0/3852486962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 msgr2=0x56283b7a3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.186+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b7a3d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0314e0 tx=0x7f2e0c005a20 comp rx=0 tx=0).stop 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- 172.21.15.45:0/3852486962 shutdown_connections 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b7887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78e980 0x56283b794100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3852486962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78d290 0x56283b7a3d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- 172.21.15.45:0/3852486962 >> 172.21.15.45:0/3852486962 conn(0x56283b755950 msgr2=0x56283b7880c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- 172.21.15.45:0/3852486962 shutdown_connections 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- 172.21.15.45:0/3852486962 wait complete. 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- start start 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 Processor -- start 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b80bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 0x56283b80c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1aff50 con 0x56283b798010 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1af880 con 0x56283b78e980 2024-04-25T00:22:03.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1acac0 con 0x56283b78d290 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e245cc740 10 client.0 _ll_register_callbacks cb 0x7f2e140bc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53042/0 (socket says 172.21.15.45:53042) 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3951573701 learned_addr learned my addr 172.21.15.45:0/3951573701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.187+0000 7f2e1a7fc640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 0x56283b80c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b80bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3951573701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 msgr2=0x56283b80bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b80bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 msgr2=0x56283b80c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 0x56283b80c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 -- 172.21.15.45:0/3951573701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283b3c5170 con 0x56283b798010 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1a7fc640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 0x56283b80c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e1b7fe640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004e90 tx=0x7f2e140058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad2fd94d8c3487c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2dfeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140a8070 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f2e140a4670 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c062020 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e14098130 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e245cc740 10 client.15759 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56283b813150 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.188+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e0c062020 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e140ab020 con 0x56283b798010 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 10 client.15759 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 20 client.15759 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b815810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 4 client.15759 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56283b6fe800 con 0x56283b7a81c0 2024-04-25T00:22:03.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b815810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2e1affd640 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b815810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ab58a13fe2f868 server_cookie=3b781a7ba645dc25 in_seq=0 out_seq=0 2024-04-25T00:22:03.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.189+0000 7f2dfeff5640 10 client.15759 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2dfeff5640 1 -- 172.21.15.45:0/3951573701 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56283b6fe800 con 0x56283b7a81c0 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2dfeff5640 10 client.15759 handle_command_reply: tid=0 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 client.15759 shutdown 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 msgr2=0x56283b806720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004e90 tx=0x7f2e140058d0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 shutdown_connections 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56283b7a81c0 0x56283b815810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56283b798010 0x56283b806720 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56283b78e980 0x56283b80c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 --2- 172.21.15.45:0/3951573701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283b78d290 0x56283b80bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2dfeff5640 0 client.15759 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.190+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 >> 172.21.15.45:0/3951573701 conn(0x56283b755950 msgr2=0x56283b788950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 shutdown_connections 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e245cc740 1 -- 172.21.15.45:0/3951573701 wait complete. 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e245cc740 20 client.15759 trim_cache size 0 max 16384 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "629d137a-7915-489c-b380-da53c69f64cd", 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:22:03.197 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 msgr2=0x7f2e040a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 0x7f2e040a5250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c004770 tx=0x7f2e0c0058d0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 msgr2=0x7f2e1c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400eb10 tx=0x7f2e1400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 shutdown_connections 2024-04-25T00:22:03.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f2e040a2d90 0x7f2e040a5250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e1c1066b0 0x7f2e1c19c880 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e1c100fd0 0x7f2e1c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 --2- 172.21.15.45:0/4185997555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e1c100610 0x7f2e1c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 >> 172.21.15.45:0/4185997555 conn(0x7f2e1c077740 msgr2=0x7f2e1c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 shutdown_connections 2024-04-25T00:22:03.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.191+0000 7f2e217e9640 1 -- 172.21.15.45:0/4185997555 wait complete. 2024-04-25T00:22:03.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:22:03.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 -- 172.21.15.45:0/4124679730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 msgr2=0x7fc854100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 --2- 172.21.15.45:0/4124679730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc854100a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc844009930 tx=0x7fc84402a420 comp rx=0 tx=0).stop 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 -- 172.21.15.45:0/4124679730 shutdown_connections 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 --2- 172.21.15.45:0/4124679730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc854100fd0 0x7fc85410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 --2- 172.21.15.45:0/4124679730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc854100a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 --2- 172.21.15.45:0/4124679730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8541066b0 0x7fc854106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 -- 172.21.15.45:0/4124679730 >> 172.21.15.45:0/4124679730 conn(0x7fc854077740 msgr2=0x7fc854077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 -- 172.21.15.45:0/4124679730 shutdown_connections 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.307+0000 7fc85c710640 1 -- 172.21.15.45:0/4124679730 wait complete. 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 Processor -- start 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 -- start start 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854100fd0 0x7fc8541027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8541066b0 0x7fc854102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110a80 con 0x7fc854100610 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110900 con 0x7fc854100fd0 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85c710640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110c00 con 0x7fc8541066b0 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85a4ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85a4ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53064/0 (socket says 172.21.15.45:53064) 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2776975719 learned_addr learned my addr 172.21.15.45:0/2776975719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc85acad640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8541066b0 0x7fc854102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.308+0000 7fc859cab640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854100fd0 0x7fc8541027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2776975719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8541066b0 msgr2=0x7fc854102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8541066b0 0x7fc854102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2776975719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854100fd0 msgr2=0x7fc8541027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854100fd0 0x7fc8541027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2776975719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8541a64c0 con 0x7fc854100610 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9c0 tx=0x7fc85000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc84b7fe640 1 -- 172.21.15.45:0/2776975719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85000cd80 con 0x7fc854100610 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541a67b0 con 0x7fc854100610 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc84b7fe640 1 -- 172.21.15.45:0/2776975719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8500035f0 con 0x7fc854100610 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc84b7fe640 1 -- 172.21.15.45:0/2776975719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850010690 con 0x7fc854100610 2024-04-25T00:22:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.309+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a6cd0 con 0x7fc854100610 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.311+0000 7fc84b7fe640 1 -- 172.21.15.45:0/2776975719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fc8500037a0 con 0x7fc854100610 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.311+0000 7fc85f418740 1 Processor -- start 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 -- start start 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 0x55d7aa287a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287f50 0x55d7aa2920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa1ec6b0 con 0x55d7aa292620 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa194880 con 0x55d7aa287f50 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85f418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa1e8670 con 0x55d7aa287630 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc84b7fe640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 0x7fc8400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53074/0 (socket says 172.21.15.45:53074) 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 -- 172.21.15.45:0/2082185484 learned_addr learned my addr 172.21.15.45:0/2082185484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287f50 0x55d7aa2920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85acad640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 0x55d7aa287a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc85acad640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 0x7fc8400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 -- 172.21.15.45:0/2082185484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 msgr2=0x55d7aa287a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 0x55d7aa287a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 -- 172.21.15.45:0/2082185484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287f50 msgr2=0x55d7aa2920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.312+0000 7fc859cab640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287f50 0x55d7aa2920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc84b7fe640 1 -- 172.21.15.45:0/2776975719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc8500cfcd0 con 0x7fc854100610 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc859cab640 1 -- 172.21.15.45:0/2082185484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a9fca7d0 con 0x55d7aa292620 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc85acad640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 0x55d7aa287a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc859cab640 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc844009930 tx=0x7fc844005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5caaa9bc222b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc85acad640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 0x7fc8400a5250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005c40 tx=0x7fc84c009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc848ff9640 1 -- 172.21.15.45:0/2082185484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844035040 con 0x55d7aa292620 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc848ff9640 1 -- 172.21.15.45:0/2082185484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc844005520 con 0x55d7aa292620 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.313+0000 7fc848ff9640 1 -- 172.21.15.45:0/2082185484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440035f0 con 0x55d7aa292620 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- 172.21.15.45:0/2082185484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 msgr2=0x55d7aa294a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc844009930 tx=0x7fc844005de0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- 172.21.15.45:0/2082185484 shutdown_connections 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa294a10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287f50 0x55d7aa2920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- 172.21.15.45:0/2082185484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287630 0x55d7aa287a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- 172.21.15.45:0/2082185484 >> 172.21.15.45:0/2082185484 conn(0x55d7aa283950 msgr2=0x55d7aa283d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- 172.21.15.45:0/2082185484 shutdown_connections 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- 172.21.15.45:0/2082185484 wait complete. 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- start start 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 Processor -- start 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa28fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa292620 0x55d7aa290000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa2953b0 con 0x55d7aa287630 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa1a7f50 con 0x55d7aa292620 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.314+0000 7fc85f418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7aa295530 con 0x55d7aa287f50 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85f418740 10 client.0 _ll_register_callbacks cb 0x55d7aa1e3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53084/0 (socket says 172.21.15.45:53084) 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2664651919 learned_addr learned my addr 172.21.15.45:0/2664651919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85acad640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa292620 0x55d7aa290000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc859cab640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa28fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2664651919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 msgr2=0x55d7aa28fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa28fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa292620 msgr2=0x55d7aa290000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa292620 0x55d7aa290000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 -- 172.21.15.45:0/2664651919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a9f003f0 con 0x55d7aa287630 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c6a0 tx=0x7fc85009b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4342920901fa1d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500cd4b0 con 0x55d7aa287630 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc85009b170 con 0x55d7aa287630 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc844005520 con 0x55d7aa287630 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8500a00d0 con 0x55d7aa287630 2024-04-25T00:22:03.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85f418740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.315+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7a9ec00f0 con 0x55d7aa287630 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc844005520 con 0x55d7aa287630 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc8500a1260 con 0x55d7aa287630 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 20 client.15771 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa2a2870 0x55d7aa2a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 4 client.15771 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7aa1f9bf0 con 0x55d7aa2a2870 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.316+0000 7fc859cab640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa2a2870 0x55d7aa2a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc859cab640 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa2a2870 0x55d7aa2a4cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca308139445b288 server_cookie=d6e4d9d039a014b0 in_seq=0 out_seq=0 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc83dffb640 10 client.15771 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc83dffb640 1 -- 172.21.15.45:0/2664651919 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7aa1f9bf0 con 0x55d7aa2a2870 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc83dffb640 10 client.15771 handle_command_reply: tid=0 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 client.15771 shutdown 2024-04-25T00:22:03.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 msgr2=0x55d7aa28f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c6a0 tx=0x7fc85009b970 comp rx=0 tx=0).stop 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 shutdown_connections 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa292620 0x55d7aa290000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa28fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa2a2870 0x55d7aa2a4cc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc83dffb640 0 client.15771 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 --2- 172.21.15.45:0/2664651919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287630 0x55d7aa28f580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 >> 172.21.15.45:0/2664651919 conn(0x55d7aa283950 msgr2=0x55d7aa289750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 shutdown_connections 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.317+0000 7fc85f418740 1 -- 172.21.15.45:0/2664651919 wait complete. 2024-04-25T00:22:03.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.318+0000 7fc85f418740 20 client.15771 trim_cache size 0 max 16384 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.335+0000 7fc85f418740 1 Processor -- start 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 -- start start 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa29e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa291c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b0d00 con 0x55d7aa292620 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2840 con 0x55d7aa287630 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85f418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b13d0 con 0x55d7aa287f50 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53086/0 (socket says 172.21.15.45:53086) 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 -- 172.21.15.45:0/261638915 learned_addr learned my addr 172.21.15.45:0/261638915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa29e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 -- 172.21.15.45:0/261638915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 msgr2=0x55d7aa291c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc859cab640 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa291c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa291c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 -- 172.21.15.45:0/261638915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 msgr2=0x55d7aa29e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa29e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 -- 172.21.15.45:0/261638915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7aa288450 con 0x55d7aa292620 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.336+0000 7fc85acad640 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009270 tx=0x7fc84c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5605379e08f94c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc83ffff640 1 -- 172.21.15.45:0/261638915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c003e10 con 0x55d7aa292620 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc83ffff640 1 -- 172.21.15.45:0/261638915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c00f9c0 con 0x55d7aa292620 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc83ffff640 1 -- 172.21.15.45:0/261638915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c01bc10 con 0x55d7aa292620 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 -- 172.21.15.45:0/261638915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 msgr2=0x55d7aa28e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009270 tx=0x7fc84c017000 comp rx=0 tx=0).stop 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 -- 172.21.15.45:0/261638915 shutdown_connections 2024-04-25T00:22:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa292620 0x55d7aa28e4e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa287f50 0x55d7aa291c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 --2- 172.21.15.45:0/261638915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa29e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 -- 172.21.15.45:0/261638915 >> 172.21.15.45:0/261638915 conn(0x55d7aa283950 msgr2=0x55d7aa282af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 -- 172.21.15.45:0/261638915 shutdown_connections 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.337+0000 7fc85f418740 1 -- 172.21.15.45:0/261638915 wait complete. 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 -- start start 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 Processor -- start 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa28c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa292620 0x55d7aa307e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2170 con 0x55d7aa287f50 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1aa0 con 0x55d7aa287630 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541aece0 con 0x55d7aa292620 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85f418740 10 client.0 _ll_register_callbacks cb 0x7fc850003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53098/0 (socket says 172.21.15.45:53098) 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 -- 172.21.15.45:0/4057256549 learned_addr learned my addr 172.21.15.45:0/4057256549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa28c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc85acad640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa292620 0x55d7aa307e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 -- 172.21.15.45:0/4057256549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa292620 msgr2=0x55d7aa307e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa292620 0x55d7aa307e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 -- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 msgr2=0x55d7aa28c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa28c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.338+0000 7fc859cab640 1 -- 172.21.15.45:0/4057256549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7aa28d380 con 0x55d7aa287f50 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc859cab640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc844009900 tx=0x7fc844005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e119e241527946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844035030 con 0x55d7aa287f50 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc844004180 con 0x55d7aa287f50 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84c00f9c0 con 0x55d7aa287f50 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8440043c0 con 0x55d7aa287f50 2024-04-25T00:22:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 10 client.15777 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7aa309500 con 0x55d7aa287f50 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c00f9c0 con 0x55d7aa287f50 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc844031640 con 0x55d7aa287f50 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 10 client.15777 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 20 client.15777 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa3099a0 0x55d7aa30bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 4 client.15777 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.339+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7aa1f3f40 con 0x55d7aa3099a0 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa3099a0 0x55d7aa30bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc83dffb640 10 client.15777 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85a4ac640 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa3099a0 0x55d7aa30bd90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fa9a08eb46f21a server_cookie=1248c855878dfd9f in_seq=0 out_seq=0 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc83dffb640 1 -- 172.21.15.45:0/4057256549 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d7aa1f3f40 con 0x55d7aa3099a0 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc83dffb640 10 client.15777 handle_command_reply: tid=0 2024-04-25T00:22:03.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 client.15777 shutdown 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 msgr2=0x55d7aa28caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc844009900 tx=0x7fc844005b50 comp rx=0 tx=0).stop 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 shutdown_connections 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7aa292620 0x55d7aa307e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d7aa3099a0 0x55d7aa30bd90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7aa287f50 0x55d7aa28caf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 --2- 172.21.15.45:0/4057256549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aa287630 0x55d7aa28c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 >> 172.21.15.45:0/4057256549 conn(0x55d7aa283950 msgr2=0x55d7aa288d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc83dffb640 0 client.15777 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 shutdown_connections 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 1 -- 172.21.15.45:0/4057256549 wait complete. 2024-04-25T00:22:03.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.340+0000 7fc85f418740 20 client.15777 trim_cache size 0 max 16384 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "629d137a-7915-489c-b380-da53c69f64cd": { 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "629d137a-7915-489c-b380-da53c69f64cd", 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:22:03.348 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 msgr2=0x7fc8400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 0x7fc8400a5250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005c40 tx=0x7fc84c009500 comp rx=0 tx=0).stop 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 msgr2=0x7fc8541022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9c0 tx=0x7fc85000ee80 comp rx=0 tx=0).stop 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 shutdown_connections 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc8400a2d90 0x7fc8400a5250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8541066b0 0x7fc854102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854100fd0 0x7fc8541027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 --2- 172.21.15.45:0/2776975719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc854100610 0x7fc8541022b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 >> 172.21.15.45:0/2776975719 conn(0x7fc854077740 msgr2=0x7fc8540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 shutdown_connections 2024-04-25T00:22:03.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.341+0000 7fc85c710640 1 -- 172.21.15.45:0/2776975719 wait complete. 2024-04-25T00:22:03.356 INFO:tasks.cephfs.filesystem:scrub status for tag:629d137a-7915-489c-b380-da53c69f64cd - {'path': '/', 'tag': '629d137a-7915-489c-b380-da53c69f64cd', 'options': 'recursive,force'} 2024-04-25T00:22:03.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.388+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffbb0061b80 con 0x7ffb980a2e40 2024-04-25T00:22:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.449+0000 7ffbb4975640 1 -- 172.21.15.45:0/1835846911 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffbb0061b80 con 0x7ffb980a2e40 2024-04-25T00:22:03.450 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2024-04-25T00:22:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.451+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 msgr2=0x7ffb980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.451+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 0x7ffb980a5300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000c00 tx=0x7ffba0009560 comp rx=0 tx=0).stop 2024-04-25T00:22:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.451+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 msgr2=0x7ffbb0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.451+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb0198540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d6d0 tx=0x7ffba400db90 comp rx=0 tx=0).stop 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 shutdown_connections 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb980a2e40 0x7ffb980a5300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb0106630 0x7ffbb019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbb0101030 0x7ffbb0198540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 --2- 172.21.15.45:0/1835846911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb0100670 0x7ffbb0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 >> 172.21.15.45:0/1835846911 conn(0x7ffbb00777c0 msgr2=0x7ffbb0103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 shutdown_connections 2024-04-25T00:22:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.452+0000 7ffbb5977640 1 -- 172.21.15.45:0/1835846911 wait complete. 2024-04-25T00:22:03.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- 172.21.15.45:0/507163349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 msgr2=0x7f341810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 --2- 172.21.15.45:0/507163349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f341810e4d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3410009cd0 tx=0x7f341002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- 172.21.15.45:0/507163349 shutdown_connections 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 --2- 172.21.15.45:0/507163349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f341810e4d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 --2- 172.21.15.45:0/507163349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f34180ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 --2- 172.21.15.45:0/507163349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 0x7f3418106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- 172.21.15.45:0/507163349 >> 172.21.15.45:0/507163349 conn(0x7f3418076740 msgr2=0x7f34180fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- 172.21.15.45:0/507163349 shutdown_connections 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- 172.21.15.45:0/507163349 wait complete. 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 Processor -- start 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.608+0000 7f341e548640 1 -- start start 2024-04-25T00:22:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f3418197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 0x7f341819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3418111b30 con 0x7f34181001d0 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34181119b0 con 0x7f3418106860 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341e548640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3418111cb0 con 0x7f34180ff810 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53118/0 (socket says 172.21.15.45:53118) 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 -- 172.21.15.45:0/4136502736 learned_addr learned my addr 172.21.15.45:0/4136502736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f341cae5640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 0x7f341819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f3417fff640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f3418197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 -- 172.21.15.45:0/4136502736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 msgr2=0x7f3418197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f3418197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 -- 172.21.15.45:0/4136502736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 msgr2=0x7f341819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 0x7f341819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 -- 172.21.15.45:0/4136502736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341819cea0 con 0x7f34181001d0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f3417fff640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f3418197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34177fe640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f340000d770 tx=0x7f340000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.609+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400017070 con 0x7f34181001d0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.610+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400005500 con 0x7f34181001d0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.610+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341819d190 con 0x7f34181001d0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.610+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400004e00 con 0x7f34181001d0 2024-04-25T00:22:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.610+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181a4a20 con 0x7f34181001d0 2024-04-25T00:22:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.610+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341804f500 con 0x7f34181001d0 2024-04-25T00:22:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.612+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3400004fa0 con 0x7f34181001d0 2024-04-25T00:22:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.612+0000 7f34157fa640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 0x7f34040a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.612+0000 7f3417fff640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 0x7f34040a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.612+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f34000cf3d0 con 0x7f34181001d0 2024-04-25T00:22:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.613+0000 7f3417fff640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 0x7f34040a5300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3408005d00 tx=0x7f3408005c70 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:03.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.617+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f340009bb20 con 0x7f34181001d0 2024-04-25T00:22:03.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.824+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f3418105e20 con 0x7f34040a2e40 2024-04-25T00:22:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.883+0000 7f34157fa640 1 -- 172.21.15.45:0/4136502736 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f3418105e20 con 0x7f34040a2e40 2024-04-25T00:22:03.884 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 msgr2=0x7f34040a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 0x7f34040a5300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3408005d00 tx=0x7f3408005c70 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 msgr2=0x7f34181983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f340000d770 tx=0x7f340000dc30 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 shutdown_connections 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34040a2e40 0x7f34040a5300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3418106860 0x7f341819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34181001d0 0x7f34181983f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 --2- 172.21.15.45:0/4136502736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34180ff810 0x7f3418197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 >> 172.21.15.45:0/4136502736 conn(0x7f3418076740 msgr2=0x7f3418105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 shutdown_connections 2024-04-25T00:22:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.885+0000 7f341e548640 1 -- 172.21.15.45:0/4136502736 wait complete. 2024-04-25T00:22:03.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1737650388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/332895719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:22:03.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:03 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:04.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 -- 172.21.15.45:0/4116714756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 msgr2=0x7f3d6010e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 --2- 172.21.15.45:0/4116714756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d6010e1c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009ca0 tx=0x7f3d5802a950 comp rx=0 tx=0).stop 2024-04-25T00:22:04.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 -- 172.21.15.45:0/4116714756 shutdown_connections 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 --2- 172.21.15.45:0/4116714756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d6010e1c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 --2- 172.21.15.45:0/4116714756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d6010b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 --2- 172.21.15.45:0/4116714756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 0x7f3d60102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 -- 172.21.15.45:0/4116714756 >> 172.21.15.45:0/4116714756 conn(0x7f3d600777c0 msgr2=0x7f3d60077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 -- 172.21.15.45:0/4116714756 shutdown_connections 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:03.999+0000 7f3d6715a640 1 -- 172.21.15.45:0/4116714756 wait complete. 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 Processor -- start 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 -- start start 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 0x7f3d601005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d60100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60111b20 con 0x7f3d6010bd40 2024-04-25T00:22:04.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601119a0 con 0x7f3d60102970 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d6715a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60111ca0 con 0x7f3d60103290 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52018/0 (socket says 172.21.15.45:52018) 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 -- 172.21.15.45:0/554797599 learned_addr learned my addr 172.21.15.45:0/554797599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d64ef6640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 0x7f3d601005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d5ffff640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d60100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 -- 172.21.15.45:0/554797599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 msgr2=0x7f3d60100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d60100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 -- 172.21.15.45:0/554797599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 msgr2=0x7f3d601005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 0x7f3d601005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d656f7640 1 -- 172.21.15.45:0/554797599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601a6440 con 0x7f3d6010bd40 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.000+0000 7f3d5ffff640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d60100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d656f7640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3d5802ae70 tx=0x7f3d5802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58034040 con 0x7f3d6010bd40 2024-04-25T00:22:04.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d58030920 con 0x7f3d6010bd40 2024-04-25T00:22:04.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601a66d0 con 0x7f3d6010bd40 2024-04-25T00:22:04.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58045480 con 0x7f3d6010bd40 2024-04-25T00:22:04.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.001+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a6c10 con 0x7f3d6010bd40 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.002+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6004f500 con 0x7f3d6010bd40 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.003+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3d58037020 con 0x7f3d6010bd40 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.004+0000 7f3d5dffb640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 0x7f3d500a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.004+0000 7f3d64ef6640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 0x7f3d500a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.004+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f3d580f09b0 con 0x7f3d6010bd40 2024-04-25T00:22:04.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.004+0000 7f3d64ef6640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 0x7f3d500a53b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005fd0 tx=0x7f3d54005850 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.007+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3d58080740 con 0x7f3d6010bd40 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1737650388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/332895719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:22:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:03 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:04.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1737650388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/332895719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:22:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:22:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:22:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:03 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:04.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.213+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3d60061b80 con 0x7f3d500a2ef0 2024-04-25T00:22:04.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.220+0000 7f3d5dffb640 1 -- 172.21.15.45:0/554797599 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3d60061b80 con 0x7f3d500a2ef0 2024-04-25T00:22:04.221 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T00:22:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 msgr2=0x7f3d500a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 0x7f3d500a53b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005fd0 tx=0x7f3d54005850 comp rx=0 tx=0).stop 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 msgr2=0x7f3d60101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3d5802ae70 tx=0x7f3d5802f5b0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 shutdown_connections 2024-04-25T00:22:04.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d500a2ef0 0x7f3d500a53b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6010bd40 0x7f3d60101040 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d60103290 0x7f3d60100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 --2- 172.21.15.45:0/554797599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d60102970 0x7f3d601005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 >> 172.21.15.45:0/554797599 conn(0x7f3d600777c0 msgr2=0x7f3d600fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 shutdown_connections 2024-04-25T00:22:04.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.222+0000 7f3d6715a640 1 -- 172.21.15.45:0/554797599 wait complete. 2024-04-25T00:22:04.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T00:22:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 -- 172.21.15.45:0/1371077178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 msgr2=0x7fe250106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 --2- 172.21.15.45:0/1371077178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe250106a50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009930 tx=0x7fe24c02a420 comp rx=0 tx=0).stop 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 -- 172.21.15.45:0/1371077178 shutdown_connections 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 --2- 172.21.15.45:0/1371077178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe25010d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 --2- 172.21.15.45:0/1371077178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 0x7fe250100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 --2- 172.21.15.45:0/1371077178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe250106a50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 -- 172.21.15.45:0/1371077178 >> 172.21.15.45:0/1371077178 conn(0x7fe2500769c0 msgr2=0x7fe250076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 -- 172.21.15.45:0/1371077178 shutdown_connections 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.378+0000 7fe2573ee640 1 -- 172.21.15.45:0/1371077178 wait complete. 2024-04-25T00:22:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 Processor -- start 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 -- start start 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 0x7fe250198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe2501992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe250110a40 con 0x7fe250106670 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501108c0 con 0x7fe250101020 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe2573ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe250110bc0 con 0x7fe250100660 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52036/0 (socket says 172.21.15.45:52036) 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 -- 172.21.15.45:0/555065285 learned_addr learned my addr 172.21.15.45:0/555065285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe254989640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe2501992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25518a640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 0x7fe250198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 -- 172.21.15.45:0/555065285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 msgr2=0x7fe250198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 0x7fe250198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 -- 172.21.15.45:0/555065285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 msgr2=0x7fe2501992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe2501992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe25598b640 1 -- 172.21.15.45:0/555065285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2501a4320 con 0x7fe250106670 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.379+0000 7fe254989640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe2501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe25598b640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eaa0 tx=0x7fe24800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800cdb0 con 0x7fe250106670 2024-04-25T00:22:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe248005020 con 0x7fe250106670 2024-04-25T00:22:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2501a44f0 con 0x7fe250106670 2024-04-25T00:22:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480116b0 con 0x7fe250106670 2024-04-25T00:22:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.380+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a49c0 con 0x7fe250106670 2024-04-25T00:22:04.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.381+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25004f500 con 0x7fe250106670 2024-04-25T00:22:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.382+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fe248005560 con 0x7fe250106670 2024-04-25T00:22:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.383+0000 7fe2467fc640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 0x7fe23c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.383+0000 7fe25518a640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 0x7fe23c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.383+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe2480cfc70 con 0x7fe250106670 2024-04-25T00:22:04.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.383+0000 7fe25518a640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 0x7fe23c0a5300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000c00 tx=0x7fe24c005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:04.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.387+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe248016030 con 0x7fe250106670 2024-04-25T00:22:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.593+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe2500fea10 con 0x7fe23c0a2e40 2024-04-25T00:22:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.600+0000 7fe2467fc640 1 -- 172.21.15.45:0/555065285 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe2500fea10 con 0x7fe23c0a2e40 2024-04-25T00:22:04.601 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 msgr2=0x7fe23c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 0x7fe23c0a5300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe24c000c00 tx=0x7fe24c005a20 comp rx=0 tx=0).stop 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 msgr2=0x7fe2501a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe24800eaa0 tx=0x7fe24800ef60 comp rx=0 tx=0).stop 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 shutdown_connections 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe23c0a2e40 0x7fe23c0a5300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe250106670 0x7fe2501a3de0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe250101020 0x7fe2501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 --2- 172.21.15.45:0/555065285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe250100660 0x7fe250198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:04.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 >> 172.21.15.45:0/555065285 conn(0x7fe2500769c0 msgr2=0x7fe2500fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:04.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 shutdown_connections 2024-04-25T00:22:04.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:04.602+0000 7fe2573ee640 1 -- 172.21.15.45:0/555065285 wait complete. 2024-04-25T00:22:04.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T00:22:04.640 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T00:22:04.650 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T00:22:04.674 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:22:04.674 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T00:22:04.674 INFO:tasks.workunit:timeout=3h 2024-04-25T00:22:04.674 INFO:tasks.workunit:cleanup=True 2024-04-25T00:22:04.675 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:22:04.696 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:22:04.696 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T00:22:04.696 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T00:22:04.696 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:22:04.697 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T00:22:04.697 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 00:20:02.075764066 +0000 2024-04-25T00:22:04.697 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 00:20:02.075764066 +0000 2024-04-25T00:22:04.697 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 00:21:54.853283094 +0000 2024-04-25T00:22:04.697 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 00:20:02.075764066 +0000 2024-04-25T00:22:04.697 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T00:22:04.697 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T00:22:04.769 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:22:04.825 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T00:22:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:04 smithi045 ceph-mon[29194]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:04 smithi045 ceph-mon[29194]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:04 smithi045 ceph-mon[29194]: from='client.15783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:04 smithi122 ceph-mon[32959]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:04 smithi122 ceph-mon[32959]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:04 smithi122 ceph-mon[32959]: from='client.15783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:05.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:04 smithi071 ceph-mon[32667]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:22:05.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:04 smithi071 ceph-mon[32667]: from='client.25339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:05.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:04 smithi071 ceph-mon[32667]: from='client.15783 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:05 smithi122 ceph-mon[32959]: from='client.15789 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:05 smithi122 ceph-mon[32959]: from='client.15795 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:05 smithi071 ceph-mon[32667]: from='client.15789 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:05 smithi071 ceph-mon[32667]: from='client.15795 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:05 smithi045 ceph-mon[29194]: from='client.15789 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:06.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:05 smithi045 ceph-mon[29194]: from='client.15795 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:22:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:06 smithi122 ceph-mon[32959]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 24 op/s 2024-04-25T00:22:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:06 smithi071 ceph-mon[32667]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 24 op/s 2024-04-25T00:22:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:06 smithi045 ceph-mon[29194]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 24 op/s 2024-04-25T00:22:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:08 smithi122 ceph-mon[32959]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-04-25T00:22:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:08 smithi071 ceph-mon[32667]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-04-25T00:22:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:08 smithi045 ceph-mon[29194]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-04-25T00:22:10.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:10 smithi045 ceph-mon[29194]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T00:22:10.921 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:10 smithi071 ceph-mon[32667]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T00:22:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:10 smithi122 ceph-mon[32959]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T00:22:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:12 smithi122 ceph-mon[32959]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:13.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:12 smithi071 ceph-mon[32667]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:13.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:12 smithi045 ceph-mon[29194]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:14 smithi122 ceph-mon[32959]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:14 smithi071 ceph-mon[32667]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:15.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:14 smithi045 ceph-mon[29194]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-25T00:22:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:16 smithi122 ceph-mon[32959]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-25T00:22:17.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:16 smithi071 ceph-mon[32667]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-25T00:22:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:16 smithi045 ceph-mon[29194]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-25T00:22:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:17 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:22:18.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:17 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:22:18.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:17 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:22:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:18 smithi122 ceph-mon[32959]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-25T00:22:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:18 smithi071 ceph-mon[32667]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-25T00:22:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:18 smithi045 ceph-mon[29194]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-25T00:22:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:20 smithi122 ceph-mon[32959]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-25T00:22:21.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:20 smithi071 ceph-mon[32667]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-25T00:22:21.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:20 smithi045 ceph-mon[29194]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-25T00:22:23.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:22 smithi122 ceph-mon[32959]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:23.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:22 smithi071 ceph-mon[32667]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:22 smithi045 ceph-mon[29194]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:24 smithi122 ceph-mon[32959]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:25.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:24 smithi071 ceph-mon[32667]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:24 smithi045 ceph-mon[29194]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:27.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:26 smithi122 ceph-mon[32959]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:26 smithi071 ceph-mon[32667]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:26 smithi045 ceph-mon[29194]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:28 smithi122 ceph-mon[32959]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:28 smithi071 ceph-mon[32667]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:28 smithi045 ceph-mon[29194]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:30.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:30 smithi045 ceph-mon[29194]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:30 smithi122 ceph-mon[32959]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:31.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:30 smithi071 ceph-mon[32667]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:32 smithi122 ceph-mon[32959]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:33.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:32 smithi071 ceph-mon[32667]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:32 smithi045 ceph-mon[29194]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:33.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:22:33.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 -- 172.21.15.45:0/1909013809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c107730 msgr2=0x7fea7c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 --2- 172.21.15.45:0/1909013809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c107730 0x7fea7c107b10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009930 tx=0x7fea6c02a420 comp rx=0 tx=0).stop 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 -- 172.21.15.45:0/1909013809 shutdown_connections 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 --2- 172.21.15.45:0/1909013809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c06a780 0x7fea7c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 --2- 172.21.15.45:0/1909013809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 0x7fea7c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 --2- 172.21.15.45:0/1909013809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c107730 0x7fea7c107b10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.477+0000 7fea80f4f640 1 -- 172.21.15.45:0/1909013809 >> 172.21.15.45:0/1909013809 conn(0x7fea7c0776a0 msgr2=0x7fea7c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.478+0000 7fea80f4f640 1 -- 172.21.15.45:0/1909013809 shutdown_connections 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.478+0000 7fea80f4f640 1 -- 172.21.15.45:0/1909013809 wait complete. 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.478+0000 7fea80f4f640 1 Processor -- start 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.478+0000 7fea80f4f640 1 -- start start 2024-04-25T00:22:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 0x7fea7c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c107730 0x7fea7c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c111b40 con 0x7fea7c06a780 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1119c0 con 0x7fea7c069dc0 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c111cc0 con 0x7fea7c107730 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59930/0 (socket says 172.21.15.45:59930) 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 -- 172.21.15.45:0/910027822 learned_addr learned my addr 172.21.15.45:0/910027822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c107730 0x7fea7c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea7a59c640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 0x7fea7c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 -- 172.21.15.45:0/910027822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c107730 msgr2=0x7fea7c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c107730 0x7fea7c19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 -- 172.21.15.45:0/910027822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 msgr2=0x7fea7c197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 0x7fea7c197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 -- 172.21.15.45:0/910027822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c19cfb0 con 0x7fea7c06a780 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea79d9b640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9f0 tx=0x7fea7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea6b7fe640 1 -- 172.21.15.45:0/910027822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7000cd80 con 0x7fea7c06a780 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c19d2a0 con 0x7fea7c06a780 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea6b7fe640 1 -- 172.21.15.45:0/910027822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea70010910 con 0x7fea7c06a780 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.479+0000 7fea6b7fe640 1 -- 172.21.15.45:0/910027822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea700188f0 con 0x7fea7c06a780 2024-04-25T00:22:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.480+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1a4b30 con 0x7fea7c06a780 2024-04-25T00:22:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.482+0000 7fea6b7fe640 1 -- 172.21.15.45:0/910027822 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fea70010ac0 con 0x7fea7c06a780 2024-04-25T00:22:33.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.482+0000 7fea83c56740 1 Processor -- start 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.482+0000 7fea83c56740 1 -- start start 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea6b7fe640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 0x7fea640a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 0x55cadea9b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 0x55cadea9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cade9f56b0 con 0x55cadea9b620 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cade99d880 con 0x55cadea90f50 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea83c56740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cade9f1670 con 0x55cadea90630 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 0x7fea640a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea6b7fe640 1 -- 172.21.15.45:0/910027822 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea700cfe10 con 0x7fea7c06a780 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 0x55cadea9b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 0x55cadea9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35228/0 (socket says 172.21.15.45:35228) 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 0x55cadea9b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:52298/0 (socket says 172.21.15.45:52298) 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 -- 172.21.15.45:0/4182558722 learned_addr learned my addr 172.21.15.45:0/4182558722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 0x55cadea9da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59936/0 (socket says 172.21.15.45:59936) 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 -- 172.21.15.45:0/4182558722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 msgr2=0x55cadea9b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 0x55cadea9b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 -- 172.21.15.45:0/4182558722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 msgr2=0x55cadea9da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 0x55cadea9da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.483+0000 7fea7a59c640 1 -- 172.21.15.45:0/4182558722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cade7d37d0 con 0x55cadea90630 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.484+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 0x7fea640a5250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74009da0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.484+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea6c02a940 tx=0x7fea6c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177683216d5c65e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.484+0000 7fea68ff9640 1 -- 172.21.15.45:0/4182558722 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c03f070 con 0x55cadea90630 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.484+0000 7fea68ff9640 1 -- 172.21.15.45:0/4182558722 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c004030 con 0x55cadea90630 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.484+0000 7fea68ff9640 1 -- 172.21.15.45:0/4182558722 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c035070 con 0x55cadea90630 2024-04-25T00:22:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- 172.21.15.45:0/4182558722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 msgr2=0x55cadea90a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea6c02a940 tx=0x7fea6c005dd0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- 172.21.15.45:0/4182558722 shutdown_connections 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea9b620 0x55cadea9da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90f50 0x55cadea9b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- 172.21.15.45:0/4182558722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea90a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- 172.21.15.45:0/4182558722 >> 172.21.15.45:0/4182558722 conn(0x55cadea8c950 msgr2=0x55cadea8cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- 172.21.15.45:0/4182558722 shutdown_connections 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- 172.21.15.45:0/4182558722 wait complete. 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- start start 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 Processor -- start 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 0x55cadea985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea9b620 0x55cadea99040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cadea9e3b0 con 0x55cadea90f50 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cade9b0f50 con 0x55cadea90630 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cadea9e530 con 0x55cadea9b620 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.485+0000 7fea83c56740 10 client.0 _ll_register_callbacks cb 0x55cade9ec000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59948/0 (socket says 172.21.15.45:59948) 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 -- 172.21.15.45:0/1388930693 learned_addr learned my addr 172.21.15.45:0/1388930693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea9b620 0x55cadea99040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea7a59c640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 0x55cadea985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 -- 172.21.15.45:0/1388930693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea9b620 msgr2=0x55cadea99040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea9b620 0x55cadea99040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 -- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 msgr2=0x55cadea985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 0x55cadea985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 -- 172.21.15.45:0/1388930693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cade7093f0 con 0x55cadea90f50 2024-04-25T00:22:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea7a59c640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 0x55cadea985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea79d9b640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea700987d0 tx=0x7fea70061470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4495715a4ea1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70066440 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea70065f90 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6c004030 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea7009be00 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea83c56740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.486+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cade6c90f0 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c004030 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea700cd420 con 0x55cadea90f50 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 20 client.15807 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeaab840 0x55cadeaadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 4 client.15807 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cadea02bf0 con 0x55cadeaab840 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.487+0000 7fea7a59c640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeaab840 0x55cadeaadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea61ffb640 10 client.15807 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea7a59c640 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeaab840 0x55cadeaadcb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2988d437065c885 server_cookie=69966be27eb1cb86 in_seq=0 out_seq=0 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea61ffb640 1 -- 172.21.15.45:0/1388930693 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cadea02bf0 con 0x55cadeaab840 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea61ffb640 10 client.15807 handle_command_reply: tid=0 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 client.15807 shutdown 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 msgr2=0x55cadea98b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea700987d0 tx=0x7fea70061470 comp rx=0 tx=0).stop 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 shutdown_connections 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea9b620 0x55cadea99040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeaab840 0x55cadeaadcb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea98b00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea83c56740 1 --2- 172.21.15.45:0/1388930693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea90630 0x55cadea985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.488+0000 7fea61ffb640 0 client.15807 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.489+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 >> 172.21.15.45:0/1388930693 conn(0x55cadea8c950 msgr2=0x55cadea92750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.489+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 shutdown_connections 2024-04-25T00:22:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.489+0000 7fea83c56740 1 -- 172.21.15.45:0/1388930693 wait complete. 2024-04-25T00:22:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.489+0000 7fea83c56740 20 client.15807 trim_cache size 0 max 16384 2024-04-25T00:22:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.506+0000 7fea83c56740 1 Processor -- start 2024-04-25T00:22:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.506+0000 7fea83c56740 1 -- start start 2024-04-25T00:22:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadeaa7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1ae9c0 con 0x55cadea90f50 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1b0500 con 0x55cadea9b620 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea83c56740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1af090 con 0x55cadea90630 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59956/0 (socket says 172.21.15.45:59956) 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 -- 172.21.15.45:0/75656849 learned_addr learned my addr 172.21.15.45:0/75656849 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea7a59c640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadeaa7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 -- 172.21.15.45:0/75656849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 msgr2=0x55cadea9acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea9acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 -- 172.21.15.45:0/75656849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 msgr2=0x55cadeaa7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadeaa7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 -- 172.21.15.45:0/75656849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cadea7e090 con 0x55cadea90f50 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadeaa7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea79d9b640 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea70098260 tx=0x7fea7009a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16ffaf4f7e1a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea63fff640 1 -- 172.21.15.45:0/75656849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7009b2a0 con 0x55cadea90f50 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea63fff640 1 -- 172.21.15.45:0/75656849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea700a01b0 con 0x55cadea90f50 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.507+0000 7fea63fff640 1 -- 172.21.15.45:0/75656849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70018da0 con 0x55cadea90f50 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- 172.21.15.45:0/75656849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 msgr2=0x55cadea97500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fea70098260 tx=0x7fea7009a310 comp rx=0 tx=0).stop 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- 172.21.15.45:0/75656849 shutdown_connections 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadeaa7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90f50 0x55cadea97500 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 --2- 172.21.15.45:0/75656849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90630 0x55cadea9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- 172.21.15.45:0/75656849 >> 172.21.15.45:0/75656849 conn(0x55cadea8c950 msgr2=0x55cadea8baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- 172.21.15.45:0/75656849 shutdown_connections 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- 172.21.15.45:0/75656849 wait complete. 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 -- start start 2024-04-25T00:22:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.508+0000 7fea83c56740 1 Processor -- start 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 0x55cadea9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadea9fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1afe30 con 0x55cadea90630 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1af760 con 0x55cadea9b620 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c1ac9a0 con 0x55cadea90f50 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 10 client.0 _ll_register_callbacks cb 0x7fea7000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59966/0 (socket says 172.21.15.45:59966) 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 -- 172.21.15.45:0/4239773355 learned_addr learned my addr 172.21.15.45:0/4239773355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadea9fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea79d9b640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 0x55cadea9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 -- 172.21.15.45:0/4239773355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 msgr2=0x55cadea9f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 0x55cadea9f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 -- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 msgr2=0x55cadea9fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadea9fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 -- 172.21.15.45:0/4239773355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cadeb10dc0 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea79d9b640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 0x55cadea9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7ad9d640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadea9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea7a59c640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea6c005ec0 tx=0x7fea6c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c258eb75ae48ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c055060 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea6c0317b0 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea700a01b0 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea6c05c050 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.509+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cadeb11fe0 con 0x55cadea90630 2024-04-25T00:22:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea700a01b0 con 0x55cadea90630 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea6c0506e0 con 0x55cadea90630 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 20 client.15813 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeb12320 0x55cadeb14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 4 client.15813 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cade9fcf40 con 0x55cadeb12320 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea79d9b640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeb12320 0x55cadeb14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea79d9b640 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeb12320 0x55cadeb14710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3f05bcd42b2520 server_cookie=217aa5abf03b4bb6 in_seq=0 out_seq=0 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.510+0000 7fea61ffb640 10 client.15813 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea61ffb640 1 -- 172.21.15.45:0/4239773355 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cade9fcf40 con 0x55cadeb12320 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea61ffb640 10 client.15813 handle_command_reply: tid=0 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 client.15813 shutdown 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 msgr2=0x55cadea9f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea6c005ec0 tx=0x7fea6c02a420 comp rx=0 tx=0).stop 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 shutdown_connections 2024-04-25T00:22:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cadea9b620 0x55cadea9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cadea90f50 0x55cadea9f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55cadeb12320 0x55cadeb14710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 --2- 172.21.15.45:0/4239773355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cadea90630 0x55cadea9f0e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea61ffb640 0 client.15813 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 >> 172.21.15.45:0/4239773355 conn(0x55cadea8c950 msgr2=0x55cadea91e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 shutdown_connections 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 1 -- 172.21.15.45:0/4239773355 wait complete. 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea83c56740 20 client.15813 trim_cache size 0 max 16384 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:22:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:22:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:22:33.519 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 msgr2=0x7fea640a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 0x7fea640a5250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74009da0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 msgr2=0x7fea7c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9f0 tx=0x7fea7000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 shutdown_connections 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea640a2d90 0x7fea640a5250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea7c107730 0x7fea7c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea7c06a780 0x7fea7c198500 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.511+0000 7fea80f4f640 1 --2- 172.21.15.45:0/910027822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea7c069dc0 0x7fea7c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.512+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 >> 172.21.15.45:0/910027822 conn(0x7fea7c0776a0 msgr2=0x7fea7c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.512+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 shutdown_connections 2024-04-25T00:22:33.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.512+0000 7fea80f4f640 1 -- 172.21.15.45:0/910027822 wait complete. 2024-04-25T00:22:33.523 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:22:33.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- 172.21.15.45:0/1215803427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c10e150 msgr2=0x7f877c110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 --2- 172.21.15.45:0/1215803427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c10e150 0x7f877c110540 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8774009ca0 tx=0x7f877402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- 172.21.15.45:0/1215803427 shutdown_connections 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 --2- 172.21.15.45:0/1215803427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c10e150 0x7f877c110540 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 --2- 172.21.15.45:0/1215803427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 0x7f877c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 --2- 172.21.15.45:0/1215803427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c1005a0 0x7f877c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- 172.21.15.45:0/1215803427 >> 172.21.15.45:0/1215803427 conn(0x7f877c0777a0 msgr2=0x7f877c077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- 172.21.15.45:0/1215803427 shutdown_connections 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- 172.21.15.45:0/1215803427 wait complete. 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 Processor -- start 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.625+0000 7f8782a29640 1 -- start start 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 0x7f877c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c10e150 0x7f877c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f877c113d60 con 0x7f877c1005a0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f877c113be0 con 0x7f877c10e150 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8782a29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f877c113ee0 con 0x7f877c100ec0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59974/0 (socket says 172.21.15.45:59974) 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 -- 172.21.15.45:0/1300700900 learned_addr learned my addr 172.21.15.45:0/1300700900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f8780fc6640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c10e150 0x7f877c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877b7fe640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 0x7f877c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 -- 172.21.15.45:0/1300700900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 msgr2=0x7f877c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 0x7f877c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 -- 172.21.15.45:0/1300700900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c10e150 msgr2=0x7f877c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c10e150 0x7f877c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 -- 172.21.15.45:0/1300700900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c19f1d0 con 0x7f877c1005a0 2024-04-25T00:22:33.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.626+0000 7f877bfff640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f876400e9f0 tx=0x7f876400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.627+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876400cd80 con 0x7f877c1005a0 2024-04-25T00:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.627+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c19f4c0 con 0x7f877c1005a0 2024-04-25T00:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.627+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8764010910 con 0x7f877c1005a0 2024-04-25T00:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.627+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764018990 con 0x7f877c1005a0 2024-04-25T00:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.627+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1a6d50 con 0x7f877c1005a0 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.628+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877c0689b0 con 0x7f877c1005a0 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.629+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8764010ac0 con 0x7f877c1005a0 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.629+0000 7f87797fa640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 0x7f87680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.629+0000 7f877b7fe640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 0x7f87680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.629+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f87640d00e0 con 0x7f877c1005a0 2024-04-25T00:22:33.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.630+0000 7f877b7fe640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 0x7f87680a5250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f876c005e00 tx=0x7f876c005c50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.633+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f876409c830 con 0x7f877c1005a0 2024-04-25T00:22:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.858+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f877c19b530 con 0x7f877c1005a0 2024-04-25T00:22:33.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.859+0000 7f87797fa640 1 -- 172.21.15.45:0/1300700900 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f876409c130 con 0x7f877c1005a0 2024-04-25T00:22:33.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:22:33.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:22:33.860 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:22:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 msgr2=0x7f87680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 0x7f87680a5250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f876c005e00 tx=0x7f876c005c50 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 msgr2=0x7f877c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f876400e9f0 tx=0x7f876400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 shutdown_connections 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87680a2d90 0x7f87680a5250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877c10e150 0x7f877c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877c100ec0 0x7f877c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 --2- 172.21.15.45:0/1300700900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f877c1005a0 0x7f877c19a1e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 >> 172.21.15.45:0/1300700900 conn(0x7f877c0777a0 msgr2=0x7f877c10f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 shutdown_connections 2024-04-25T00:22:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.861+0000 7f8782a29640 1 -- 172.21.15.45:0/1300700900 wait complete. 2024-04-25T00:22:33.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:22:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- 172.21.15.45:0/1968826716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101710 msgr2=0x7f6110101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 --2- 172.21.15.45:0/1968826716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101710 0x7f6110101b90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6100009930 tx=0x7f610002a420 comp rx=0 tx=0).stop 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- 172.21.15.45:0/1968826716 shutdown_connections 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 --2- 172.21.15.45:0/1968826716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61101020d0 0x7f611010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 --2- 172.21.15.45:0/1968826716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6110101710 0x7f6110101b90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 --2- 172.21.15.45:0/1968826716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110107720 0x7f6110107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- 172.21.15.45:0/1968826716 >> 172.21.15.45:0/1968826716 conn(0x7f6110078b90 msgr2=0x7f6110078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- 172.21.15.45:0/1968826716 shutdown_connections 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- 172.21.15.45:0/1968826716 wait complete. 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 Processor -- start 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.980+0000 7f61177ee640 1 -- start start 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 0x7f611019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 0x7f611019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6110111b30 con 0x7f61101020d0 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61101119b0 con 0x7f6110107720 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6110111cb0 con 0x7f6110101710 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39924/0 (socket says 172.21.15.45:39924) 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 -- 172.21.15.45:0/1718270227 learned_addr learned my addr 172.21.15.45:0/1718270227 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f611558a640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 0x7f611019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 0x7f611019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 -- 172.21.15.45:0/1718270227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 msgr2=0x7f611019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 0x7f611019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 -- 172.21.15.45:0/1718270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 msgr2=0x7f611019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 0x7f611019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 -- 172.21.15.45:0/1718270227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611019f290 con 0x7f61101020d0 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 0x7f611019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f611558a640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 0x7f611019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f6114d89640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6100009230 tx=0x7f6100004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61067fc640 1 -- 172.21.15.45:0/1718270227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610003f030 con 0x7f61101020d0 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61067fc640 1 -- 172.21.15.45:0/1718270227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6100003ad0 con 0x7f61101020d0 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611019f520 con 0x7f61101020d0 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.981+0000 7f61067fc640 1 -- 172.21.15.45:0/1718270227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100035070 con 0x7f61101020d0 2024-04-25T00:22:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.982+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61101a6e10 con 0x7f61101020d0 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f61067fc640 1 -- 172.21.15.45:0/1718270227 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6100042030 con 0x7f61101020d0 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 Processor -- start 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 -- start start 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f61067fc640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 0x7f60fc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee09c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee09c570 0x5605ee09e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605edff6670 con 0x5605ee09c570 2024-04-25T00:22:33.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5605edf9e970 con 0x5605ee091580 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.984+0000 7f611a5d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5605edff2630 con 0x5605ee091ea0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 0x7f60fc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f61067fc640 1 -- 172.21.15.45:0/1718270227 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f61000eff00 con 0x7f61101020d0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:36996/0 (socket says 172.21.15.45:36996) 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 -- 172.21.15.45:0/513160807 learned_addr learned my addr 172.21.15.45:0/513160807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f6115d8b640 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee09c570 0x5605ee09e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f6114d89640 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee09c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 -- 172.21.15.45:0/513160807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 msgr2=0x5605ee09c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee09c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 -- 172.21.15.45:0/513160807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee09c570 msgr2=0x5605ee09e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee09c570 0x5605ee09e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 -- 172.21.15.45:0/513160807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605edfc2400 con 0x5605ee091580 2024-04-25T00:22:33.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 0x7f60fc0a5250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6108009440 tx=0x7f6108009e30 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.985+0000 7f611558a640 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f610c00bcc0 tx=0x7f610c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a52c24ff55b77c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f60fb7fe640 1 -- 172.21.15.45:0/513160807 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c005020 con 0x5605ee091580 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f60fb7fe640 1 -- 172.21.15.45:0/513160807 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610c0047a0 con 0x5605ee091580 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f60fb7fe640 1 -- 172.21.15.45:0/513160807 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c011780 con 0x5605ee091580 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 -- 172.21.15.45:0/513160807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 msgr2=0x5605ee091960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f610c00bcc0 tx=0x7f610c00f730 comp rx=0 tx=0).stop 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 -- 172.21.15.45:0/513160807 shutdown_connections 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee09c570 0x5605ee09e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee09c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 --2- 172.21.15.45:0/513160807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091580 0x5605ee091960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 -- 172.21.15.45:0/513160807 >> 172.21.15.45:0/513160807 conn(0x5605ee08d8a0 msgr2=0x5605ee08dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 -- 172.21.15.45:0/513160807 shutdown_connections 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.986+0000 7f611a5d8740 1 -- 172.21.15.45:0/513160807 wait complete. 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 -- start start 2024-04-25T00:22:33.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 Processor -- start 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091580 0x5605ee09be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee099660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605ee09f360 con 0x5605ee091ea0 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5605edfb2050 con 0x5605ee09c570 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5605ee09f4e0 con 0x5605ee091580 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f611a5d8740 10 client.0 _ll_register_callbacks cb 0x5605edd09be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f6114d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f6114d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39948/0 (socket says 172.21.15.45:39948) 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.987+0000 7f6114d89640 1 -- 172.21.15.45:0/2385103506 learned_addr learned my addr 172.21.15.45:0/2385103506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f611558a640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091580 0x5605ee09be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6115d8b640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee099660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 -- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091580 msgr2=0x5605ee09be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091580 0x5605ee09be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 -- 172.21.15.45:0/2385103506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 msgr2=0x5605ee099660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee099660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 -- 172.21.15.45:0/2385103506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605edf83dc0 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6115d8b640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee099660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f6114d89640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f61000b8920 tx=0x7f61000bb970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de769102fa9479a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f60f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100049070 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f61000b85c0 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c0047a0 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61000bc650 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f611a5d8740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605edd0a3f0 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.988+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610c0047a0 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6100003c80 con 0x5605ee091ea0 2024-04-25T00:22:33.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 20 client.15831 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee0ac4a0 0x5605ee0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 4 client.15831 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ee003bb0 con 0x5605ee0ac4a0 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611558a640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee0ac4a0 0x5605ee0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f60f8ff9640 10 client.15831 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.989+0000 7f611558a640 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee0ac4a0 0x5605ee0ae8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ebc6df3d6bbcc443 server_cookie=c284bdad2771b306 in_seq=0 out_seq=0 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f60f8ff9640 1 -- 172.21.15.45:0/2385103506 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605ee003bb0 con 0x5605ee0ac4a0 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f60f8ff9640 10 client.15831 handle_command_reply: tid=0 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 client.15831 shutdown 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 msgr2=0x5605ee099120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f61000b8920 tx=0x7f61000bb970 comp rx=0 tx=0).stop 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 shutdown_connections 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee0ac4a0 0x5605ee0ae8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee099660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f60f8ff9640 0 client.15831 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091ea0 0x5605ee099120 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 --2- 172.21.15.45:0/2385103506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091580 0x5605ee09be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.990+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 >> 172.21.15.45:0/2385103506 conn(0x5605ee08d8a0 msgr2=0x5605ee0936a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.991+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 shutdown_connections 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.991+0000 7f611a5d8740 1 -- 172.21.15.45:0/2385103506 wait complete. 2024-04-25T00:22:33.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:33.991+0000 7f611a5d8740 20 client.15831 trim_cache size 0 max 16384 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 Processor -- start 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 -- start start 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 0x5605ee0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 0x5605ee0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b0d30 con 0x5605ee091580 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b2870 con 0x5605ee091ea0 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611a5d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b1400 con 0x5605ee09c570 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611558a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611558a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39952/0 (socket says 172.21.15.45:39952) 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.004+0000 7f611558a640 1 -- 172.21.15.45:0/1686963377 learned_addr learned my addr 172.21.15.45:0/1686963377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 0x5605ee0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f6114d89640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 0x5605ee0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 -- 172.21.15.45:0/1686963377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 msgr2=0x5605ee0a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 0x5605ee0a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 -- 172.21.15.45:0/1686963377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 msgr2=0x5605ee0a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 0x5605ee0a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 -- 172.21.15.45:0/1686963377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605eddd47d0 con 0x5605ee091580 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f6114d89640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 0x5605ee0a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f6115d8b640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 0x5605ee0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611558a640 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f610c00bcc0 tx=0x7f610c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b3572a5375b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f60faffd640 1 -- 172.21.15.45:0/1686963377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c0052b0 con 0x5605ee091580 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f60faffd640 1 -- 172.21.15.45:0/1686963377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610c005450 con 0x5605ee091580 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f60faffd640 1 -- 172.21.15.45:0/1686963377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c003940 con 0x5605ee091580 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 -- 172.21.15.45:0/1686963377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 msgr2=0x5605ee09b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f610c00bcc0 tx=0x7f610c005040 comp rx=0 tx=0).stop 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 -- 172.21.15.45:0/1686963377 shutdown_connections 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee09c570 0x5605ee0a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee091ea0 0x5605ee0a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.005+0000 7f611a5d8740 1 --2- 172.21.15.45:0/1686963377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee09b960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- 172.21.15.45:0/1686963377 >> 172.21.15.45:0/1686963377 conn(0x5605ee08d8a0 msgr2=0x5605ee0a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- 172.21.15.45:0/1686963377 shutdown_connections 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- 172.21.15.45:0/1686963377 wait complete. 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- start start 2024-04-25T00:22:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 Processor -- start 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee0a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee111a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b21a0 con 0x5605ee091580 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b1ad0 con 0x5605ee09c570 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61101aed10 con 0x5605ee091ea0 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611a5d8740 10 client.0 _ll_register_callbacks cb 0x7f6100009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611558a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611558a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39960/0 (socket says 172.21.15.45:39960) 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f611558a640 1 -- 172.21.15.45:0/256311018 learned_addr learned my addr 172.21.15.45:0/256311018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.006+0000 7f6115d8b640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee111a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f6114d89640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee0a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 -- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 msgr2=0x5605ee0a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee0a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 -- 172.21.15.45:0/256311018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 msgr2=0x5605ee111a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee111a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 -- 172.21.15.45:0/256311018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ee111f80 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611558a640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f610c00ff20 tx=0x7f610c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaa6bbf9e69e8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f60f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c038060 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f610c029320 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f610c005450 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f610c00b560 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611a5d8740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605ee113030 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.007+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f610c005450 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f610c011440 con 0x5605ee091580 2024-04-25T00:22:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 10 client.15837 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 20 client.15837 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee113580 0x5605ee1159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 4 client.15837 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605edffdf00 con 0x5605ee113580 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f6114d89640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee113580 0x5605ee1159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f6114d89640 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee113580 0x5605ee1159a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c30542b810acc9b8 server_cookie=e9b6a4d40527e326 in_seq=0 out_seq=0 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.008+0000 7f60f8ff9640 10 client.15837 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f60f8ff9640 1 -- 172.21.15.45:0/256311018 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605edffdf00 con 0x5605ee113580 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f60f8ff9640 10 client.15837 handle_command_reply: tid=0 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 client.15837 shutdown 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 msgr2=0x5605ee0a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f610c00ff20 tx=0x7f610c00f730 comp rx=0 tx=0).stop 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 shutdown_connections 2024-04-25T00:22:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5605ee113580 0x5605ee1159a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5605ee09c570 0x5605ee111a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605ee091ea0 0x5605ee0a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f60f8ff9640 0 client.15837 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 --2- 172.21.15.45:0/256311018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605ee091580 0x5605ee0a2540 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 >> 172.21.15.45:0/256311018 conn(0x5605ee08d8a0 msgr2=0x5605ee0a7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 shutdown_connections 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 1 -- 172.21.15.45:0/256311018 wait complete. 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.009+0000 7f611a5d8740 20 client.15837 trim_cache size 0 max 16384 2024-04-25T00:22:34.016 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 msgr2=0x7f60fc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 0x7f60fc0a5250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6108009440 tx=0x7f6108009e30 comp rx=0 tx=0).stop 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 msgr2=0x7f611019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6100009230 tx=0x7f6100004270 comp rx=0 tx=0).stop 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 shutdown_connections 2024-04-25T00:22:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60fc0a2d90 0x7f60fc0a5250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6110107720 0x7f611019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61101020d0 0x7f611019a830 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 --2- 172.21.15.45:0/1718270227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6110101710 0x7f611019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 >> 172.21.15.45:0/1718270227 conn(0x7f6110078b90 msgr2=0x7f61100fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 shutdown_connections 2024-04-25T00:22:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.010+0000 7f61177ee640 1 -- 172.21.15.45:0/1718270227 wait complete. 2024-04-25T00:22:34.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 -- 172.21.15.45:0/4104278358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110101720 msgr2=0x7fd110101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 --2- 172.21.15.45:0/4104278358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110101720 0x7fd110101ba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd104009930 tx=0x7fd10402a420 comp rx=0 tx=0).stop 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 -- 172.21.15.45:0/4104278358 shutdown_connections 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 --2- 172.21.15.45:0/4104278358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1101020e0 0x7fd11010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 --2- 172.21.15.45:0/4104278358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110101720 0x7fd110101ba0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 --2- 172.21.15.45:0/4104278358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd110107730 0x7fd110107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 -- 172.21.15.45:0/4104278358 >> 172.21.15.45:0/4104278358 conn(0x7fd110078b40 msgr2=0x7fd110078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 -- 172.21.15.45:0/4104278358 shutdown_connections 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.128+0000 7fd115116640 1 -- 172.21.15.45:0/4104278358 wait complete. 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.129+0000 7fd115116640 1 Processor -- start 2024-04-25T00:22:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.129+0000 7fd115116640 1 -- start start 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd110101720 0x7fd110198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1101020e0 0x7fd1101985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111b40 con 0x7fd110107730 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101119c0 con 0x7fd110101720 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd110111cc0 con 0x7fd1101020e0 2024-04-25T00:22:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39978/0 (socket says 172.21.15.45:39978) 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 -- 172.21.15.45:0/2115285947 learned_addr learned my addr 172.21.15.45:0/2115285947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd110101720 0x7fd110198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10e59c640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1101020e0 0x7fd1101985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 -- 172.21.15.45:0/2115285947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1101020e0 msgr2=0x7fd1101985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1101020e0 0x7fd1101985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 -- 172.21.15.45:0/2115285947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd110101720 msgr2=0x7fd110198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd110101720 0x7fd110198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 -- 172.21.15.45:0/2115285947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11019d050 con 0x7fd110107730 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd10f59e640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd108009fd0 tx=0x7fd10800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd0f7fff640 1 -- 172.21.15.45:0/2115285947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108017070 con 0x7fd110107730 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11019d340 con 0x7fd110107730 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.130+0000 7fd0f7fff640 1 -- 172.21.15.45:0/2115285947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd108003e50 con 0x7fd110107730 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.131+0000 7fd0f7fff640 1 -- 172.21.15.45:0/2115285947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1080106c0 con 0x7fd110107730 2024-04-25T00:22:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.131+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1101a4bd0 con 0x7fd110107730 2024-04-25T00:22:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd0f7fff640 1 -- 172.21.15.45:0/2115285947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fd108003970 con 0x7fd110107730 2024-04-25T00:22:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 Processor -- start 2024-04-25T00:22:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 -- start start 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4524480 0x55c7c4524860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4533440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c7c4489670 con 0x55c7c4533980 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c7c4431970 con 0x55c7c4524da0 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd117e1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7c4485630 con 0x55c7c4524480 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39990/0 (socket says 172.21.15.45:39990) 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd10e59c640 1 -- 172.21.15.45:0/762306140 learned_addr learned my addr 172.21.15.45:0/762306140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd0f7fff640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 0x7fd0f80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.133+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4533440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4524480 0x55c7c4524860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10f59e640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 0x7fd0f80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 -- 172.21.15.45:0/762306140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4524480 msgr2=0x55c7c4524860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4524480 0x55c7c4524860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 -- 172.21.15.45:0/762306140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 msgr2=0x55c7c4533440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4533440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 -- 172.21.15.45:0/762306140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c4455400 con 0x55c7c4533980 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd0f7fff640 1 -- 172.21.15.45:0/2115285947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd1080cf620 con 0x7fd110107730 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10e59c640 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd10402a940 tx=0x7fd104005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4e514f784093bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd0f57fa640 1 -- 172.21.15.45:0/762306140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104035040 con 0x55c7c4533980 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd10f59e640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 0x7fd0f80a5250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cc9b0 tx=0x7fd108004020 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd0f57fa640 1 -- 172.21.15.45:0/762306140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104031a20 con 0x55c7c4533980 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.134+0000 7fd0f57fa640 1 -- 172.21.15.45:0/762306140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104004a00 con 0x55c7c4533980 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- 172.21.15.45:0/762306140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 msgr2=0x55c7c4535d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd10402a940 tx=0x7fd104005fd0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- 172.21.15.45:0/762306140 shutdown_connections 2024-04-25T00:22:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4533980 0x55c7c4535d70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4533440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- 172.21.15.45:0/762306140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4524480 0x55c7c4524860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- 172.21.15.45:0/762306140 >> 172.21.15.45:0/762306140 conn(0x55c7c45208a0 msgr2=0x55c7c4520c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- 172.21.15.45:0/762306140 shutdown_connections 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- 172.21.15.45:0/762306140 wait complete. 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 -- start start 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 Processor -- start 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.135+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c45304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4530a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c7c45366e0 con 0x55c7c4524480 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c7c4445050 con 0x55c7c4524da0 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7c4536860 con 0x55c7c4533980 2024-04-25T00:22:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40006/0 (socket says 172.21.15.45:40006) 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 -- 172.21.15.45:0/3657342720 learned_addr learned my addr 172.21.15.45:0/3657342720 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 10 client.0 _ll_register_callbacks cb 0x55c7c419cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c45304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4530a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 -- 172.21.15.45:0/3657342720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 msgr2=0x55c7c4530a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4530a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 -- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 msgr2=0x55c7c45304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c45304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 -- 172.21.15.45:0/3657342720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c4416dc0 con 0x55c7c4524480 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd10e59c640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd104009a50 tx=0x7fd104005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c55f430ccce184b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd0f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10404d070 con 0x55c7c4524480 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd104004980 con 0x55c7c4524480 2024-04-25T00:22:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104031a20 con 0x55c7c4524480 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd104035040 con 0x55c7c4524480 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.136+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7c419d3f0 con 0x55c7c4524480 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd104031a20 con 0x55c7c4524480 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd104049b30 con 0x55c7c4524480 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 10 client.15849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 20 client.15849 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c4543790 0x55c7c4545b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 4 client.15849 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c4496bb0 con 0x55c7c4543790 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.137+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c4543790 0x55c7c4545b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c4543790 0x55c7c4545b80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa4d80a5033f7194 server_cookie=8a3fc6eb993c5bed in_seq=0 out_seq=0 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd0f2ff5640 10 client.15849 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/3657342720 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7c4496bb0 con 0x55c7c4543790 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd0f2ff5640 10 client.15849 handle_command_reply: tid=0 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd117e1b740 1 client.15849 shutdown 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 msgr2=0x55c7c4533200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.138+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd104009a50 tx=0x7fd104005650 comp rx=0 tx=0).stop 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 shutdown_connections 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c4543790 0x55c7c4545b80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4530a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd0f2ff5640 0 client.15849 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c45304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 --2- 172.21.15.45:0/3657342720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4533200 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 >> 172.21.15.45:0/3657342720 conn(0x55c7c45208a0 msgr2=0x55c7c45266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 shutdown_connections 2024-04-25T00:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 1 -- 172.21.15.45:0/3657342720 wait complete. 2024-04-25T00:22:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.139+0000 7fd117e1b740 20 client.15849 trim_cache size 0 max 16384 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 Processor -- start 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 -- start start 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 0x55c7c452f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4532d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101ac910 con 0x55c7c4524da0 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101ae450 con 0x55c7c4524480 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd117e1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101acfe0 con 0x55c7c4533980 2024-04-25T00:22:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40018/0 (socket says 172.21.15.45:40018) 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10ed9d640 1 -- 172.21.15.45:0/2793528131 learned_addr learned my addr 172.21.15.45:0/2793528131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10e59c640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 0x55c7c452f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10ed9d640 1 -- 172.21.15.45:0/2793528131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 msgr2=0x55c7c4532d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.152+0000 7fd10e59c640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4532d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4532d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10ed9d640 1 -- 172.21.15.45:0/2793528131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 msgr2=0x55c7c452f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 0x55c7c452f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10ed9d640 1 -- 172.21.15.45:0/2793528131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c42677d0 con 0x55c7c4524da0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10e59c640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 0x55c7c452f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd10000b850 tx=0x7fd10000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceb11c7c60fff52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd0f4ff9640 1 -- 172.21.15.45:0/2793528131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10000bcf0 con 0x55c7c4524da0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd0f4ff9640 1 -- 172.21.15.45:0/2793528131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10000be90 con 0x55c7c4524da0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd0f4ff9640 1 -- 172.21.15.45:0/2793528131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100004e40 con 0x55c7c4524da0 2024-04-25T00:22:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 -- 172.21.15.45:0/2793528131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 msgr2=0x55c7c451f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd10000b850 tx=0x7fd10000b390 comp rx=0 tx=0).stop 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 -- 172.21.15.45:0/2793528131 shutdown_connections 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4532d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524da0 0x55c7c451f9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 --2- 172.21.15.45:0/2793528131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524480 0x55c7c452f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.153+0000 7fd117e1b740 1 -- 172.21.15.45:0/2793528131 >> 172.21.15.45:0/2793528131 conn(0x55c7c45208a0 msgr2=0x55c7c451f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- 172.21.15.45:0/2793528131 shutdown_connections 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- 172.21.15.45:0/2793528131 wait complete. 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- start start 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 Processor -- start 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4546ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4547000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101add80 con 0x55c7c4524480 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101ad6b0 con 0x55c7c4524da0 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1101aa8f0 con 0x55c7c4533980 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd117e1b740 10 client.0 _ll_register_callbacks cb 0x7fd104005280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40024/0 (socket says 172.21.15.45:40024) 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 -- 172.21.15.45:0/468855871 learned_addr learned my addr 172.21.15.45:0/468855871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4546ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 -- 172.21.15.45:0/468855871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 msgr2=0x55c7c4547000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4547000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 -- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 msgr2=0x55c7c4546ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4546ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10e59c640 1 -- 172.21.15.45:0/468855871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c45a8970 con 0x55c7c4524480 2024-04-25T00:22:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.154+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4546ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd10e59c640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd104009850 tx=0x7fd104005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c951f181a04872ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10405b070 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd104004dc0 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd10000be90 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd104067050 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7c45a9bf0 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10000be90 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd10403aa00 con 0x55c7c4524480 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 10 client.15855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 20 client.15855 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c45aa230 0x55c7c45ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 4 client.15855 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.155+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7c4490f00 con 0x55c7c45aa230 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c45aa230 0x55c7c45ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd10ed9d640 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c45aa230 0x55c7c45ac650 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fc7181c70e5ecc3 server_cookie=c7076ba3543ce2c2 in_seq=0 out_seq=0 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd0f2ff5640 10 client.15855 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd0f2ff5640 1 -- 172.21.15.45:0/468855871 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7c4490f00 con 0x55c7c45aa230 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd0f2ff5640 10 client.15855 handle_command_reply: tid=0 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 client.15855 shutdown 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 msgr2=0x55c7c4546580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd104009850 tx=0x7fd104005510 comp rx=0 tx=0).stop 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 shutdown_connections 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c7c45aa230 0x55c7c45ac650 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7c4533980 0x55c7c4547000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c7c4524da0 0x55c7c4546ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 --2- 172.21.15.45:0/468855871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c7c4524480 0x55c7c4546580 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd0f2ff5640 0 client.15855 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 >> 172.21.15.45:0/468855871 conn(0x55c7c45208a0 msgr2=0x55c7c451fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 shutdown_connections 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 1 -- 172.21.15.45:0/468855871 wait complete. 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.156+0000 7fd117e1b740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T00:22:34.164 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 msgr2=0x7fd0f80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 0x7fd0f80a5250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cc9b0 tx=0x7fd108004020 comp rx=0 tx=0).stop 2024-04-25T00:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 msgr2=0x7fd11019c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd108009fd0 tx=0x7fd10800b260 comp rx=0 tx=0).stop 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 shutdown_connections 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd0f80a2d90 0x7fd0f80a5250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd110107730 0x7fd11019c920 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1101020e0 0x7fd1101985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 --2- 172.21.15.45:0/2115285947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd110101720 0x7fd110198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 >> 172.21.15.45:0/2115285947 conn(0x7fd110078b40 msgr2=0x7fd110104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 shutdown_connections 2024-04-25T00:22:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.157+0000 7fd115116640 1 -- 172.21.15.45:0/2115285947 wait complete. 2024-04-25T00:22:34.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.275+0000 7f37b70df640 1 -- 172.21.15.45:0/157084709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01029c0 msgr2=0x7f37b0102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.275+0000 7f37b70df640 1 --2- 172.21.15.45:0/157084709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01029c0 0x7f37b0102da0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009900 tx=0x7f37a002a510 comp rx=0 tx=0).stop 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- 172.21.15.45:0/157084709 shutdown_connections 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- 172.21.15.45:0/157084709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 0x7f37b010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- 172.21.15.45:0/157084709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01032e0 0x7f37b010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- 172.21.15.45:0/157084709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01029c0 0x7f37b0102da0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- 172.21.15.45:0/157084709 >> 172.21.15.45:0/157084709 conn(0x7f37b0077740 msgr2=0x7f37b0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- 172.21.15.45:0/157084709 shutdown_connections 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- 172.21.15.45:0/157084709 wait complete. 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 Processor -- start 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- start start 2024-04-25T00:22:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01029c0 0x7f37b0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 0x7f37b019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0111b50 con 0x7f37b01032e0 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01119d0 con 0x7f37b01029c0 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.276+0000 7f37b70df640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0111cd0 con 0x7f37b010bd00 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40038/0 (socket says 172.21.15.45:40038) 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 -- 172.21.15.45:0/2862571761 learned_addr learned my addr 172.21.15.45:0/2862571761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37b567c640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 0x7f37b019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01029c0 0x7f37b0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 -- 172.21.15.45:0/2862571761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 msgr2=0x7f37b019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 0x7f37b019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 -- 172.21.15.45:0/2862571761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01029c0 msgr2=0x7f37b0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01029c0 0x7f37b0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 -- 172.21.15.45:0/2862571761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b019d010 con 0x7f37b01032e0 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37affff640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c890 tx=0x7f37a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37adffb640 1 -- 172.21.15.45:0/2862571761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4004220 con 0x7f37b01032e0 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37adffb640 1 -- 172.21.15.45:0/2862571761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a40043c0 con 0x7f37b01032e0 2024-04-25T00:22:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37adffb640 1 -- 172.21.15.45:0/2862571761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40049e0 con 0x7f37b01032e0 2024-04-25T00:22:34.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.277+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b019d300 con 0x7f37b01032e0 2024-04-25T00:22:34.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.278+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b01a4b90 con 0x7f37b01032e0 2024-04-25T00:22:34.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37adffb640 1 -- 172.21.15.45:0/2862571761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f37a40079b0 con 0x7f37b01032e0 2024-04-25T00:22:34.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 Processor -- start 2024-04-25T00:22:34.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 -- start start 2024-04-25T00:22:34.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d86030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556c40ce0670 con 0x556c40d7b580 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c40c88970 con 0x556c40d7bea0 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b9de0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c40cdc630 con 0x556c40d86570 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b567c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b567c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40042/0 (socket says 172.21.15.45:40042) 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37b567c640 1 -- 172.21.15.45:0/2031690947 learned_addr learned my addr 172.21.15.45:0/2031690947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.280+0000 7f37adffb640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 0x7f37980a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d86030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37affff640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 -- 172.21.15.45:0/2031690947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 msgr2=0x556c40d88960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d88960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 -- 172.21.15.45:0/2031690947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 msgr2=0x556c40d86030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d86030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 -- 172.21.15.45:0/2031690947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c40cac400 con 0x556c40d7b580 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d86030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37affff640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d88960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 0x7f37980a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37adffb640 1 -- 172.21.15.45:0/2862571761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f37a40cf880 con 0x7f37b01032e0 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f37b567c640 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f37a8010440 tx=0x7f37a8010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f48f6b4a157ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f3796ffd640 1 -- 172.21.15.45:0/2031690947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a801d400 con 0x556c40d7b580 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f3796ffd640 1 -- 172.21.15.45:0/2031690947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8013520 con 0x556c40d7b580 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.281+0000 7f3796ffd640 1 -- 172.21.15.45:0/2031690947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8011990 con 0x556c40d7b580 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- 172.21.15.45:0/2031690947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 msgr2=0x556c40d7b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f37a8010440 tx=0x7f37a8010900 comp rx=0 tx=0).stop 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b567c640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 0x7f37980a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37a8021000 tx=0x7f37a801e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- 172.21.15.45:0/2031690947 shutdown_connections 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d88960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d86030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 --2- 172.21.15.45:0/2031690947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d7b960 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- 172.21.15.45:0/2031690947 >> 172.21.15.45:0/2031690947 conn(0x556c40d778a0 msgr2=0x556c40d77c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- 172.21.15.45:0/2031690947 shutdown_connections 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- 172.21.15.45:0/2031690947 wait complete. 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 -- start start 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.282+0000 7f37b9de0740 1 Processor -- start 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556c40d89360 con 0x556c40d7b580 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c40c9c050 con 0x556c40d7bea0 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c40d894e0 con 0x556c40d86570 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b9de0740 10 client.0 _ll_register_callbacks cb 0x556c409f3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40056/0 (socket says 172.21.15.45:40056) 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 -- 172.21.15.45:0/64838257 learned_addr learned my addr 172.21.15.45:0/64838257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37affff640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b567c640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d83730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 -- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 msgr2=0x556c40d83730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d83730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 -- 172.21.15.45:0/64838257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 msgr2=0x556c40d831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 -- 172.21.15.45:0/64838257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c40c6ddc0 con 0x556c40d7b580 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f37a0031550 tx=0x7f37a0004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34030f1099d33e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.283+0000 7f378ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0035040 con 0x556c40d7b580 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f37a0005510 con 0x556c40d7b580 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a8013520 con 0x556c40d7b580 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37a00046c0 con 0x556c40d7b580 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c409f43f0 con 0x556c40d7b580 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8013520 con 0x556c40d7b580 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f37a0003bf0 con 0x556c40d7b580 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 20 client.15867 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40d96530 0x556c40d98980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 4 client.15867 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.284+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c40cedbb0 con 0x556c40d96530 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.285+0000 7f37affff640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40d96530 0x556c40d98980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.285+0000 7f37affff640 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40d96530 0x556c40d98980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f755e5157312088e server_cookie=4ba663be0324631e in_seq=0 out_seq=0 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.285+0000 7f378ffff640 10 client.15867 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f378ffff640 1 -- 172.21.15.45:0/64838257 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c40cedbb0 con 0x556c40d96530 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f378ffff640 10 client.15867 handle_command_reply: tid=0 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 client.15867 shutdown 2024-04-25T00:22:34.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 msgr2=0x556c40d85ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f37a0031550 tx=0x7f37a0004460 comp rx=0 tx=0).stop 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 shutdown_connections 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40d96530 0x556c40d98980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7bea0 0x556c40d831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f378ffff640 0 client.15867 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 --2- 172.21.15.45:0/64838257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7b580 0x556c40d85ee0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 >> 172.21.15.45:0/64838257 conn(0x556c40d778a0 msgr2=0x556c40d76b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 shutdown_connections 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 1 -- 172.21.15.45:0/64838257 wait complete. 2024-04-25T00:22:34.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.286+0000 7f37b9de0740 20 client.15867 trim_cache size 0 max 16384 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.299+0000 7f37b9de0740 1 Processor -- start 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.299+0000 7f37b9de0740 1 -- start start 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d92220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01aec50 con 0x556c40d7bea0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b0790 con 0x556c40d7b580 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b9de0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01af320 con 0x556c40d86570 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40072/0 (socket says 172.21.15.45:40072) 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 -- 172.21.15.45:0/1202244518 learned_addr learned my addr 172.21.15.45:0/1202244518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b567c640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 -- 172.21.15.45:0/1202244518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 msgr2=0x556c40d85f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d85f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d92220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 -- 172.21.15.45:0/1202244518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 msgr2=0x556c40d92220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d92220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 -- 172.21.15.45:0/1202244518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c40abe7d0 con 0x556c40d7bea0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d92220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37affff640 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f37a4097de0 tx=0x7f37a40c6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fe03fcd57809d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37967fc640 1 -- 172.21.15.45:0/1202244518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a401d040 con 0x556c40d7bea0 2024-04-25T00:22:34.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37967fc640 1 -- 172.21.15.45:0/1202244518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a409b980 con 0x556c40d7bea0 2024-04-25T00:22:34.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.300+0000 7f37967fc640 1 -- 172.21.15.45:0/1202244518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40cce60 con 0x556c40d7bea0 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- 172.21.15.45:0/1202244518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 msgr2=0x556c40d85a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f37a4097de0 tx=0x7f37a40c6460 comp rx=0 tx=0).stop 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- 172.21.15.45:0/1202244518 shutdown_connections 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40d85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d85a10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 --2- 172.21.15.45:0/1202244518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d92220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- 172.21.15.45:0/1202244518 >> 172.21.15.45:0/1202244518 conn(0x556c40d778a0 msgr2=0x556c40d7d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- 172.21.15.45:0/1202244518 shutdown_connections 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- 172.21.15.45:0/1202244518 wait complete. 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.301+0000 7f37b9de0740 1 -- start start 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 Processor -- start 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40dfbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b00c0 con 0x556c40d7bea0 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01af9f0 con 0x556c40d7b580 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01acc30 con 0x556c40d86570 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b9de0740 10 client.0 _ll_register_callbacks cb 0x7f37a0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40080/0 (socket says 172.21.15.45:40080) 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 -- 172.21.15.45:0/3345206665 learned_addr learned my addr 172.21.15.45:0/3345206665 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d8a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b567c640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40dfbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 -- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 msgr2=0x556c40dfbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40dfbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 -- 172.21.15.45:0/3345206665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 msgr2=0x556c40d8a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d8a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37affff640 1 -- 172.21.15.45:0/3345206665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c40d8b320 con 0x556c40d7bea0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b567c640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40dfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.302+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d8a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37affff640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f37a4098350 tx=0x7f37a409fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3bfe0c3c755f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a40b3070 con 0x556c40d7bea0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f37a40cce60 con 0x556c40d7bea0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a409b980 con 0x556c40d7bea0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37a401d040 con 0x556c40d7bea0 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c40dfd070 con 0x556c40d7bea0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a409b980 con 0x556c40d7bea0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f37a40860b0 con 0x556c40d7bea0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 20 client.15873 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40dfd7f0 0x556c40dffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 4 client.15873 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c40ce7f00 con 0x556c40dfd7f0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.303+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40dfd7f0 0x556c40dffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b4e7b640 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40dfd7f0 0x556c40dffc10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de3adc4b5bc1c4c4 server_cookie=7278e69774f3ebf5 in_seq=0 out_seq=0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f378ffff640 10 client.15873 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f378ffff640 1 -- 172.21.15.45:0/3345206665 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c40ce7f00 con 0x556c40dfd7f0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f378ffff640 10 client.15873 handle_command_reply: tid=0 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 client.15873 shutdown 2024-04-25T00:22:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 msgr2=0x556c40d8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f37a4098350 tx=0x7f37a409fcf0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 shutdown_connections 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556c40dfd7f0 0x556c40dffc10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c40d86570 0x556c40dfbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c40d7bea0 0x556c40d8aa20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f378ffff640 0 client.15873 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 --2- 172.21.15.45:0/3345206665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c40d7b580 0x556c40d8a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 >> 172.21.15.45:0/3345206665 conn(0x556c40d778a0 msgr2=0x556c40d7dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 shutdown_connections 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 1 -- 172.21.15.45:0/3345206665 wait complete. 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.304+0000 7f37b9de0740 20 client.15873 trim_cache size 0 max 16384 2024-04-25T00:22:34.311 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:22:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 msgr2=0x7f37980a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 0x7f37980a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37a8021000 tx=0x7f37a801e000 comp rx=0 tx=0).stop 2024-04-25T00:22:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 msgr2=0x7f37b01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c890 tx=0x7f37a400cd50 comp rx=0 tx=0).stop 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 shutdown_connections 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f37980a2e10 0x7f37980a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37b010bd00 0x7f37b019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37b01032e0 0x7f37b01985b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 --2- 172.21.15.45:0/2862571761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37b01029c0 0x7f37b0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 >> 172.21.15.45:0/2862571761 conn(0x7f37b0077740 msgr2=0x7f37b00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 shutdown_connections 2024-04-25T00:22:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.305+0000 7f37b70df640 1 -- 172.21.15.45:0/2862571761 wait complete. 2024-04-25T00:22:34.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:22:34.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.419+0000 7f3967f53640 1 -- 172.21.15.45:0/1401247034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960100fe0 msgr2=0x7f396010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.419+0000 7f3967f53640 1 --2- 172.21.15.45:0/1401247034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960100fe0 0x7f396010d4f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3958009cd0 tx=0x7f395802a8f0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 -- 172.21.15.45:0/1401247034 shutdown_connections 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 --2- 172.21.15.45:0/1401247034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960100fe0 0x7f396010d4f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 --2- 172.21.15.45:0/1401247034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100620 0x7f3960100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 --2- 172.21.15.45:0/1401247034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960106630 0x7f3960106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 -- 172.21.15.45:0/1401247034 >> 172.21.15.45:0/1401247034 conn(0x7f39600776a0 msgr2=0x7f3960077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 -- 172.21.15.45:0/1401247034 shutdown_connections 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 -- 172.21.15.45:0/1401247034 wait complete. 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 Processor -- start 2024-04-25T00:22:34.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.420+0000 7f3967f53640 1 -- start start 2024-04-25T00:22:34.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960100620 0x7f396010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100fe0 0x7f396010e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3960111b40 con 0x7f3960106630 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f39601119c0 con 0x7f3960100620 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3960111cc0 con 0x7f3960100fe0 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40106/0 (socket says 172.21.15.45:40106) 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 -- 172.21.15.45:0/716376311 learned_addr learned my addr 172.21.15.45:0/716376311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3965cef640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960100620 0x7f396010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39654ee640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100fe0 0x7f396010e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 -- 172.21.15.45:0/716376311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100fe0 msgr2=0x7f396010e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100fe0 0x7f396010e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 -- 172.21.15.45:0/716376311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960100620 msgr2=0x7f396010df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960100620 0x7f396010df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 -- 172.21.15.45:0/716376311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39601a63e0 con 0x7f3960106630 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f39664f0640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ca0 tx=0x7f3958030ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3956ffd640 1 -- 172.21.15.45:0/716376311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958003e00 con 0x7f3960106630 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3956ffd640 1 -- 172.21.15.45:0/716376311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3958040070 con 0x7f3960106630 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601a6670 con 0x7f3960106630 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3956ffd640 1 -- 172.21.15.45:0/716376311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958045c20 con 0x7f3960106630 2024-04-25T00:22:34.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.421+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a6bc0 con 0x7f3960106630 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f3956ffd640 1 -- 172.21.15.45:0/716376311 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3958004040 con 0x7f3960106630 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 Processor -- start 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 -- start start 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f3956ffd640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 0x7f394c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af45030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af47960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56007ae9f670 con 0x56007af3aea0 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56007ae47970 con 0x56007af45570 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.424+0000 7f396ac5d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56007ae9b630 con 0x56007af3a580 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39654ee640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 0x7f394c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3956ffd640 1 -- 172.21.15.45:0/716376311 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f39580f09d0 con 0x7f3960106630 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af45030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af45030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40120/0 (socket says 172.21.15.45:40120) 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39654ee640 1 -- 172.21.15.45:0/4034033015 learned_addr learned my addr 172.21.15.45:0/4034033015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39664f0640 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af47960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 -- 172.21.15.45:0/4034033015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 msgr2=0x56007af47960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af47960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 -- 172.21.15.45:0/4034033015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 msgr2=0x56007af45030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af45030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 -- 172.21.15.45:0/4034033015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56007ae6b400 con 0x56007af3a580 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39664f0640 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af47960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f39654ee640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 0x7f394c0a5250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3950013990 tx=0x7f395000bb70 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f3965cef640 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af3a960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f395c00bb70 tx=0x7f395c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af8bc40d43c6d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f394bfff640 1 -- 172.21.15.45:0/4034033015 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00be60 con 0x56007af3a580 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f394bfff640 1 -- 172.21.15.45:0/4034033015 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395c0035f0 con 0x56007af3a580 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.425+0000 7f394bfff640 1 -- 172.21.15.45:0/4034033015 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0116a0 con 0x56007af3a580 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 -- 172.21.15.45:0/4034033015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 msgr2=0x56007af3a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af3a960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f395c00bb70 tx=0x7f395c0107a0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 -- 172.21.15.45:0/4034033015 shutdown_connections 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af47960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af45030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 --2- 172.21.15.45:0/4034033015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af3a960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 -- 172.21.15.45:0/4034033015 >> 172.21.15.45:0/4034033015 conn(0x56007af368a0 msgr2=0x56007af36c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 -- 172.21.15.45:0/4034033015 shutdown_connections 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.426+0000 7f396ac5d740 1 -- 172.21.15.45:0/4034033015 wait complete. 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 -- start start 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 Processor -- start 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af44ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af42730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56007af48360 con 0x56007af3aea0 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56007ae5b050 con 0x56007af45570 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56007af484e0 con 0x56007af3a580 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f396ac5d740 10 client.0 _ll_register_callbacks cb 0x56007abb2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40134/0 (socket says 172.21.15.45:40134) 2024-04-25T00:22:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f39654ee640 1 -- 172.21.15.45:0/3939409242 learned_addr learned my addr 172.21.15.45:0/3939409242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f3965cef640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af44ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.427+0000 7f39664f0640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af42730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 -- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 msgr2=0x56007af44ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af44ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 -- 172.21.15.45:0/3939409242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 msgr2=0x56007af42730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af42730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 -- 172.21.15.45:0/3939409242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007ae2cdc0 con 0x56007af3aea0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39664f0640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af42730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39654ee640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3950013310 tx=0x7f395001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cbf57f4b97aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39497fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950024030 con 0x56007af3aea0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f3950011bd0 con 0x56007af3aea0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f395c0035f0 con 0x56007af3aea0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3950031050 con 0x56007af3aea0 2024-04-25T00:22:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f396ac5d740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56007abb33f0 con 0x56007af3aea0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.428+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395c0035f0 con 0x56007af3aea0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f395002b4d0 con 0x56007af3aea0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 20 client.15885 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007af554d0 0x56007af57900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 4 client.15885 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56007aeacbb0 con 0x56007af554d0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f3965cef640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007af554d0 0x56007af57900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f39497fa640 10 client.15885 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.429+0000 7f3965cef640 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007af554d0 0x56007af57900 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d118d92ca853f355 server_cookie=b230acaa6a6fecc1 in_seq=0 out_seq=0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f39497fa640 1 -- 172.21.15.45:0/3939409242 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56007aeacbb0 con 0x56007af554d0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f39497fa640 10 client.15885 handle_command_reply: tid=0 2024-04-25T00:22:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 client.15885 shutdown 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 msgr2=0x56007af421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3950013310 tx=0x7f395001a000 comp rx=0 tx=0).stop 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 shutdown_connections 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007af554d0 0x56007af57900 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af45570 0x56007af42730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af421d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3939409242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3a580 0x56007af44ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f39497fa640 0 client.15885 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 >> 172.21.15.45:0/3939409242 conn(0x56007af368a0 msgr2=0x56007af3c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 shutdown_connections 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.430+0000 7f396ac5d740 1 -- 172.21.15.45:0/3939409242 wait complete. 2024-04-25T00:22:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.431+0000 7f396ac5d740 20 client.15885 trim_cache size 0 max 16384 2024-04-25T00:22:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.444+0000 7f396ac5d740 1 Processor -- start 2024-04-25T00:22:34.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.444+0000 7f396ac5d740 1 -- start start 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.444+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.444+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af45570 0x56007af5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f396ac5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b0b70 con 0x56007af3aea0 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f396ac5d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b26b0 con 0x56007af3a580 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f396ac5d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b1240 con 0x56007af45570 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40144/0 (socket says 172.21.15.45:40144) 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 -- 172.21.15.45:0/3204556184 learned_addr learned my addr 172.21.15.45:0/3204556184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f3965cef640 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39664f0640 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af45570 0x56007af5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 -- 172.21.15.45:0/3204556184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af45570 msgr2=0x56007af5d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af45570 0x56007af5d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 -- 172.21.15.45:0/3204556184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 msgr2=0x56007af51220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af51220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 -- 172.21.15.45:0/3204556184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56007af3acd0 con 0x56007af3aea0 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f39654ee640 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f39500058c0 tx=0x7f395000b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc8daf9981fba1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f394b7fe640 1 -- 172.21.15.45:0/3204556184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950024070 con 0x56007af3aea0 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f394b7fe640 1 -- 172.21.15.45:0/3204556184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3950011bd0 con 0x56007af3aea0 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.445+0000 7f394b7fe640 1 -- 172.21.15.45:0/3204556184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395002b340 con 0x56007af3aea0 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- 172.21.15.45:0/3204556184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 msgr2=0x56007af5d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f39500058c0 tx=0x7f395000b380 comp rx=0 tx=0).stop 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- 172.21.15.45:0/3204556184 shutdown_connections 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af45570 0x56007af5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af3aea0 0x56007af5d1d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3204556184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af51220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- 172.21.15.45:0/3204556184 >> 172.21.15.45:0/3204556184 conn(0x56007af368a0 msgr2=0x56007af40aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- 172.21.15.45:0/3204556184 shutdown_connections 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- 172.21.15.45:0/3204556184 wait complete. 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 -- start start 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.446+0000 7f396ac5d740 1 Processor -- start 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af4b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3aea0 0x56007af4b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b1fe0 con 0x56007af45570 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f39601b1910 con 0x56007af3a580 2024-04-25T00:22:34.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39601aeb50 con 0x56007af3aea0 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f396ac5d740 10 client.0 _ll_register_callbacks cb 0x7f395000b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40152/0 (socket says 172.21.15.45:40152) 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 -- 172.21.15.45:0/3333737233 learned_addr learned my addr 172.21.15.45:0/3333737233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f3965cef640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af4b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39654ee640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3aea0 0x56007af4b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 -- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3aea0 msgr2=0x56007af4b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3aea0 0x56007af4b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 -- 172.21.15.45:0/3333737233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 msgr2=0x56007af4b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af4b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 -- 172.21.15.45:0/3333737233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007ac7d7d0 con 0x56007af45570 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f3965cef640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af4b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:34.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39664f0640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3958005cc0 tx=0x7f39580bc730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af621f43e8858ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.447+0000 7f39497fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580bde60 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f39580edfb0 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3950011bd0 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3958087e50 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56007afbbea0 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3950011bd0 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f395804d030 con 0x56007af45570 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 20 client.15891 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007afbc5c0 0x56007afbe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 4 client.15891 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:34.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.448+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56007aea6f00 con 0x56007afbc5c0 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f3965cef640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007afbc5c0 0x56007afbe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f3965cef640 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007afbc5c0 0x56007afbe9e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dba7c60dd8dbdbd9 server_cookie=a6a9519fd1b8eebf in_seq=0 out_seq=0 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f39497fa640 10 client.15891 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f39497fa640 1 -- 172.21.15.45:0/3333737233 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56007aea6f00 con 0x56007afbc5c0 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f39497fa640 10 client.15891 handle_command_reply: tid=0 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 client.15891 shutdown 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 msgr2=0x56007af4bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3958005cc0 tx=0x7f39580bc730 comp rx=0 tx=0).stop 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 shutdown_connections 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56007afbc5c0 0x56007afbe9e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56007af45570 0x56007af4bf10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f39497fa640 0 client.15891 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56007af3aea0 0x56007af4b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 --2- 172.21.15.45:0/3333737233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56007af3a580 0x56007af4b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.449+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 >> 172.21.15.45:0/3333737233 conn(0x56007af368a0 msgr2=0x56007af50210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 shutdown_connections 2024-04-25T00:22:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f396ac5d740 1 -- 172.21.15.45:0/3333737233 wait complete. 2024-04-25T00:22:34.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f396ac5d740 20 client.15891 trim_cache size 0 max 16384 2024-04-25T00:22:34.457 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 msgr2=0x7f394c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 0x7f394c0a5250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3950013990 tx=0x7f395000bb70 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 msgr2=0x7f396010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3958009ca0 tx=0x7f3958030ee0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 shutdown_connections 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f394c0a2d90 0x7f394c0a5250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3960106630 0x7f396010ea00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3960100fe0 0x7f396010e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 --2- 172.21.15.45:0/716376311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3960100620 0x7f396010df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 >> 172.21.15.45:0/716376311 conn(0x7f39600776a0 msgr2=0x7f39601030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 shutdown_connections 2024-04-25T00:22:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.450+0000 7f3967f53640 1 -- 172.21.15.45:0/716376311 wait complete. 2024-04-25T00:22:34.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:22:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- 172.21.15.45:0/3610447719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f810bd00 msgr2=0x7fe1f810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3610447719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f810bd00 0x7fe1f810e1a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009ca0 tx=0x7fe1f002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- 172.21.15.45:0/3610447719 shutdown_connections 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3610447719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f810bd00 0x7fe1f810e1a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3610447719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f81032e0 0x7fe1f810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3610447719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81029c0 0x7fe1f8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- 172.21.15.45:0/3610447719 >> 172.21.15.45:0/3610447719 conn(0x7fe1f8077740 msgr2=0x7fe1f8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- 172.21.15.45:0/3610447719 shutdown_connections 2024-04-25T00:22:34.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- 172.21.15.45:0/3610447719 wait complete. 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 Processor -- start 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.566+0000 7fe1fca91640 1 -- start start 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 0x7fe1f81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 0x7fe1f819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111b00 con 0x7fe1f81029c0 2024-04-25T00:22:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111980 con 0x7fe1f81032e0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111c80 con 0x7fe1f810bd00 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40166/0 (socket says 172.21.15.45:40166) 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 -- 172.21.15.45:0/3325210997 learned_addr learned my addr 172.21.15.45:0/3325210997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 0x7fe1f81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 0x7fe1f819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 -- 172.21.15.45:0/3325210997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 msgr2=0x7fe1f819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 0x7fe1f819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 -- 172.21.15.45:0/3325210997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 msgr2=0x7fe1f81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 0x7fe1f81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 -- 172.21.15.45:0/3325210997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f819cf90 con 0x7fe1f81029c0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 0x7fe1f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 0x7fe1f819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800e9f0 tx=0x7fe1e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1e77fe640 1 -- 172.21.15.45:0/3325210997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800cd80 con 0x7fe1f81029c0 2024-04-25T00:22:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1e77fe640 1 -- 172.21.15.45:0/3325210997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e8010910 con 0x7fe1f81029c0 2024-04-25T00:22:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.567+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f819d280 con 0x7fe1f81029c0 2024-04-25T00:22:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.568+0000 7fe1e77fe640 1 -- 172.21.15.45:0/3325210997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8018990 con 0x7fe1f81029c0 2024-04-25T00:22:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.568+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81a4b10 con 0x7fe1f81029c0 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.570+0000 7fe1e77fe640 1 -- 172.21.15.45:0/3325210997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fe1e8010ac0 con 0x7fe1f81029c0 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.570+0000 7fe1ff795740 1 Processor -- start 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.570+0000 7fe1ff795740 1 -- start start 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.570+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.570+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1ff795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b39e41670 con 0x562b39edc580 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1ff795740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b39de9970 con 0x562b39ee7570 2024-04-25T00:22:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1ff795740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b39e3d630 con 0x562b39edcea0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1e77fe640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 0x7fe1e00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40182/0 (socket says 172.21.15.45:40182) 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 -- 172.21.15.45:0/395338526 learned_addr learned my addr 172.21.15.45:0/395338526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 0x7fe1e00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f659c640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1e77fe640 1 -- 172.21.15.45:0/3325210997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe1e80cfda0 con 0x7fe1f81029c0 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 -- 172.21.15.45:0/395338526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 msgr2=0x562b39ee7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 -- 172.21.15.45:0/395338526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 msgr2=0x562b39ee9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 -- 172.21.15.45:0/395338526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b39e0d400 con 0x562b39edc580 2024-04-25T00:22:34.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003cc0 tx=0x7fe1f0003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3955f7cf90a98e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 0x7fe1e00a5250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003d60 tx=0x7fe1f0033000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.571+0000 7fe1e4ff9640 1 -- 172.21.15.45:0/395338526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0033070 con 0x562b39edc580 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1e4ff9640 1 -- 172.21.15.45:0/395338526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f00363c0 con 0x562b39edc580 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1e4ff9640 1 -- 172.21.15.45:0/395338526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0041040 con 0x562b39edc580 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 -- 172.21.15.45:0/395338526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 msgr2=0x562b39edc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003cc0 tx=0x7fe1f0003cf0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 -- 172.21.15.45:0/395338526 shutdown_connections 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 --2- 172.21.15.45:0/395338526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39edc960 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 -- 172.21.15.45:0/395338526 >> 172.21.15.45:0/395338526 conn(0x562b39ed88a0 msgr2=0x562b39ed8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 -- 172.21.15.45:0/395338526 shutdown_connections 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.572+0000 7fe1ff795740 1 -- 172.21.15.45:0/395338526 wait complete. 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 -- start start 2024-04-25T00:22:34.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 Processor -- start 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b39eea300 con 0x562b39edc580 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b39dfd050 con 0x562b39ee7570 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b39eea480 con 0x562b39edcea0 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1ff795740 10 client.0 _ll_register_callbacks cb 0x562b39b54be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 -- 172.21.15.45:0/3035779418 learned_addr learned my addr 172.21.15.45:0/3035779418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 -- 172.21.15.45:0/3035779418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 msgr2=0x562b39ee4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 -- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 msgr2=0x562b39ee4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f659c640 1 -- 172.21.15.45:0/3035779418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b39dcedc0 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.573+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1f659c640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe1e809bd80 tx=0x7fe1e801da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de298620f7a71ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80cd580 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe1e8018b30 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b39e4ebb0 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1e80a4a60 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 10 client.15903 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b39b553f0 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b39e4ebb0 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e8092030 con 0x562b39edc580 2024-04-25T00:22:34.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 20 client.15903 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39ef7670 0x562b39ef9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 4 client.15903 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.574+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b39e48f00 con 0x562b39ef7670 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.575+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39ef7670 0x562b39ef9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.575+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39ef7670 0x562b39ef9aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81baecd5a73c06d4 server_cookie=7fe1dc048ae1f3de in_seq=0 out_seq=0 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.575+0000 7fe1ddffb640 10 client.15903 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ddffb640 1 -- 172.21.15.45:0/3035779418 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b39e48f00 con 0x562b39ef7670 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ddffb640 10 client.15903 handle_command_reply: tid=0 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 client.15903 shutdown 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 msgr2=0x562b39ee6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe1e809bd80 tx=0x7fe1e801da00 comp rx=0 tx=0).stop 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 shutdown_connections 2024-04-25T00:22:34.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39ee4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edcea0 0x562b39ee4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39ef7670 0x562b39ef9aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 --2- 172.21.15.45:0/3035779418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ee6f80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 >> 172.21.15.45:0/3035779418 conn(0x562b39ed88a0 msgr2=0x562b39ede6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ddffb640 0 client.15903 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 shutdown_connections 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 1 -- 172.21.15.45:0/3035779418 wait complete. 2024-04-25T00:22:34.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.576+0000 7fe1ff795740 20 client.15903 trim_cache size 0 max 16384 2024-04-25T00:22:34.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 Processor -- start 2024-04-25T00:22:34.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 -- start start 2024-04-25T00:22:34.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39edcea0 0x562b39ee6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 0x562b39eff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81aeb30 con 0x562b39edc580 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b0670 con 0x562b39edcea0 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.589+0000 7fe1ff795740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81af200 con 0x562b39ee7570 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40200/0 (socket says 172.21.15.45:40200) 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 -- 172.21.15.45:0/1273265670 learned_addr learned my addr 172.21.15.45:0/1273265670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39edcea0 0x562b39ee6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 0x562b39eff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 -- 172.21.15.45:0/1273265670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 msgr2=0x562b39eff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 0x562b39eff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 -- 172.21.15.45:0/1273265670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39edcea0 msgr2=0x562b39ee6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39edcea0 0x562b39ee6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 -- 172.21.15.45:0/1273265670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b39edccd0 con 0x562b39edc580 2024-04-25T00:22:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 0x562b39eff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1f659c640 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe1e809baf0 tx=0x7fe1e80cd1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db9dd76f40eac03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1dffff640 1 -- 172.21.15.45:0/1273265670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8025170 con 0x562b39edc580 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1dffff640 1 -- 172.21.15.45:0/1273265670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e8097dc0 con 0x562b39edc580 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.590+0000 7fe1dffff640 1 -- 172.21.15.45:0/1273265670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8018e20 con 0x562b39edc580 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- 172.21.15.45:0/1273265670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 msgr2=0x562b39ef31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe1e809baf0 tx=0x7fe1e80cd1b0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- 172.21.15.45:0/1273265670 shutdown_connections 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39ee7570 0x562b39eff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39edcea0 0x562b39ee6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- 172.21.15.45:0/1273265670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edc580 0x562b39ef31c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- 172.21.15.45:0/1273265670 >> 172.21.15.45:0/1273265670 conn(0x562b39ed88a0 msgr2=0x562b39edf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- 172.21.15.45:0/1273265670 shutdown_connections 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- 172.21.15.45:0/1273265670 wait complete. 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- start start 2024-04-25T00:22:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 Processor -- start 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edc580 0x562b39eef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39eeab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81affa0 con 0x562b39edcea0 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81af8d0 con 0x562b39ee7570 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81acb10 con 0x562b39edc580 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.591+0000 7fe1ff795740 10 client.0 _ll_register_callbacks cb 0x7fe1e80980d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40210/0 (socket says 172.21.15.45:40210) 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 -- 172.21.15.45:0/2793986364 learned_addr learned my addr 172.21.15.45:0/2793986364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f659c640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edc580 0x562b39eef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39eeab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 -- 172.21.15.45:0/2793986364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edc580 msgr2=0x562b39eef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edc580 0x562b39eef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 -- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 msgr2=0x562b39eeab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39eeab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 -- 172.21.15.45:0/2793986364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b39c1f7d0 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f6d9d640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39eeab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1f5d9b640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x562b39ed9a30 tx=0x7fe1ec004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f959f793205f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0037f0 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe1ec003990 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e8097dc0 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1ec02e050 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ff795740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b39f5d5b0 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.592+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e8097dc0 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1ec00f040 con 0x562b39edcea0 2024-04-25T00:22:34.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 20 client.15909 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39f5dd00 0x562b39f60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 4 client.15909 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b39dc6820 con 0x562b39f5dd00 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1f659c640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39f5dd00 0x562b39f60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1f659c640 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39f5dd00 0x562b39f60120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e212c64af50f1f1 server_cookie=2045f2f99f8e8d06 in_seq=0 out_seq=0 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ddffb640 10 client.15909 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ddffb640 1 -- 172.21.15.45:0/2793986364 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b39dc6820 con 0x562b39f5dd00 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ddffb640 10 client.15909 handle_command_reply: tid=0 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.593+0000 7fe1ff795740 1 client.15909 shutdown 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 msgr2=0x562b39eefb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x562b39ed9a30 tx=0x7fe1ec004410 comp rx=0 tx=0).stop 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 shutdown_connections 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b39ee7570 0x562b39eeab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562b39f5dd00 0x562b39f60120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b39edcea0 0x562b39eefb50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 --2- 172.21.15.45:0/2793986364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b39edc580 0x562b39eef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 >> 172.21.15.45:0/2793986364 conn(0x562b39ed88a0 msgr2=0x562b39eddcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ddffb640 0 client.15909 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 shutdown_connections 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 1 -- 172.21.15.45:0/2793986364 wait complete. 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1ff795740 20 client.15909 trim_cache size 0 max 16384 2024-04-25T00:22:34.601 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:22:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 msgr2=0x7fe1e00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 0x7fe1e00a5250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003d60 tx=0x7fe1f0033000 comp rx=0 tx=0).stop 2024-04-25T00:22:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 msgr2=0x7fe1f8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800e9f0 tx=0x7fe1e800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 shutdown_connections 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1e00a2d90 0x7fe1e00a5250 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f810bd00 0x7fe1f819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f81032e0 0x7fe1f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 --2- 172.21.15.45:0/3325210997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f81029c0 0x7fe1f8197fa0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 >> 172.21.15.45:0/3325210997 conn(0x7fe1f8077740 msgr2=0x7fe1f80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 shutdown_connections 2024-04-25T00:22:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:34.594+0000 7fe1fca91640 1 -- 172.21.15.45:0/3325210997 wait complete. 2024-04-25T00:22:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:34 smithi045 ceph-mon[29194]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:22:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:34 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1300700900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:35.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:34 smithi122 ceph-mon[32959]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:22:35.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:34 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1300700900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:34 smithi071 ceph-mon[32667]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T00:22:35.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:34 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1300700900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:35.607 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:22:35.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:22:35.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:22:35.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 -- 172.21.15.45:0/935085270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 msgr2=0x7f0c44101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 --2- 172.21.15.45:0/935085270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c44101be0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009930 tx=0x7f0c3802a420 comp rx=0 tx=0).stop 2024-04-25T00:22:35.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 -- 172.21.15.45:0/935085270 shutdown_connections 2024-04-25T00:22:35.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 --2- 172.21.15.45:0/935085270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 0x7f0c4410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 --2- 172.21.15.45:0/935085270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c44101be0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 --2- 172.21.15.45:0/935085270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 0x7f0c44107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 -- 172.21.15.45:0/935085270 >> 172.21.15.45:0/935085270 conn(0x7f0c440fd250 msgr2=0x7f0c440ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 -- 172.21.15.45:0/935085270 shutdown_connections 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.714+0000 7f0c4a876640 1 -- 172.21.15.45:0/935085270 wait complete. 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 Processor -- start 2024-04-25T00:22:35.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 -- start start 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 0x7f0c44198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 0x7f0c4419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44111b80 con 0x7f0c44101760 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44111a00 con 0x7f0c44107770 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c4a876640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44111d00 con 0x7f0c44102120 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40218/0 (socket says 172.21.15.45:40218) 2024-04-25T00:22:35.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 -- 172.21.15.45:0/1970813902 learned_addr learned my addr 172.21.15.45:0/1970813902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 -- 172.21.15.45:0/1970813902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 msgr2=0x7f0c44198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c437fe640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 0x7f0c44198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 0x7f0c44198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c48e13640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 0x7f0c4419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 -- 172.21.15.45:0/1970813902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 msgr2=0x7f0c4419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.715+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 0x7f0c4419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c43fff640 1 -- 172.21.15.45:0/1970813902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4419d0e0 con 0x7f0c44101760 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9f0 tx=0x7f0c3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c417fa640 1 -- 172.21.15.45:0/1970813902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400cd80 con 0x7f0c44101760 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4419d3d0 con 0x7f0c44101760 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c417fa640 1 -- 172.21.15.45:0/1970813902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c34010910 con 0x7f0c44101760 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c441a4c60 con 0x7f0c44101760 2024-04-25T00:22:35.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.716+0000 7f0c417fa640 1 -- 172.21.15.45:0/1970813902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c340188f0 con 0x7f0c44101760 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c417fa640 1 -- 172.21.15.45:0/1970813902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f0c34018a90 con 0x7f0c44101760 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 Processor -- start 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 -- start start 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd75980 0x56056bd73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bd85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56056bcd82c0 con 0x56056bd74290 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56056bc80910 con 0x56056bd75980 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c4d585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56056bcd4280 con 0x56056bd83420 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.719+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40220/0 (socket says 172.21.15.45:40220) 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 -- 172.21.15.45:0/1968407937 learned_addr learned my addr 172.21.15.45:0/1968407937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c417fa640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 0x7f0c2c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c437fe640 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bd85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c48e13640 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd75980 0x56056bd73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c48e13640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 0x7f0c2c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c417fa640 1 -- 172.21.15.45:0/1970813902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f0c340cfde0 con 0x7f0c44101760 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 -- 172.21.15.45:0/1968407937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 msgr2=0x56056bd85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bd85810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 -- 172.21.15.45:0/1968407937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd75980 msgr2=0x56056bd73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd75980 0x56056bd73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 -- 172.21.15.45:0/1968407937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056bc91fb0 con 0x56056bd74290 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0c3409bcf0 tx=0x7f0c34010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916912f84d170608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c48e13640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 0x7f0c2c0a5250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005fd0 tx=0x7f0c3c009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1968407937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c340a4aa0 con 0x56056bd74290 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1968407937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3409a2f0 con 0x56056bd74290 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.720+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1968407937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34097fa0 con 0x56056bd74290 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- 172.21.15.45:0/1968407937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 msgr2=0x56056bd74710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0c3409bcf0 tx=0x7f0c34010ac0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- 172.21.15.45:0/1968407937 shutdown_connections 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bd85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd74710 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1968407937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd75980 0x56056bd73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- 172.21.15.45:0/1968407937 >> 172.21.15.45:0/1968407937 conn(0x56056bd3c950 msgr2=0x56056bd707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- 172.21.15.45:0/1968407937 shutdown_connections 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- 172.21.15.45:0/1968407937 wait complete. 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.721+0000 7f0c4d585740 1 -- start start 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 Processor -- start 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd74290 0x56056bd80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd80dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56056bd861f0 con 0x56056bd75980 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56056bc93e90 con 0x56056bd83420 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56056bd86370 con 0x56056bd74290 2024-04-25T00:22:35.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40230/0 (socket says 172.21.15.45:40230) 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 -- 172.21.15.45:0/4111386941 learned_addr learned my addr 172.21.15.45:0/4111386941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c4d585740 10 client.0 _ll_register_callbacks cb 0x56056bccf970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd74290 0x56056bd80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c48e13640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd80dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 -- 172.21.15.45:0/4111386941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd74290 msgr2=0x56056bd80340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd74290 0x56056bd80340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 -- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 msgr2=0x56056bd80dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd80dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.722+0000 7f0c437fe640 1 -- 172.21.15.45:0/4111386941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056bab47e0 con 0x56056bd75980 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c437fe640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009930 tx=0x7f0c38032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9e58a6374f5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c237fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:35.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38035040 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f0c380319c0 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c3409a2f0 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c3804c050 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c4d585740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56056bb14f70 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.723+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3409a2f0 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c38004750 con 0x56056bd75980 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 20 client.15921 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bd93480 0x56056bd958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 4 client.15921 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56056bcac5a0 con 0x56056bd93480 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.724+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bd93480 0x56056bd958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.725+0000 7f0c237fe640 10 client.15921 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.725+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bd93480 0x56056bd958f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad5de69b2a4f302 server_cookie=13ff8a280b90bdba in_seq=0 out_seq=0 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.725+0000 7f0c237fe640 1 -- 172.21.15.45:0/4111386941 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56056bcac5a0 con 0x56056bd93480 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.725+0000 7f0c237fe640 10 client.15921 handle_command_reply: tid=0 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.725+0000 7f0c4d585740 1 client.15921 shutdown 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 msgr2=0x56056bd80880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009930 tx=0x7f0c38032e60 comp rx=0 tx=0).stop 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 shutdown_connections 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd80dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bd93480 0x56056bd958f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd80880 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4111386941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd74290 0x56056bd80340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 >> 172.21.15.45:0/4111386941 conn(0x56056bd3c950 msgr2=0x56056bd76660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c237fe640 0 client.15921 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 shutdown_connections 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 1 -- 172.21.15.45:0/4111386941 wait complete. 2024-04-25T00:22:35.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.726+0000 7f0c4d585740 20 client.15921 trim_cache size 0 max 16384 2024-04-25T00:22:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 Processor -- start 2024-04-25T00:22:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 -- start start 2024-04-25T00:22:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd75980 0x56056bd6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441aca50 con 0x56056bd74290 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441ae590 con 0x56056bd83420 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c4d585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441ad120 con 0x56056bd75980 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40236/0 (socket says 172.21.15.45:40236) 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 -- 172.21.15.45:0/4152202189 learned_addr learned my addr 172.21.15.45:0/4152202189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 -- 172.21.15.45:0/4152202189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd75980 msgr2=0x56056bd6f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd75980 0x56056bd6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 -- 172.21.15.45:0/4152202189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 msgr2=0x56056bd9b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 -- 172.21.15.45:0/4152202189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056b9ac170 con 0x56056bd74290 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.744+0000 7f0c43fff640 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c340cd1f0 tx=0x7f0c34098390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2cd8ee75fb2f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c29ffb640 1 -- 172.21.15.45:0/4152202189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c340cd3c0 con 0x56056bd74290 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c29ffb640 1 -- 172.21.15.45:0/4152202189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c340cd560 con 0x56056bd74290 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c29ffb640 1 -- 172.21.15.45:0/4152202189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3409b110 con 0x56056bd74290 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 -- 172.21.15.45:0/4152202189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 msgr2=0x56056bd8f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0c340cd1f0 tx=0x7f0c34098390 comp rx=0 tx=0).stop 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 -- 172.21.15.45:0/4152202189 shutdown_connections 2024-04-25T00:22:35.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd83420 0x56056bd9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd75980 0x56056bd6f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 --2- 172.21.15.45:0/4152202189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd74290 0x56056bd8f0f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 -- 172.21.15.45:0/4152202189 >> 172.21.15.45:0/4152202189 conn(0x56056bd3c950 msgr2=0x56056bd6f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 -- 172.21.15.45:0/4152202189 shutdown_connections 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.745+0000 7f0c4d585740 1 -- 172.21.15.45:0/4152202189 wait complete. 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 -- start start 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 Processor -- start 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 0x56056bd7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bdf8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441adec0 con 0x56056bd75980 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441ad7f0 con 0x56056bd74290 2024-04-25T00:22:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c441aaa30 con 0x56056bd83420 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c4d585740 10 client.0 _ll_register_callbacks cb 0x7f0c3c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40242/0 (socket says 172.21.15.45:40242) 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c437fe640 1 -- 172.21.15.45:0/1902704745 learned_addr learned my addr 172.21.15.45:0/1902704745 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c48e13640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bdf8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.746+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 0x56056bd7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 -- 172.21.15.45:0/1902704745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 msgr2=0x56056bdf8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bdf8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 -- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 msgr2=0x56056bd7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 0x56056bd7d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 -- 172.21.15.45:0/1902704745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56056bdf8f70 con 0x56056bd75980 2024-04-25T00:22:35.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 0x56056bd7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c437fe640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47680c12e27b745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38035070 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f0c38004030 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c340cd560 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c3805e050 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c4d585740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56056bdfa0b0 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c340cd560 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.747+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c380317c0 con 0x56056bd75980 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:35.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 20 client.15927 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bdfa8b0 0x56056bdfccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 4 client.15927 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56056bce5800 con 0x56056bdfa8b0 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bdfa8b0 0x56056bdfccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c43fff640 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bdfa8b0 0x56056bdfccd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0a88e5ff1aa97e server_cookie=267f2c4ab46d97d2 in_seq=0 out_seq=0 2024-04-25T00:22:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.748+0000 7f0c237fe640 10 client.15927 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c237fe640 1 -- 172.21.15.45:0/1902704745 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56056bce5800 con 0x56056bdfa8b0 2024-04-25T00:22:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c237fe640 10 client.15927 handle_command_reply: tid=0 2024-04-25T00:22:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 client.15927 shutdown 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 msgr2=0x56056bd7d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3803f000 comp rx=0 tx=0).stop 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 shutdown_connections 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56056bd83420 0x56056bdf8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56056bdfa8b0 0x56056bdfccd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56056bd75980 0x56056bd7d880 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c237fe640 0 client.15927 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 --2- 172.21.15.45:0/1902704745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56056bd74290 0x56056bd7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 >> 172.21.15.45:0/1902704745 conn(0x56056bd3c950 msgr2=0x56056bd6f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 shutdown_connections 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 1 -- 172.21.15.45:0/1902704745 wait complete. 2024-04-25T00:22:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.750+0000 7f0c4d585740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T00:22:35.758 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:22:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:22:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f0793b19-5a52-4e0e-a3b0-6badcd644c77", 2024-04-25T00:22:35.758 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:22:35.758 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:22:35.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 msgr2=0x7f0c2c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 0x7f0c2c0a5250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005fd0 tx=0x7f0c3c009500 comp rx=0 tx=0).stop 2024-04-25T00:22:35.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 msgr2=0x7f0c441980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9f0 tx=0x7f0c3400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 shutdown_connections 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0c2c0a2d90 0x7f0c2c0a5250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c44107770 0x7f0c4419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c44102120 0x7f0c44198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 --2- 172.21.15.45:0/1970813902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c44101760 0x7f0c441980f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 >> 172.21.15.45:0/1970813902 conn(0x7f0c440fd250 msgr2=0x7f0c440fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 shutdown_connections 2024-04-25T00:22:35.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.751+0000 7f0c4a876640 1 -- 172.21.15.45:0/1970813902 wait complete. 2024-04-25T00:22:35.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:22:35.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- 172.21.15.45:0/3252852973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f762810bd00 msgr2=0x7f762810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 --2- 172.21.15.45:0/3252852973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f762810bd00 0x7f762810e1a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f762002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- 172.21.15.45:0/3252852973 shutdown_connections 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 --2- 172.21.15.45:0/3252852973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f762810bd00 0x7f762810e1a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 --2- 172.21.15.45:0/3252852973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f762810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 --2- 172.21.15.45:0/3252852973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76281029c0 0x7f7628102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- 172.21.15.45:0/3252852973 >> 172.21.15.45:0/3252852973 conn(0x7f7628077740 msgr2=0x7f7628077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- 172.21.15.45:0/3252852973 shutdown_connections 2024-04-25T00:22:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- 172.21.15.45:0/3252852973 wait complete. 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 Processor -- start 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.871+0000 7f762f370640 1 -- start start 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f7628198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f762810bd00 0x7f762819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7628111b00 con 0x7f76281029c0 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7628111980 con 0x7f76281032e0 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7628111c80 con 0x7f762810bd00 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40246/0 (socket says 172.21.15.45:40246) 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 -- 172.21.15.45:0/918048116 learned_addr learned my addr 172.21.15.45:0/918048116 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762c90b640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f7628198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d90d640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f762810bd00 0x7f762819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 -- 172.21.15.45:0/918048116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f762810bd00 msgr2=0x7f762819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f762810bd00 0x7f762819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 -- 172.21.15.45:0/918048116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 msgr2=0x7f7628198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f7628198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 -- 172.21.15.45:0/918048116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762819cf30 con 0x7f76281029c0 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762c90b640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f7628198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762d10c640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9f0 tx=0x7f762400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f761e7fc640 1 -- 172.21.15.45:0/918048116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400cd80 con 0x7f76281029c0 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762819d220 con 0x7f76281029c0 2024-04-25T00:22:35.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f761e7fc640 1 -- 172.21.15.45:0/918048116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624005500 con 0x7f76281029c0 2024-04-25T00:22:35.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.872+0000 7f761e7fc640 1 -- 172.21.15.45:0/918048116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624010690 con 0x7f76281029c0 2024-04-25T00:22:35.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.873+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a4ab0 con 0x7f76281029c0 2024-04-25T00:22:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.875+0000 7f761e7fc640 1 -- 172.21.15.45:0/918048116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f76240046c0 con 0x7f76281029c0 2024-04-25T00:22:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.875+0000 7f7632082740 1 Processor -- start 2024-04-25T00:22:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.875+0000 7f7632082740 1 -- start start 2024-04-25T00:22:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f761e7fc640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037f50 0x55d6670420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f761e7fc640 1 -- 172.21.15.45:0/918048116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f76240cfe90 con 0x7f76281029c0 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d667044a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d666f9c6b0 con 0x55d667042620 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d666f44880 con 0x55d667037f50 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f7632082740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d666f98670 con 0x55d667037630 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d90d640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d90d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d667044a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762c90b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037f50 0x55d6670420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:39824/0 (socket says 172.21.15.45:39824) 2024-04-25T00:22:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d90d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d667044a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40256/0 (socket says 172.21.15.45:40256) 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 -- 172.21.15.45:0/4284098414 learned_addr learned my addr 172.21.15.45:0/4284098414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 -- 172.21.15.45:0/4284098414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037f50 msgr2=0x55d6670420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037f50 0x55d6670420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 -- 172.21.15.45:0/4284098414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 msgr2=0x55d667044a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d667044a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d10c640 1 -- 172.21.15.45:0/4284098414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d666d7a7d0 con 0x55d667037630 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.876+0000 7f762d90d640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 0x7f76140a5250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f762002ec10 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.877+0000 7f762d10c640 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9c0 tx=0x7f762409bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6caccb4f55fe2b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.877+0000 7f76137fe640 1 -- 172.21.15.45:0/4284098414 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409bee0 con 0x55d667037630 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.877+0000 7f76137fe640 1 -- 172.21.15.45:0/4284098414 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76240a0290 con 0x55d667037630 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.877+0000 7f76137fe640 1 -- 172.21.15.45:0/4284098414 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240a46c0 con 0x55d667037630 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- 172.21.15.45:0/4284098414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 msgr2=0x55d667037a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9c0 tx=0x7f762409bae0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- 172.21.15.45:0/4284098414 shutdown_connections 2024-04-25T00:22:35.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d667044a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037f50 0x55d6670420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 --2- 172.21.15.45:0/4284098414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037630 0x55d667037a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- 172.21.15.45:0/4284098414 >> 172.21.15.45:0/4284098414 conn(0x55d667033950 msgr2=0x55d667033d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- 172.21.15.45:0/4284098414 shutdown_connections 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- 172.21.15.45:0/4284098414 wait complete. 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 -- start start 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.878+0000 7f7632082740 1 Processor -- start 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66703f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667040010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d667045410 con 0x55d667037f50 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d666f57f50 con 0x55d667037630 2024-04-25T00:22:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d667045590 con 0x55d667042620 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f7632082740 10 client.0 _ll_register_callbacks cb 0x55d666f93000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40268/0 (socket says 172.21.15.45:40268) 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 -- 172.21.15.45:0/2917221214 learned_addr learned my addr 172.21.15.45:0/2917221214 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762d10c640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66703f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762d90d640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667040010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 -- 172.21.15.45:0/2917221214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 msgr2=0x55d667040010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667040010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 -- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 msgr2=0x55d66703f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66703f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.879+0000 7f762c90b640 1 -- 172.21.15.45:0/2917221214 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d666cb03f0 con 0x55d667037f50 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f762c90b640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7618011830 tx=0x7f7618011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44213098ad776210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800f9a0 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f761800fb40 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76240a0290 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f761800fd40 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d666c700f0 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76240a0290 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7618018020 con 0x55d667037f50 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 20 client.15939 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d667054d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 4 client.15939 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.880+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d666fa9bf0 con 0x55d667052930 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.881+0000 7f762d10c640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d667054d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.881+0000 7f762d10c640 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d667054d80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fda4a5ee92a8bf9 server_cookie=6a9e65025f7b2fd5 in_seq=0 out_seq=0 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.881+0000 7f7610ff9640 10 client.15939 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7610ff9640 1 -- 172.21.15.45:0/2917221214 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d666fa9bf0 con 0x55d667052930 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7610ff9640 10 client.15939 handle_command_reply: tid=0 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 client.15939 shutdown 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 msgr2=0x55d66703fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7618011830 tx=0x7f7618011cf0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 shutdown_connections 2024-04-25T00:22:35.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667040010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d667054d80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703fad0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7610ff9640 0 client.15939 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 --2- 172.21.15.45:0/2917221214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66703f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 >> 172.21.15.45:0/2917221214 conn(0x55d667033950 msgr2=0x55d667039750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 shutdown_connections 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 1 -- 172.21.15.45:0/2917221214 wait complete. 2024-04-25T00:22:35.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.882+0000 7f7632082740 20 client.15939 trim_cache size 0 max 16384 2024-04-25T00:22:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.913+0000 7f7632082740 1 Processor -- start 2024-04-25T00:22:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 -- start start 2024-04-25T00:22:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667039ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76281aea90 con 0x55d667037f50 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76281b05d0 con 0x55d667037630 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7632082740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76281af160 con 0x55d667042620 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40274/0 (socket says 172.21.15.45:40274) 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 -- 172.21.15.45:0/2687344102 learned_addr learned my addr 172.21.15.45:0/2687344102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762d10c640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762d90d640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667039ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 -- 172.21.15.45:0/2687344102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 msgr2=0x55d667039ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667039ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 -- 172.21.15.45:0/2687344102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 msgr2=0x55d66704e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 -- 172.21.15.45:0/2687344102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d667025090 con 0x55d667037f50 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762d10c640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f762c90b640 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f761801a750 tx=0x7f7618008640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e630732957e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.914+0000 7f7612ffd640 1 -- 172.21.15.45:0/2687344102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761802c070 con 0x55d667037f50 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7612ffd640 1 -- 172.21.15.45:0/2687344102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761800f790 con 0x55d667037f50 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7612ffd640 1 -- 172.21.15.45:0/2687344102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180038c0 con 0x55d667037f50 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 -- 172.21.15.45:0/2687344102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 msgr2=0x55d66703e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f761801a750 tx=0x7f7618008640 comp rx=0 tx=0).stop 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 -- 172.21.15.45:0/2687344102 shutdown_connections 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667042620 0x55d667039ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667037f50 0x55d66703e4d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 --2- 172.21.15.45:0/2687344102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 -- 172.21.15.45:0/2687344102 >> 172.21.15.45:0/2687344102 conn(0x55d667033950 msgr2=0x55d66703fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 -- 172.21.15.45:0/2687344102 shutdown_connections 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.915+0000 7f7632082740 1 -- 172.21.15.45:0/2687344102 wait complete. 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 -- start start 2024-04-25T00:22:35.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 Processor -- start 2024-04-25T00:22:35.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037f50 0x55d667045da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76281aff00 con 0x55d667042620 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76281af830 con 0x55d667037630 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76281aca70 con 0x55d667037f50 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f7632082740 10 client.0 _ll_register_callbacks cb 0x7f7618008760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f762d90d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f762d90d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40276/0 (socket says 172.21.15.45:40276) 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f762d90d640 1 -- 172.21.15.45:0/1565459095 learned_addr learned my addr 172.21.15.45:0/1565459095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f762c90b640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037f50 0x55d667045da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.916+0000 7f762d10c640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 -- 172.21.15.45:0/1565459095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037f50 msgr2=0x55d667045da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037f50 0x55d667045da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 -- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 msgr2=0x55d66704a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 -- 172.21.15.45:0/1565459095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6670468a0 con 0x55d667042620 2024-04-25T00:22:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f762d90d640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f762002e5c0 tx=0x7f7620044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519a353b7fd23000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620033050 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7620041040 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f761800f790 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f762005f050 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7632082740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6670b8ce0 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.917+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761800f790 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76200365d0 con 0x55d667042620 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 20 client.15945 populate_metadata read hostname 'smithi045' 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d6670bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 4 client.15945 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d666fa3f40 con 0x55d667052930 2024-04-25T00:22:35.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f762d10c640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d6670bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7610ff9640 10 client.15945 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f762d10c640 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d6670bb550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf35d53651ad5694 server_cookie=7345c76c0739a7b4 in_seq=0 out_seq=0 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7610ff9640 1 -- 172.21.15.45:0/1565459095 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d666fa3f40 con 0x55d667052930 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7610ff9640 10 client.15945 handle_command_reply: tid=0 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.918+0000 7f7632082740 1 client.15945 shutdown 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 msgr2=0x55d6670462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f762002e5c0 tx=0x7f7620044000 comp rx=0 tx=0).stop 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 shutdown_connections 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d667052930 0x55d6670bb550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d667042620 0x55d6670462e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d667037f50 0x55d667045da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 --2- 172.21.15.45:0/1565459095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d667037630 0x55d66704a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 >> 172.21.15.45:0/1565459095 conn(0x55d667033950 msgr2=0x55d667032260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7610ff9640 0 client.15945 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:22:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 shutdown_connections 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 1 -- 172.21.15.45:0/1565459095 wait complete. 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f7632082740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "f0793b19-5a52-4e0e-a3b0-6badcd644c77": { 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "f0793b19-5a52-4e0e-a3b0-6badcd644c77", 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:22:35.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:22:35.927 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:22:35.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 msgr2=0x7f76140a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 0x7f76140a5250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f762002ec10 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 msgr2=0x7f7628197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9f0 tx=0x7f762400edf0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 shutdown_connections 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f762810bd00 0x7f762819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76281032e0 0x7f7628198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 --2- 172.21.15.45:0/918048116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76281029c0 0x7f7628197f40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.919+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 >> 172.21.15.45:0/918048116 conn(0x7f7628077740 msgr2=0x7f76280fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.920+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 shutdown_connections 2024-04-25T00:22:35.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:22:35.920+0000 7f762f370640 1 -- 172.21.15.45:0/918048116 wait complete. 2024-04-25T00:22:35.931 INFO:tasks.cephfs.filesystem:scrub status for tag:f0793b19-5a52-4e0e-a3b0-6badcd644c77 - {'path': '/', 'tag': 'f0793b19-5a52-4e0e-a3b0-6badcd644c77', 'options': 'recursive,force'} 2024-04-25T00:22:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:36 smithi122 ceph-mon[32959]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:36 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:22:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:36 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:36 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:22:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:36 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:37.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:36 smithi071 ceph-mon[32667]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:37.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:36 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:22:37.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:36 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:37.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:36 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:22:37.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:36 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:36 smithi045 ceph-mon[29194]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:22:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:36 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:22:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:36 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:36 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:22:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:36 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:38.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:37 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:22:38.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:37 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:22:38.148 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:37 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:22:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:38 smithi122 ceph-mon[32959]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:39.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:38 smithi071 ceph-mon[32667]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:38 smithi045 ceph-mon[29194]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:40 smithi122 ceph-mon[32959]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:22:41.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:40 smithi071 ceph-mon[32667]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:22:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:40 smithi045 ceph-mon[29194]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:22:43.052 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:42 smithi122 ceph-mon[32959]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:22:43.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:42 smithi071 ceph-mon[32667]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:22:43.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:42 smithi045 ceph-mon[29194]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:22:45.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:44 smithi122 ceph-mon[32959]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T00:22:45.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:44 smithi071 ceph-mon[32667]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T00:22:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:44 smithi045 ceph-mon[29194]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T00:22:47.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:46 smithi122 ceph-mon[32959]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:46 smithi071 ceph-mon[32667]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:47.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:46 smithi045 ceph-mon[29194]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:49.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:48 smithi122 ceph-mon[32959]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:49.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:48 smithi071 ceph-mon[32667]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:49.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:48 smithi045 ceph-mon[29194]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:50 smithi122 ceph-mon[32959]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T00:22:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:50 smithi071 ceph-mon[32667]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T00:22:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:50 smithi045 ceph-mon[29194]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T00:22:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:52 smithi122 ceph-mon[32959]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:53.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:52 smithi071 ceph-mon[32667]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:52 smithi045 ceph-mon[29194]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T00:22:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:54 smithi122 ceph-mon[32959]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:22:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:54 smithi071 ceph-mon[32667]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:22:55.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:54 smithi045 ceph-mon[29194]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T00:22:57.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:56 smithi122 ceph-mon[32959]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:57.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:56 smithi071 ceph-mon[32667]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:57.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:56 smithi045 ceph-mon[29194]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:22:59.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:22:58 smithi122 ceph-mon[32959]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:22:58 smithi071 ceph-mon[32667]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:22:59.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:22:58 smithi045 ceph-mon[29194]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:23:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:00 smithi122 ceph-mon[32959]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:23:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:00 smithi071 ceph-mon[32667]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:23:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-mon[29194]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:23:01.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:23:00.894+0000 7fe368786700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:23:01.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:23:00.894+0000 7fe368786700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:23:01.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:23:00.894+0000 7fe368786700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:23:01.149 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:23:00.894+0000 7fe368786700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:23:01.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:23:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:23:00.894+0000 7fe368786700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:23:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:02 smithi122 ceph-mon[32959]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:23:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:02 smithi122 ceph-mon[32959]: mgrmap e24: x(active, since 64s), standbys: y, z 2024-04-25T00:23:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:02 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:02 smithi071 ceph-mon[32667]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:23:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:02 smithi071 ceph-mon[32667]: mgrmap e24: x(active, since 64s), standbys: y, z 2024-04-25T00:23:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:02 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:02 smithi045 ceph-mon[29194]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:23:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:02 smithi045 ceph-mon[29194]: mgrmap e24: x(active, since 64s), standbys: y, z 2024-04-25T00:23:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:02 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:04.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:04.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:03 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:04.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:03 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:04.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:04.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:03 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:23:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:04 smithi122 ceph-mon[32959]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-04-25T00:23:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:04 smithi071 ceph-mon[32667]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-04-25T00:23:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:04 smithi045 ceph-mon[29194]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-04-25T00:23:05.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:05.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 -- 172.21.15.45:0/1579376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa81032e0 msgr2=0x7f6fa810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 --2- 172.21.15.45:0/1579376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa81032e0 0x7f6fa810b7c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009900 tx=0x7f6f9c02a510 comp rx=0 tx=0).stop 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 -- 172.21.15.45:0/1579376676 shutdown_connections 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 --2- 172.21.15.45:0/1579376676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa810bd00 0x7f6fa810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 --2- 172.21.15.45:0/1579376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa81032e0 0x7f6fa810b7c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 --2- 172.21.15.45:0/1579376676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 -- 172.21.15.45:0/1579376676 >> 172.21.15.45:0/1579376676 conn(0x7f6fa8077740 msgr2=0x7f6fa8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 -- 172.21.15.45:0/1579376676 shutdown_connections 2024-04-25T00:23:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.046+0000 7f6fae96a640 1 -- 172.21.15.45:0/1579376676 wait complete. 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 Processor -- start 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 -- start start 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 0x7f6fa8100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8111b00 con 0x7f6fa810bd00 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8111980 con 0x7f6fa81029c0 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.047+0000 7f6fae96a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8111c80 con 0x7f6fa81032e0 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49416/0 (socket says 172.21.15.45:49416) 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 0x7f6fa8100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6facf07640 1 -- 172.21.15.45:0/3393555410 learned_addr learned my addr 172.21.15.45:0/3393555410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 -- 172.21.15.45:0/3393555410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 msgr2=0x7f6fa8100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 -- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 msgr2=0x7f6fa8101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 -- 172.21.15.45:0/3393555410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa81a6540 con 0x7f6fa81032e0 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6facf07640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.048+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 0x7f6fa8100b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005940 tx=0x7f6f9c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.049+0000 7f6fa57fa640 1 -- 172.21.15.45:0/3393555410 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c03f030 con 0x7f6fa81032e0 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.049+0000 7f6fa57fa640 1 -- 172.21.15.45:0/3393555410 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c003ad0 con 0x7f6fa81032e0 2024-04-25T00:23:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.049+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa81a67d0 con 0x7f6fa81032e0 2024-04-25T00:23:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.049+0000 7f6fa57fa640 1 -- 172.21.15.45:0/3393555410 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c035070 con 0x7f6fa81032e0 2024-04-25T00:23:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.049+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa81a6c00 con 0x7f6fa81032e0 2024-04-25T00:23:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 Processor -- start 2024-04-25T00:23:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fa57fa640 1 -- 172.21.15.45:0/3393555410 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f6f9c042030 con 0x7f6fa81032e0 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 -- start start 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a25620 0x55e374a27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e37497f6b0 con 0x55e374a1af50 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e374927880 con 0x55e374a1a630 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fb167e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e37497b670 con 0x55e374a25620 2024-04-25T00:23:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49418/0 (socket says 172.21.15.45:49418) 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fa7fff640 1 -- 172.21.15.45:0/567819337 learned_addr learned my addr 172.21.15.45:0/567819337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6facf07640 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.052+0000 7f6fa7fff640 1 -- 172.21.15.45:0/567819337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a25620 msgr2=0x55e374a27a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a25620 0x55e374a27a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa7fff640 1 -- 172.21.15.45:0/567819337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 msgr2=0x55e374a1aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a1aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa7fff640 1 -- 172.21.15.45:0/567819337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37475d7d0 con 0x55e374a1af50 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6facf07640 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800bcc0 tx=0x7f6f9800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989e05dc7aafef54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa57fa640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 0x7f6f900a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6f8e7fc640 1 -- 172.21.15.45:0/567819337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98005020 con 0x55e374a1af50 2024-04-25T00:23:06.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6fa57fa640 1 -- 172.21.15.45:0/3393555410 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6f9c0efff0 con 0x7f6fa81032e0 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6f8e7fc640 1 -- 172.21.15.45:0/567819337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f980047a0 con 0x55e374a1af50 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6f8e7fc640 1 -- 172.21.15.45:0/567819337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980116a0 con 0x55e374a1af50 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.053+0000 7f6facf07640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 0x7f6f900a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- 172.21.15.45:0/567819337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 msgr2=0x55e374a250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800bcc0 tx=0x7f6f9800f730 comp rx=0 tx=0).stop 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6facf07640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 0x7f6f900a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00049d0 tx=0x7f6fa0009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- 172.21.15.45:0/567819337 shutdown_connections 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a25620 0x55e374a27a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1af50 0x55e374a250e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 --2- 172.21.15.45:0/567819337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- 172.21.15.45:0/567819337 >> 172.21.15.45:0/567819337 conn(0x55e374a16950 msgr2=0x55e374a16d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- 172.21.15.45:0/567819337 shutdown_connections 2024-04-25T00:23:06.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- 172.21.15.45:0/567819337 wait complete. 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.054+0000 7f6fb167e740 1 -- start start 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 Processor -- start 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a22ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e374a39420 con 0x55e374a25620 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e37493af50 con 0x55e374a1a630 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e374a395a0 con 0x55e374a1af50 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fb167e740 10 client.0 _ll_register_callbacks cb 0x55e374976000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49430/0 (socket says 172.21.15.45:49430) 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6facf07640 1 -- 172.21.15.45:0/3374221896 learned_addr learned my addr 172.21.15.45:0/3374221896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a22ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.055+0000 7f6facf07640 1 -- 172.21.15.45:0/3374221896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 msgr2=0x55e374a22ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6facf07640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a22ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6facf07640 1 -- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 msgr2=0x55e374a22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6facf07640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6facf07640 1 -- 172.21.15.45:0/3374221896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3746933f0 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6facf07640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0012290 tx=0x7f6fa0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c537166632660644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0011cd0 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6fa0029440 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f980047a0 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0011e70 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6fb167e740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.056+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3746530f0 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f980047a0 con 0x55e374a25620 2024-04-25T00:23:06.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fa0027020 con 0x55e374a25620 2024-04-25T00:23:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:06.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 20 client.15963 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374a48d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 4 client.15963 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37498cbf0 con 0x55e374a46940 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.057+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374a48d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6f877fe640 10 client.15963 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374a48d90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f9f6d0cf8fb890 server_cookie=7cc8a6c83f5a97b3 in_seq=0 out_seq=0 2024-04-25T00:23:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6f877fe640 1 -- 172.21.15.45:0/3374221896 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e37498cbf0 con 0x55e374a46940 2024-04-25T00:23:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6f877fe640 10 client.15963 handle_command_reply: tid=0 2024-04-25T00:23:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6fb167e740 1 client.15963 shutdown 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 msgr2=0x55e374a23000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.058+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0012290 tx=0x7f6fa0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 shutdown_connections 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374a48d90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6f877fe640 0 client.15963 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a23000 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a22ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 --2- 172.21.15.45:0/3374221896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 >> 172.21.15.45:0/3374221896 conn(0x55e374a16950 msgr2=0x55e374a15af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 shutdown_connections 2024-04-25T00:23:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 1 -- 172.21.15.45:0/3374221896 wait complete. 2024-04-25T00:23:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.059+0000 7f6fb167e740 20 client.15963 trim_cache size 0 max 16384 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 Processor -- start 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 -- start start 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a21700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8070bc0 con 0x55e374a25620 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8072700 con 0x55e374a1a630 2024-04-25T00:23:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.077+0000 7f6fb167e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8071290 con 0x55e374a1af50 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49432/0 (socket says 172.21.15.45:49432) 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 -- 172.21.15.45:0/2334598520 learned_addr learned my addr 172.21.15.45:0/2334598520 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 -- 172.21.15.45:0/2334598520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 msgr2=0x55e374a21700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a21700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 -- 172.21.15.45:0/2334598520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 msgr2=0x55e374a422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 -- 172.21.15.45:0/2334598520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e374a08090 con 0x55e374a25620 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6facf07640 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00126b0 tx=0x7f6fa0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d13e23d94bda87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.078+0000 7f6f8dffb640 1 -- 172.21.15.45:0/2334598520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0033e30 con 0x55e374a25620 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6f8dffb640 1 -- 172.21.15.45:0/2334598520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa0037070 con 0x55e374a25620 2024-04-25T00:23:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6f8dffb640 1 -- 172.21.15.45:0/2334598520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0029df0 con 0x55e374a25620 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 -- 172.21.15.45:0/2334598520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 msgr2=0x55e374a15970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00126b0 tx=0x7f6fa0005bd0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 -- 172.21.15.45:0/2334598520 shutdown_connections 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a25620 0x55e374a15970 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a21700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 --2- 172.21.15.45:0/2334598520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a1a630 0x55e374a422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 -- 172.21.15.45:0/2334598520 >> 172.21.15.45:0/2334598520 conn(0x55e374a16950 msgr2=0x55e374a15260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.079+0000 7f6fb167e740 1 -- 172.21.15.45:0/2334598520 shutdown_connections 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 -- 172.21.15.45:0/2334598520 wait complete. 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 -- start start 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 Processor -- start 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.080+0000 7f6fb167e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a25620 0x55e374aabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fb167e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8072030 con 0x55e374a1a630 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fb167e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8071960 con 0x55e374a25620 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fb167e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa806eba0 con 0x55e374a1af50 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fb167e740 10 client.0 _ll_register_callbacks cb 0x7f6fa00128e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49444/0 (socket says 172.21.15.45:49444) 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 -- 172.21.15.45:0/1312993992 learned_addr learned my addr 172.21.15.45:0/1312993992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6facf07640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a25620 0x55e374aabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 -- 172.21.15.45:0/1312993992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 msgr2=0x55e374a1fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a1fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 -- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a25620 msgr2=0x55e374aabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a25620 0x55e374aabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 -- 172.21.15.45:0/1312993992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e374aac410 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6fa7fff640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6f98000c00 tx=0x7f6f9800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6712beac62ff98b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.081+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98005000 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6f980051a0 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa0037070 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f98037050 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e374aad560 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f98004c80 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f98029290 con 0x55e374a1a630 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 20 client.15969 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374aafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 4 client.15969 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.082+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6fa0037070 con 0x55e374a46940 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374aafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6f877fe640 10 client.15969 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6fa77fe640 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374aafa40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ab8fdd7838a6ae server_cookie=8f8c5286cccd4ed0 in_seq=0 out_seq=0 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6f877fe640 1 -- 172.21.15.45:0/1312993992 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6fa0037070 con 0x55e374a46940 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6f877fe640 10 client.15969 handle_command_reply: tid=0 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.083+0000 7f6fb167e740 1 client.15969 shutdown 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 msgr2=0x55e374a1f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6f98000c00 tx=0x7f6f9800b930 comp rx=0 tx=0).stop 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 shutdown_connections 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e374a25620 0x55e374aabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e374a1af50 0x55e374a1fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e374a46940 0x55e374aafa40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 --2- 172.21.15.45:0/1312993992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e374a1a630 0x55e374a1f630 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 >> 172.21.15.45:0/1312993992 conn(0x55e374a16950 msgr2=0x55e374a22540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6f877fe640 0 client.15969 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 shutdown_connections 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 1 -- 172.21.15.45:0/1312993992 wait complete. 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fb167e740 20 client.15969 trim_cache size 0 max 16384 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:23:06.089 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 msgr2=0x7f6f900a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 0x7f6f900a5360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00049d0 tx=0x7f6fa0009500 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 msgr2=0x7f6fa8100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.084+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 0x7f6fa8100b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005940 tx=0x7f6f9c004270 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 shutdown_connections 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6f900a2ea0 0x7f6f900a5360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fa810bd00 0x7f6fa8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa81032e0 0x7f6fa8100b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 --2- 172.21.15.45:0/3393555410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fa81029c0 0x7f6fa8100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 >> 172.21.15.45:0/3393555410 conn(0x7f6fa8077740 msgr2=0x7f6fa80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 shutdown_connections 2024-04-25T00:23:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.085+0000 7f6fae96a640 1 -- 172.21.15.45:0/3393555410 wait complete. 2024-04-25T00:23:06.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:23:06.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 -- 172.21.15.45:0/361109360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281029d0 msgr2=0x7f5228102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 --2- 172.21.15.45:0/361109360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281029d0 0x7f5228102db0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521802a510 comp rx=0 tx=0).stop 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 -- 172.21.15.45:0/361109360 shutdown_connections 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 --2- 172.21.15.45:0/361109360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 --2- 172.21.15.45:0/361109360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281032f0 0x7f522810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 --2- 172.21.15.45:0/361109360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281029d0 0x7f5228102db0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 -- 172.21.15.45:0/361109360 >> 172.21.15.45:0/361109360 conn(0x7f5228077740 msgr2=0x7f5228077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 -- 172.21.15.45:0/361109360 shutdown_connections 2024-04-25T00:23:06.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.206+0000 7f522e4d6640 1 -- 172.21.15.45:0/361109360 wait complete. 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 Processor -- start 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 -- start start 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281029d0 0x7f5228198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5228111b60 con 0x7f52281032f0 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52281119e0 con 0x7f522810bd10 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f522e4d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5228111ce0 con 0x7f52281029d0 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f52277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f52277fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49464/0 (socket says 172.21.15.45:49464) 2024-04-25T00:23:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.207+0000 7f52277fe640 1 -- 172.21.15.45:0/1047055176 learned_addr learned my addr 172.21.15.45:0/1047055176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f5227fff640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281029d0 0x7f5228198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f522ca73640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 -- 172.21.15.45:0/1047055176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281029d0 msgr2=0x7f5228198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281029d0 0x7f5228198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 -- 172.21.15.45:0/1047055176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 msgr2=0x7f522819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 -- 172.21.15.45:0/1047055176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522819cff0 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f522ca73640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52277fe640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f521c00ea60 tx=0x7f521c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c00cde0 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521c0035f0 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522819d2e0 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.208+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c010670 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.209+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52281a4b70 con 0x7f52281032f0 2024-04-25T00:23:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.210+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5228068560 con 0x7f52281032f0 2024-04-25T00:23:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.212+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f521c0037a0 con 0x7f52281032f0 2024-04-25T00:23:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.213+0000 7f52257fa640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 0x7f52100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.213+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f521c0cfdf0 con 0x7f52281032f0 2024-04-25T00:23:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.213+0000 7f5227fff640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 0x7f52100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.213+0000 7f5227fff640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 0x7f52100a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5218000c00 tx=0x7f5218005b50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.217+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f521c09c460 con 0x7f52281032f0 2024-04-25T00:23:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.444+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5228106900 con 0x7f52281032f0 2024-04-25T00:23:06.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.445+0000 7f52257fa640 1 -- 172.21.15.45:0/1047055176 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f521c09bd60 con 0x7f52281032f0 2024-04-25T00:23:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:23:06.446 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:23:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:23:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 msgr2=0x7f52100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 0x7f52100a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5218000c00 tx=0x7f5218005b50 comp rx=0 tx=0).stop 2024-04-25T00:23:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 msgr2=0x7f5228198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f521c00ea60 tx=0x7f521c00ef20 comp rx=0 tx=0).stop 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 shutdown_connections 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f52100a2e20 0x7f52100a52e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f522810bd10 0x7f522819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52281032f0 0x7f5228198540 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 --2- 172.21.15.45:0/1047055176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52281029d0 0x7f5228198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 >> 172.21.15.45:0/1047055176 conn(0x7f5228077740 msgr2=0x7f52280fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 shutdown_connections 2024-04-25T00:23:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.448+0000 7f522e4d6640 1 -- 172.21.15.45:0/1047055176 wait complete. 2024-04-25T00:23:06.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 -- 172.21.15.45:0/3468733779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039010e0f0 msgr2=0x7f03901104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 --2- 172.21.15.45:0/3468733779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039010e0f0 0x7f03901104e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0388009ca0 tx=0x7f038802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 -- 172.21.15.45:0/3468733779 shutdown_connections 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 --2- 172.21.15.45:0/3468733779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039010e0f0 0x7f03901104e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 --2- 172.21.15.45:0/3468733779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390100e60 0x7f039010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 --2- 172.21.15.45:0/3468733779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f0390100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 -- 172.21.15.45:0/3468733779 >> 172.21.15.45:0/3468733779 conn(0x7f0390077740 msgr2=0x7f0390077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 -- 172.21.15.45:0/3468733779 shutdown_connections 2024-04-25T00:23:06.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.571+0000 7f0397cdf640 1 -- 172.21.15.45:0/3468733779 wait complete. 2024-04-25T00:23:06.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 Processor -- start 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 -- start start 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f039019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010e0f0 0x7f039019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0390113d70 con 0x7f0390100e60 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0390113bf0 con 0x7f039010e0f0 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f0397cdf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0390113ef0 con 0x7f0390100540 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f039527a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f039527a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49480/0 (socket says 172.21.15.45:49480) 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.572+0000 7f039527a640 1 -- 172.21.15.45:0/2483475183 learned_addr learned my addr 172.21.15.45:0/2483475183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f039019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039627c640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010e0f0 0x7f039019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 -- 172.21.15.45:0/2483475183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 msgr2=0x7f039019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f039019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 -- 172.21.15.45:0/2483475183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010e0f0 msgr2=0x7f039019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010e0f0 0x7f039019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 -- 172.21.15.45:0/2483475183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039019f1a0 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f039019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f039527a640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f038000d950 tx=0x7f038000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0386ffd640 1 -- 172.21.15.45:0/2483475183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380017030 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0386ffd640 1 -- 172.21.15.45:0/2483475183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380004020 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0386ffd640 1 -- 172.21.15.45:0/2483475183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03800047a0 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039019f490 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.573+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a6d20 con 0x7f0390100e60 2024-04-25T00:23:06.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.576+0000 7f0386ffd640 1 -- 172.21.15.45:0/2483475183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f0380004940 con 0x7f0390100e60 2024-04-25T00:23:06.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f0386ffd640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 0x7f037c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f039a9e0740 1 Processor -- start 2024-04-25T00:23:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f0386ffd640 1 -- 172.21.15.45:0/2483475183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f03800cf440 con 0x7f0390100e60 2024-04-25T00:23:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 0x7f037c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f039a9e0740 1 -- start start 2024-04-25T00:23:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.577+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b5060960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039a9e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4fb8670 con 0x55c5b5053580 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039a9e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4f60970 con 0x55c5b505e570 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039a9e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4fb4630 con 0x55c5b5053ea0 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49494/0 (socket says 172.21.15.45:49494) 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 -- 172.21.15.45:0/53057941 learned_addr learned my addr 172.21.15.45:0/53057941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039527a640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 0x7f037c0a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f038c009c70 tx=0x7f038c0117b0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 -- 172.21.15.45:0/53057941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 msgr2=0x55c5b505e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039627c640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b5060960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 -- 172.21.15.45:0/53057941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 msgr2=0x55c5b5060960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b5060960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 -- 172.21.15.45:0/53057941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b4f84400 con 0x55c5b5053580 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f039627c640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b5060960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.578+0000 7f0395a7b640 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f038c014780 tx=0x7f038c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290246d904ea3317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.579+0000 7f037bfff640 1 -- 172.21.15.45:0/53057941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c014c40 con 0x55c5b5053580 2024-04-25T00:23:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.579+0000 7f037bfff640 1 -- 172.21.15.45:0/53057941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038c01b040 con 0x55c5b5053580 2024-04-25T00:23:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.579+0000 7f037bfff640 1 -- 172.21.15.45:0/53057941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c012ab0 con 0x55c5b5053580 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- 172.21.15.45:0/53057941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 msgr2=0x55c5b5053960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f038c014780 tx=0x7f038c01f000 comp rx=0 tx=0).stop 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- 172.21.15.45:0/53057941 shutdown_connections 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b5060960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 --2- 172.21.15.45:0/53057941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b5053580 0x55c5b5053960 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- 172.21.15.45:0/53057941 >> 172.21.15.45:0/53057941 conn(0x55c5b504f8a0 msgr2=0x55c5b504fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- 172.21.15.45:0/53057941 shutdown_connections 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- 172.21.15.45:0/53057941 wait complete. 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.580+0000 7f039a9e0740 1 -- start start 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039a9e0740 1 Processor -- start 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b505b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039627c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.581+0000 7f039627c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49496/0 (socket says 172.21.15.45:49496) 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 -- 172.21.15.45:0/2556808089 learned_addr learned my addr 172.21.15.45:0/2556808089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039527a640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b505b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c5b5061360 con 0x55c5b505bf30 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c5b4f74050 con 0x55c5b505e570 2024-04-25T00:23:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c5b50614e0 con 0x55c5b5053ea0 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039a9e0740 10 client.0 _ll_register_callbacks cb 0x55c5b4ccbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 -- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 msgr2=0x55c5b505b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 -- 172.21.15.45:0/2556808089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 msgr2=0x55c5b505b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b505b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b4f45dc0 con 0x55c5b505bf30 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039627c640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0388005e30 tx=0x7f038802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec157b753fdc53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f03797fa640 10 client.15987 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c01b040 con 0x55c5b505bf30 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.582+0000 7f03797fa640 1 -- 172.21.15.45:0/2556808089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388003e00 con 0x55c5b505bf30 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f03797fa640 1 -- 172.21.15.45:0/2556808089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f0388031450 con 0x55c5b505bf30 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f03797fa640 1 -- 172.21.15.45:0/2556808089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03880316d0 con 0x55c5b505bf30 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f039a9e0740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55c5b4ccc3f0 con 0x55c5b505bf30 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f03797fa640 1 -- 172.21.15.45:0/2556808089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0388043070 con 0x55c5b505bf30 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f039a9e0740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f039a9e0740 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.583+0000 7f039a9e0740 20 client.15987 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b506ea50 0x55c5b5070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f039a9e0740 4 client.15987 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f038c01b040 con 0x55c5b506ea50 2024-04-25T00:23:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b506ea50 0x55c5b5070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f03797fa640 10 client.15987 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.584+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b506ea50 0x55c5b5070ef0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8160f3fd82e39008 server_cookie=eaa6b8b424b942dc in_seq=0 out_seq=0 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f03797fa640 1 -- 172.21.15.45:0/2556808089 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f038c01b040 con 0x55c5b506ea50 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f03797fa640 10 client.15987 handle_command_reply: tid=0 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 client.15987 shutdown 2024-04-25T00:23:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 msgr2=0x55c5b506a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0388005e30 tx=0x7f038802f450 comp rx=0 tx=0).stop 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 shutdown_connections 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b506ea50 0x55c5b5070ef0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b506a7d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505e570 0x55c5b505b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2556808089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 >> 172.21.15.45:0/2556808089 conn(0x55c5b504f8a0 msgr2=0x55c5b50556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.585+0000 7f03797fa640 0 client.15987 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 shutdown_connections 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 1 -- 172.21.15.45:0/2556808089 wait complete. 2024-04-25T00:23:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.586+0000 7f039a9e0740 20 client.15987 trim_cache size 0 max 16384 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.599+0000 7f039a9e0740 1 Processor -- start 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.599+0000 7f039a9e0740 1 -- start start 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 0x55c5b504e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03901b0d30 con 0x55c5b505e570 2024-04-25T00:23:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f03901b2870 con 0x55c5b505bf30 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039a9e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03901b1400 con 0x55c5b5053ea0 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039527a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 0x55c5b504e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49502/0 (socket says 172.21.15.45:49502) 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 -- 172.21.15.45:0/1268035274 learned_addr learned my addr 172.21.15.45:0/1268035274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f0395a7b640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 -- 172.21.15.45:0/1268035274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 msgr2=0x55c5b505b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 -- 172.21.15.45:0/1268035274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 msgr2=0x55c5b504e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 0x55c5b504e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039627c640 1 -- 172.21.15.45:0/1268035274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b506b410 con 0x55c5b505e570 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f0395a7b640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.600+0000 7f039527a640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 0x55c5b504e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.601+0000 7f039627c640 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0388002230 tx=0x7f038803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5ac0732eba3cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.601+0000 7f037b7fe640 1 -- 172.21.15.45:0/1268035274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388055070 con 0x55c5b505e570 2024-04-25T00:23:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.601+0000 7f037b7fe640 1 -- 172.21.15.45:0/1268035274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388036050 con 0x55c5b505e570 2024-04-25T00:23:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.601+0000 7f037b7fe640 1 -- 172.21.15.45:0/1268035274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388004b40 con 0x55c5b505e570 2024-04-25T00:23:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- 172.21.15.45:0/1268035274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 msgr2=0x55c5b506a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0388002230 tx=0x7f038803f000 comp rx=0 tx=0).stop 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- 172.21.15.45:0/1268035274 shutdown_connections 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505e570 0x55c5b506a220 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b505bf30 0x55c5b504e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 --2- 172.21.15.45:0/1268035274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b5053ea0 0x55c5b505b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- 172.21.15.45:0/1268035274 >> 172.21.15.45:0/1268035274 conn(0x55c5b504f8a0 msgr2=0x55c5b504e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- 172.21.15.45:0/1268035274 shutdown_connections 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- 172.21.15.45:0/1268035274 wait complete. 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.602+0000 7f039a9e0740 1 -- start start 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 Processor -- start 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 0x55c5b50644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03901b21a0 con 0x55c5b505bf30 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f03901b1ad0 con 0x55c5b5053ea0 2024-04-25T00:23:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03901aed10 con 0x55c5b505e570 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039a9e0740 10 client.0 _ll_register_callbacks cb 0x7f038c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039527a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f0395a7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039527a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49516/0 (socket says 172.21.15.45:49516) 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f0395a7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60496/0 (socket says 172.21.15.45:60496) 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.603+0000 7f039527a640 1 -- 172.21.15.45:0/2645281763 learned_addr learned my addr 172.21.15.45:0/2645281763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f039627c640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 0x55c5b50644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 -- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 msgr2=0x55c5b50644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 0x55c5b50644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 -- 172.21.15.45:0/2645281763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 msgr2=0x55c5b5063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 -- 172.21.15.45:0/2645281763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b4d967d0 con 0x55c5b5053ea0 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f039527a640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f039627c640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 0x55c5b50644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f0395a7b640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f038c01a000 tx=0x7f038c0096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd256cdfd38bee8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f03797fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0143e0 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f038c00f8d0 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.604+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0388036050 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f038c00fad0 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 10 client.25559 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5b50d4f30 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388036050 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f038c012790 con 0x55c5b5053ea0 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 10 client.25559 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 10 client.25559 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 20 client.25559 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b50d5380 0x55c5b50d7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 4 client.25559 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.605+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5b4fc5bb0 con 0x55c5b50d5380 2024-04-25T00:23:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f039527a640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b50d5380 0x55c5b50d7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f03797fa640 10 client.25559 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:06.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f039527a640 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b50d5380 0x55c5b50d7770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=caef6e242e95a30e server_cookie=795779005636005b in_seq=0 out_seq=0 2024-04-25T00:23:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f03797fa640 1 -- 172.21.15.45:0/2645281763 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5b4fc5bb0 con 0x55c5b50d5380 2024-04-25T00:23:06.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f03797fa640 10 client.25559 handle_command_reply: tid=0 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f039a9e0740 1 client.25559 shutdown 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.606+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 msgr2=0x55c5b5066bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f038c01a000 tx=0x7f038c0096b0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 shutdown_connections 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55c5b50d5380 0x55c5b50d7770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5b505e570 0x55c5b50644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f03797fa640 0 client.25559 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5b505bf30 0x55c5b5063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 --2- 172.21.15.45:0/2645281763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5b5053ea0 0x55c5b5066bb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 >> 172.21.15.45:0/2645281763 conn(0x55c5b504f8a0 msgr2=0x55c5b504eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 shutdown_connections 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 1 -- 172.21.15.45:0/2645281763 wait complete. 2024-04-25T00:23:06.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.607+0000 7f039a9e0740 20 client.25559 trim_cache size 0 max 16384 2024-04-25T00:23:06.615 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 msgr2=0x7f037c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 0x7f037c0a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f038c009c70 tx=0x7f038c0117b0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 msgr2=0x7f039019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f038000d950 tx=0x7f038000de10 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 shutdown_connections 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f037c0a2e20 0x7f037c0a52e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039010e0f0 0x7f039019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0390100e60 0x7f039019a6f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 --2- 172.21.15.45:0/2483475183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0390100540 0x7f039019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 >> 172.21.15.45:0/2483475183 conn(0x7f0390077740 msgr2=0x7f039010f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 shutdown_connections 2024-04-25T00:23:06.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.608+0000 7f0397cdf640 1 -- 172.21.15.45:0/2483475183 wait complete. 2024-04-25T00:23:06.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 -- 172.21.15.45:0/3086893397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c101720 msgr2=0x7f363c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 --2- 172.21.15.45:0/3086893397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c101720 0x7f363c101ba0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f362c009930 tx=0x7f362c02a420 comp rx=0 tx=0).stop 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 -- 172.21.15.45:0/3086893397 shutdown_connections 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 --2- 172.21.15.45:0/3086893397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c1020e0 0x7f363c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 --2- 172.21.15.45:0/3086893397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c101720 0x7f363c101ba0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 --2- 172.21.15.45:0/3086893397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c107730 0x7f363c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.730+0000 7f36436d4640 1 -- 172.21.15.45:0/3086893397 >> 172.21.15.45:0/3086893397 conn(0x7f363c078b40 msgr2=0x7f363c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.731+0000 7f36436d4640 1 -- 172.21.15.45:0/3086893397 shutdown_connections 2024-04-25T00:23:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.731+0000 7f36436d4640 1 -- 172.21.15.45:0/3086893397 wait complete. 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.731+0000 7f36436d4640 1 Processor -- start 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.731+0000 7f36436d4640 1 -- start start 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 0x7f363c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c1020e0 0x7f363c073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f363c111b90 con 0x7f363c107730 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f363c111a10 con 0x7f363c1020e0 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f36436d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f363c111d10 con 0x7f363c101720 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49522/0 (socket says 172.21.15.45:49522) 2024-04-25T00:23:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 -- 172.21.15.45:0/1572431479 learned_addr learned my addr 172.21.15.45:0/1572431479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641470640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 0x7f363c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3640c6f640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c1020e0 0x7f363c073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 -- 172.21.15.45:0/1572431479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 msgr2=0x7f363c073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 0x7f363c073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 -- 172.21.15.45:0/1572431479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c1020e0 msgr2=0x7f363c073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c1020e0 0x7f363c073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 -- 172.21.15.45:0/1572431479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1a6510 con 0x7f363c107730 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641470640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 0x7f363c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.732+0000 7f3641c71640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f363400eaa0 tx=0x7f363400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.733+0000 7f36327fc640 1 -- 172.21.15.45:0/1572431479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400cdb0 con 0x7f363c107730 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.733+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a6800 con 0x7f363c107730 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.733+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a6d70 con 0x7f363c107730 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.733+0000 7f36327fc640 1 -- 172.21.15.45:0/1572431479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634005020 con 0x7f363c107730 2024-04-25T00:23:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.733+0000 7f36327fc640 1 -- 172.21.15.45:0/1572431479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400cdb0 con 0x7f363c107730 2024-04-25T00:23:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.736+0000 7f36463e4740 1 Processor -- start 2024-04-25T00:23:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.736+0000 7f36327fc640 1 -- 172.21.15.45:0/1572431479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f3634004a40 con 0x7f363c107730 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 -- start start 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9da960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5574bd932670 con 0x5574bd9cd580 2024-04-25T00:23:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49528/0 (socket says 172.21.15.45:49528) 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f3640c6f640 1 -- 172.21.15.45:0/824296530 learned_addr learned my addr 172.21.15.45:0/824296530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5574bd8da970 con 0x5574bd9cdea0 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f3641470640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5574bd92e630 con 0x5574bd9d8570 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.737+0000 7f36327fc640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 0x7f36280a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 -- 172.21.15.45:0/824296530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 msgr2=0x5574bd9da960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9da960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 -- 172.21.15.45:0/824296530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 msgr2=0x5574bd9d8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 -- 172.21.15.45:0/824296530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bd8fe400 con 0x5574bd9cd580 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3641470640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3641c71640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 0x7f36280a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f3640c6f640 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f362c009930 tx=0x7f362c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f64408f400634f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f36327fc640 1 -- 172.21.15.45:0/1572431479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f36340cf960 con 0x7f363c107730 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f36277fe640 1 -- 172.21.15.45:0/824296530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c044020 con 0x5574bd9cd580 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f36277fe640 1 -- 172.21.15.45:0/824296530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c0044b0 con 0x5574bd9cd580 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.738+0000 7f36277fe640 1 -- 172.21.15.45:0/824296530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c035040 con 0x5574bd9cd580 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f3641c71640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 0x7f36280a5590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36340ccc30 tx=0x7f3634004910 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 msgr2=0x5574bd9cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f362c009930 tx=0x7f362c005b50 comp rx=0 tx=0).stop 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 shutdown_connections 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9da960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 --2- 172.21.15.45:0/824296530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9cd960 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 >> 172.21.15.45:0/824296530 conn(0x5574bd9c98a0 msgr2=0x5574bd9c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 shutdown_connections 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.739+0000 7f36463e4740 1 -- 172.21.15.45:0/824296530 wait complete. 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 -- start start 2024-04-25T00:23:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 Processor -- start 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 0x5574bd9d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5574bd9db300 con 0x5574bd9cdea0 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f3641470640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5574bd8ee050 con 0x5574bd9cd580 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f3641470640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49538/0 (socket says 172.21.15.45:49538) 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f3641470640 1 -- 172.21.15.45:0/2962008206 learned_addr learned my addr 172.21.15.45:0/2962008206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5574bd9db480 con 0x5574bd9d8570 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.740+0000 7f3640c6f640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 0x5574bd9d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f36463e4740 10 client.0 _ll_register_callbacks cb 0x5574bd645be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 -- 172.21.15.45:0/2962008206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 msgr2=0x5574bd9d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 -- 172.21.15.45:0/2962008206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 msgr2=0x5574bd9d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 -- 172.21.15.45:0/2962008206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bd8bfdc0 con 0x5574bd9cd580 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3641470640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3640c6f640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 0x5574bd9d7e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f362c009930 tx=0x7f362c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28f147894d8fccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:23:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c044070 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c0044b0 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.741+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f362c0315b0 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f362c035040 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 10 client.25571 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574bd6463f0 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c0044b0 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f362c04a690 con 0x5574bd9cd580 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 10 client.25571 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 10 client.25571 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 20 client.25571 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bd9e8390 0x5574bd9ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 4 client.25571 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574bd93fbb0 con 0x5574bd9e8390 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.742+0000 7f3641470640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bd9e8390 0x5574bd9ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.743+0000 7f3641470640 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bd9e8390 0x5574bd9ea780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=697a5f481c231740 server_cookie=bd1a4b809e3fa574 in_seq=0 out_seq=0 2024-04-25T00:23:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.743+0000 7f3624ff9640 10 client.25571 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.743+0000 7f3624ff9640 1 -- 172.21.15.45:0/2962008206 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574bd93fbb0 con 0x5574bd9e8390 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.743+0000 7f3624ff9640 10 client.25571 handle_command_reply: tid=0 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.743+0000 7f36463e4740 1 client.25571 shutdown 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 msgr2=0x5574bd9d7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 0x5574bd9d7e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f362c009930 tx=0x7f362c005850 comp rx=0 tx=0).stop 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 shutdown_connections 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bd9e8390 0x5574bd9ea780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9d8570 0x5574bd9d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cdea0 0x5574bd9d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f3624ff9640 0 client.25571 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 --2- 172.21.15.45:0/2962008206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cd580 0x5574bd9d7e50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 >> 172.21.15.45:0/2962008206 conn(0x5574bd9c98a0 msgr2=0x5574bd9c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 shutdown_connections 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 1 -- 172.21.15.45:0/2962008206 wait complete. 2024-04-25T00:23:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.744+0000 7f36463e4740 20 client.25571 trim_cache size 0 max 16384 2024-04-25T00:23:06.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.757+0000 7f36463e4740 1 Processor -- start 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 -- start start 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cd580 0x5574bd9cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f362000fdf0 con 0x5574bd9d8570 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3620011930 con 0x5574bd9cdea0 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f36463e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36200104c0 con 0x5574bd9cd580 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49540/0 (socket says 172.21.15.45:49540) 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 -- 172.21.15.45:0/607542731 learned_addr learned my addr 172.21.15.45:0/607542731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cd580 0x5574bd9cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3641470640 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 -- 172.21.15.45:0/607542731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cd580 msgr2=0x5574bd9cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cd580 0x5574bd9cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 -- 172.21.15.45:0/607542731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 msgr2=0x5574bd9d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.758+0000 7f3640c6f640 1 -- 172.21.15.45:0/607542731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bd9cdcd0 con 0x5574bd9d8570 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f3640c6f640 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f362c004930 tx=0x7f362c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52279ff586503855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f3626ffd640 1 -- 172.21.15.45:0/607542731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c044040 con 0x5574bd9d8570 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f3626ffd640 1 -- 172.21.15.45:0/607542731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c04f030 con 0x5574bd9d8570 2024-04-25T00:23:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f3626ffd640 1 -- 172.21.15.45:0/607542731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0315b0 con 0x5574bd9d8570 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f36463e4740 1 -- 172.21.15.45:0/607542731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 msgr2=0x5574bd9ed7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.759+0000 7f36463e4740 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f362c004930 tx=0x7f362c02a420 comp rx=0 tx=0).stop 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 -- 172.21.15.45:0/607542731 shutdown_connections 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9d8570 0x5574bd9ed7a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9cdea0 0x5574bd9d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- 172.21.15.45:0/607542731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cd580 0x5574bd9cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 -- 172.21.15.45:0/607542731 >> 172.21.15.45:0/607542731 conn(0x5574bd9c98a0 msgr2=0x5574bd9cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 -- 172.21.15.45:0/607542731 shutdown_connections 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 -- 172.21.15.45:0/607542731 wait complete. 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 -- start start 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 Processor -- start 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cdea0 0x5574bd9eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.760+0000 7f36463e4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9d8570 0x5574bd9eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f36463e4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3620011260 con 0x5574bd9cd580 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f36463e4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3620010b90 con 0x5574bd9d8570 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f36463e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f362000ddd0 con 0x5574bd9cdea0 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49542/0 (socket says 172.21.15.45:49542) 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 -- 172.21.15.45:0/1059342430 learned_addr learned my addr 172.21.15.45:0/1059342430 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f36463e4740 10 client.0 _ll_register_callbacks cb 0x7f3638009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3641470640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cdea0 0x5574bd9eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 -- 172.21.15.45:0/1059342430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cdea0 msgr2=0x5574bd9eafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cdea0 0x5574bd9eafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 -- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9d8570 msgr2=0x5574bd9eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9d8570 0x5574bd9eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 -- 172.21.15.45:0/1059342430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bd7107d0 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3640c6f640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f362c031550 tx=0x7f362c004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f7cd9f35db269c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3624ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c044040 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.761+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f362c005680 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c04f030 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f362c04a430 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574bda4e440 con 0x5574bd9cd580 2024-04-25T00:23:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c04f030 con 0x5574bd9cd580 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f362c0623f0 con 0x5574bd9cd580 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 10 client.16011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 20 client.16011 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bda4ea50 0x5574bda50e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 4 client.16011 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.762+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574bd939f00 con 0x5574bda4ea50 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3641470640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bda4ea50 0x5574bda50e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3624ff9640 10 client.16011 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3641470640 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bda4ea50 0x5574bda50e70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15974e112aafdd5c server_cookie=20fd7354bb483910 in_seq=0 out_seq=0 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3624ff9640 1 -- 172.21.15.45:0/1059342430 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574bd939f00 con 0x5574bda4ea50 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3624ff9640 10 client.16011 handle_command_reply: tid=0 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 client.16011 shutdown 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 msgr2=0x5574bd9edc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f362c031550 tx=0x7f362c004c50 comp rx=0 tx=0).stop 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 shutdown_connections 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5574bda4ea50 0x5574bda50e70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f3624ff9640 0 client.16011 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574bd9d8570 0x5574bd9eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574bd9cdea0 0x5574bd9eafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 --2- 172.21.15.45:0/1059342430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bd9cd580 0x5574bd9edc10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 >> 172.21.15.45:0/1059342430 conn(0x5574bd9c98a0 msgr2=0x5574bd9d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.763+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 shutdown_connections 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36463e4740 1 -- 172.21.15.45:0/1059342430 wait complete. 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36463e4740 20 client.16011 trim_cache size 0 max 16384 2024-04-25T00:23:06.769 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 msgr2=0x7f36280a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 0x7f36280a5590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36340ccc30 tx=0x7f3634004910 comp rx=0 tx=0).stop 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 msgr2=0x7f363c0744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f363400eaa0 tx=0x7f363400ef60 comp rx=0 tx=0).stop 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 shutdown_connections 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f36280a30d0 0x7f36280a5590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f363c107730 0x7f363c0744c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f363c1020e0 0x7f363c073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 --2- 172.21.15.45:0/1572431479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f363c101720 0x7f363c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 >> 172.21.15.45:0/1572431479 conn(0x7f363c078b40 msgr2=0x7f363c1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 shutdown_connections 2024-04-25T00:23:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.764+0000 7f36436d4640 1 -- 172.21.15.45:0/1572431479 wait complete. 2024-04-25T00:23:06.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 -- 172.21.15.45:0/3680820716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 msgr2=0x7f066810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 --2- 172.21.15.45:0/3680820716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f066810b7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f065c009860 tx=0x7f065c02a480 comp rx=0 tx=0).stop 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 -- 172.21.15.45:0/3680820716 shutdown_connections 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 --2- 172.21.15.45:0/3680820716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f066810bd00 0x7f066810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 --2- 172.21.15.45:0/3680820716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f066810b7c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 --2- 172.21.15.45:0/3680820716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06681029c0 0x7f0668102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 -- 172.21.15.45:0/3680820716 >> 172.21.15.45:0/3680820716 conn(0x7f0668077740 msgr2=0x7f0668077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 -- 172.21.15.45:0/3680820716 shutdown_connections 2024-04-25T00:23:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 -- 172.21.15.45:0/3680820716 wait complete. 2024-04-25T00:23:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.881+0000 7f066e46b640 1 Processor -- start 2024-04-25T00:23:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 -- start start 2024-04-25T00:23:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06681029c0 0x7f0668100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f0668100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 0x7f0668101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111b00 con 0x7f066810bd00 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111980 con 0x7f06681032e0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066e46b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111c80 con 0x7f06681029c0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066ca08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 0x7f0668101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066ca08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 0x7f0668101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49562/0 (socket says 172.21.15.45:49562) 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f066ca08640 1 -- 172.21.15.45:0/3744512161 learned_addr learned my addr 172.21.15.45:0/3744512161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f06677fe640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f0668100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.882+0000 7f0667fff640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06681029c0 0x7f0668100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 -- 172.21.15.45:0/3744512161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06681029c0 msgr2=0x7f0668100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06681029c0 0x7f0668100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 -- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 msgr2=0x7f0668101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 0x7f0668101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 -- 172.21.15.45:0/3744512161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06681a6500 con 0x7f06681032e0 2024-04-25T00:23:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06677fe640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f0668100b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f065c000c00 tx=0x7f065c0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06657fa640 1 -- 172.21.15.45:0/3744512161 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c040070 con 0x7f06681032e0 2024-04-25T00:23:06.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06681a6790 con 0x7f06681032e0 2024-04-25T00:23:06.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f06657fa640 1 -- 172.21.15.45:0/3744512161 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c031400 con 0x7f06681032e0 2024-04-25T00:23:06.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.883+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a6c10 con 0x7f06681032e0 2024-04-25T00:23:06.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.884+0000 7f06657fa640 1 -- 172.21.15.45:0/3744512161 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c004ab0 con 0x7f06681032e0 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 Processor -- start 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 -- start start 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f06657fa640 1 -- 172.21.15.45:0/3744512161 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f065c039040 con 0x7f06681032e0 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56396b85b670 con 0x56396b901570 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56396b803970 con 0x56396b8f6ea0 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.886+0000 7f067117a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56396b857630 con 0x56396b8f6580 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f06677fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f0667fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f06677fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49578/0 (socket says 172.21.15.45:49578) 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:39814/0 (socket says 172.21.15.45:39814) 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f0667fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60572/0 (socket says 172.21.15.45:60572) 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f06677fe640 1 -- 172.21.15.45:0/3917691218 learned_addr learned my addr 172.21.15.45:0/3917691218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 -- 172.21.15.45:0/3917691218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 msgr2=0x56396b901030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 -- 172.21.15.45:0/3917691218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 msgr2=0x56396b903960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 -- 172.21.15.45:0/3917691218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56396b827400 con 0x56396b8f6580 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f0667fff640 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f06677fe640 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f066ca08640 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0660009f30 tx=0x7f06600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5041fc6ef7411e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.887+0000 7f064e7fc640 1 -- 172.21.15.45:0/3917691218 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660004b20 con 0x56396b8f6580 2024-04-25T00:23:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f064e7fc640 1 -- 172.21.15.45:0/3917691218 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660004cc0 con 0x56396b8f6580 2024-04-25T00:23:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f064e7fc640 1 -- 172.21.15.45:0/3917691218 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660005330 con 0x56396b8f6580 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f06657fa640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 0x7f06500a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f0667fff640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 0x7f06500a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f06657fa640 1 -- 172.21.15.45:0/3744512161 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f065c0f06d0 con 0x7f06681032e0 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 -- 172.21.15.45:0/3917691218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 msgr2=0x56396b8f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0660009f30 tx=0x7f06600118d0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 -- 172.21.15.45:0/3917691218 shutdown_connections 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b901570 0x56396b903960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b901030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 --2- 172.21.15.45:0/3917691218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b8f6960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.888+0000 7f067117a740 1 -- 172.21.15.45:0/3917691218 >> 172.21.15.45:0/3917691218 conn(0x56396b8f28a0 msgr2=0x56396b8f2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f0667fff640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 0x7f06500a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x56396b8f7b40 tx=0x7f0658009260 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- 172.21.15.45:0/3917691218 shutdown_connections 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- 172.21.15.45:0/3917691218 wait complete. 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- start start 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 Processor -- start 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b900dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b8fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56396b904360 con 0x56396b8f6ea0 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56396b817050 con 0x56396b8f6580 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.889+0000 7f067117a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56396b9044e0 con 0x56396b901570 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f067117a740 10 client.0 _ll_register_callbacks cb 0x56396b56ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f066ca08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f06677fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b8fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f066ca08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:39826/0 (socket says 172.21.15.45:39826) 2024-04-25T00:23:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f066ca08640 1 -- 172.21.15.45:0/4182991062 learned_addr learned my addr 172.21.15.45:0/4182991062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f06677fe640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b8fe0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49594/0 (socket says 172.21.15.45:49594) 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b900dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 -- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 msgr2=0x56396b8fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 -- 172.21.15.45:0/4182991062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 msgr2=0x56396b8fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b8fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 -- 172.21.15.45:0/4182991062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396b7e8dc0 con 0x56396b8f6580 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f066ca08640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f0667fff640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b900dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0658009530 tx=0x7f0658004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae9234c3476d478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f06477fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065802d060 con 0x56396b8f6580 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.890+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0660004cc0 con 0x56396b8f6580 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f0658018030 con 0x56396b8f6580 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0658034050 con 0x56396b8f6580 2024-04-25T00:23:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 10 client.25586 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56396b56f3f0 con 0x56396b8f6580 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660004cc0 con 0x56396b8f6580 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f065800f3e0 con 0x56396b8f6580 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 10 client.25586 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 10 client.25586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 20 client.25586 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b911590 0x56396b913a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 4 client.25586 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.891+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56396b868bb0 con 0x56396b911590 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.892+0000 7f06677fe640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b911590 0x56396b913a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.892+0000 7f06677fe640 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b911590 0x56396b913a10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f6d7aca5b7be265 server_cookie=abcbc2ddeda0ab95 in_seq=0 out_seq=0 2024-04-25T00:23:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.892+0000 7f06477fe640 10 client.25586 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f06477fe640 1 -- 172.21.15.45:0/4182991062 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56396b868bb0 con 0x56396b911590 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f06477fe640 10 client.25586 handle_command_reply: tid=0 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 client.25586 shutdown 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 msgr2=0x56396b900dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b900dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0658009530 tx=0x7f0658004730 comp rx=0 tx=0).stop 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 shutdown_connections 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b911590 0x56396b913a10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b901570 0x56396b8fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b8fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 --2- 172.21.15.45:0/4182991062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b900dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 >> 172.21.15.45:0/4182991062 conn(0x56396b8f28a0 msgr2=0x56396b8f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f06477fe640 0 client.25586 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:06.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 shutdown_connections 2024-04-25T00:23:06.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 1 -- 172.21.15.45:0/4182991062 wait complete. 2024-04-25T00:23:06.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.893+0000 7f067117a740 20 client.25586 trim_cache size 0 max 16384 2024-04-25T00:23:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 Processor -- start 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 -- start start 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 0x56396b8f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b8f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b900900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f066ca08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b900900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f066ca08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b900900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49608/0 (socket says 172.21.15.45:49608) 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f066ca08640 1 -- 172.21.15.45:0/4025754979 learned_addr learned my addr 172.21.15.45:0/4025754979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f065400fdf0 con 0x56396b8f6ea0 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0654011930 con 0x56396b8f6580 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06540104c0 con 0x56396b8f28a0 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f0667fff640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 0x56396b8f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.907+0000 7f06677fe640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b8f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 -- 172.21.15.45:0/4025754979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 msgr2=0x56396b8f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b8f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 -- 172.21.15.45:0/4025754979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 msgr2=0x56396b900900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b900900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 -- 172.21.15.45:0/4025754979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56396b6397d0 con 0x56396b8f28a0 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f06677fe640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b8f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f0667fff640 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 0x56396b8f9860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0658006010 tx=0x7f0658000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4781dd451fc594e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f064dffb640 1 -- 172.21.15.45:0/4025754979 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065803c070 con 0x56396b8f28a0 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f064dffb640 1 -- 172.21.15.45:0/4025754979 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658009870 con 0x56396b8f28a0 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.908+0000 7f064dffb640 1 -- 172.21.15.45:0/4025754979 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580035f0 con 0x56396b8f28a0 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 msgr2=0x56396b8f9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 0x56396b8f9860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0658006010 tx=0x7f0658000bc0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 shutdown_connections 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f6ea0 0x56396b900900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6580 0x56396b8f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 --2- 172.21.15.45:0/4025754979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f28a0 0x56396b8f9860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 >> 172.21.15.45:0/4025754979 conn(0x56396b8fe760 msgr2=0x56396b8f7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 shutdown_connections 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- 172.21.15.45:0/4025754979 wait complete. 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 -- start start 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.909+0000 7f067117a740 1 Processor -- start 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b904e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b905380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f0667fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0654011260 con 0x56396b8f28a0 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f0667fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49618/0 (socket says 172.21.15.45:49618) 2024-04-25T00:23:06.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f0667fff640 1 -- 172.21.15.45:0/3642812953 learned_addr learned my addr 172.21.15.45:0/3642812953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b904e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0654010b90 con 0x56396b8f6ea0 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f065400ddd0 con 0x56396b8f6580 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f067117a740 10 client.0 _ll_register_callbacks cb 0x7f065801d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f066ca08640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b905380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 -- 172.21.15.45:0/3642812953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 msgr2=0x56396b905380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b905380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 -- 172.21.15.45:0/3642812953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 msgr2=0x56396b909b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f06677fe640 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396b905bb0 con 0x56396b8f6580 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.910+0000 7f0667fff640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:06.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06677fe640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b904e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ed9d0 tx=0x7f065c0f2be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c1925c75dd7589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c0d5060 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0658009870 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f065c005240 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f065c0da050 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f067117a740 10 client.25567 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56396b977b70 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658009870 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.911+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f065c04d030 con 0x56396b8f6580 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 10 client.25567 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 10 client.25567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 20 client.25567 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b901570 0x56396b97a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 4 client.25567 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:06.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56396b862f00 con 0x56396b901570 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f0667fff640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b901570 0x56396b97a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f06477fe640 10 client.25567 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f0667fff640 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b901570 0x56396b97a330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3735922d5f4a4595 server_cookie=180561093236438a in_seq=0 out_seq=0 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f06477fe640 1 -- 172.21.15.45:0/3642812953 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56396b862f00 con 0x56396b901570 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.912+0000 7f06477fe640 10 client.25567 handle_command_reply: tid=0 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 client.25567 shutdown 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 msgr2=0x56396b904e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b904e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ed9d0 tx=0x7f065c0f2be0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 shutdown_connections 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x56396b901570 0x56396b97a330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56396b8f6ea0 0x56396b905380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56396b8f6580 0x56396b904e40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 --2- 172.21.15.45:0/3642812953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56396b8f28a0 0x56396b909b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 >> 172.21.15.45:0/3642812953 conn(0x56396b8fe760 msgr2=0x56396b8f9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f06477fe640 0 client.25567 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:06.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 shutdown_connections 2024-04-25T00:23:06.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 1 -- 172.21.15.45:0/3642812953 wait complete. 2024-04-25T00:23:06.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f067117a740 20 client.25567 trim_cache size 0 max 16384 2024-04-25T00:23:06.920 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.913+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 msgr2=0x7f06500a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 0x7f06500a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x56396b8f7b40 tx=0x7f0658009260 comp rx=0 tx=0).stop 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 msgr2=0x7f0668100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f0668100b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f065c000c00 tx=0x7f065c0305d0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 shutdown_connections 2024-04-25T00:23:06.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f06500a2ea0 0x7f06500a5360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f066810bd00 0x7f0668101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06681032e0 0x7f0668100b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 --2- 172.21.15.45:0/3744512161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06681029c0 0x7f0668100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 >> 172.21.15.45:0/3744512161 conn(0x7f0668077740 msgr2=0x7f06680fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 shutdown_connections 2024-04-25T00:23:06.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:06.914+0000 7f066e46b640 1 -- 172.21.15.45:0/3744512161 wait complete. 2024-04-25T00:23:06.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 -- 172.21.15.45:0/513455909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81029c0 msgr2=0x7f9be8102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/513455909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81029c0 0x7f9be8102da0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009900 tx=0x7f9bd802a510 comp rx=0 tx=0).stop 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 -- 172.21.15.45:0/513455909 shutdown_connections 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/513455909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/513455909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81032e0 0x7f9be810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/513455909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81029c0 0x7f9be8102da0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 -- 172.21.15.45:0/513455909 >> 172.21.15.45:0/513455909 conn(0x7f9be8077740 msgr2=0x7f9be8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 -- 172.21.15.45:0/513455909 shutdown_connections 2024-04-25T00:23:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.031+0000 7f9bed6a5640 1 -- 172.21.15.45:0/513455909 wait complete. 2024-04-25T00:23:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 Processor -- start 2024-04-25T00:23:07.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 -- start start 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 0x7f9be8100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be8101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111b00 con 0x7f9be81032e0 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111980 con 0x7f9be81029c0 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9be67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9be67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49624/0 (socket says 172.21.15.45:49624) 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9be67fc640 1 -- 172.21.15.45:0/3811487240 learned_addr learned my addr 172.21.15.45:0/3811487240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be8111c80 con 0x7f9be810bd00 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 0x7f9be8100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.032+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be8101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 -- 172.21.15.45:0/3811487240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 msgr2=0x7f9be8101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be8101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 -- 172.21.15.45:0/3811487240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 msgr2=0x7f9be8100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 0x7f9be8100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 -- 172.21.15.45:0/3811487240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be81a6500 con 0x7f9be81032e0 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 0x7f9be8100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9be67fc640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9f0 tx=0x7f9bdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9bcffff640 1 -- 172.21.15.45:0/3811487240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00cd80 con 0x7f9be81032e0 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9bcffff640 1 -- 172.21.15.45:0/3811487240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc010910 con 0x7f9be81032e0 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be81a67f0 con 0x7f9be81032e0 2024-04-25T00:23:07.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9bcffff640 1 -- 172.21.15.45:0/3811487240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc018990 con 0x7f9be81032e0 2024-04-25T00:23:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.033+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be81a6cf0 con 0x7f9be81032e0 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.036+0000 7f9bcffff640 1 -- 172.21.15.45:0/3811487240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f9bdc018b30 con 0x7f9be81032e0 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.036+0000 7f9bf03b1740 1 Processor -- start 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.036+0000 7f9bf03b1740 1 -- start start 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e83ea0 0x559386e8e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 0x559386e90960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559386de8670 con 0x559386e83580 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559386d90970 con 0x559386e8e570 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559386de4630 con 0x559386e83ea0 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49630/0 (socket says 172.21.15.45:49630) 2024-04-25T00:23:07.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 -- 172.21.15.45:0/768581019 learned_addr learned my addr 172.21.15.45:0/768581019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e83ea0 0x559386e8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be67fc640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 0x559386e90960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bcffff640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 0x7f9bd00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 -- 172.21.15.45:0/768581019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e83ea0 msgr2=0x559386e8e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e83ea0 0x559386e8e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 -- 172.21.15.45:0/768581019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 msgr2=0x559386e90960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 0x559386e90960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be77fe640 1 -- 172.21.15.45:0/768581019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559386db4400 con 0x559386e83580 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9be67fc640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 0x559386e90960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.037+0000 7f9bcffff640 1 -- 172.21.15.45:0/3811487240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9bdc0cfe60 con 0x7f9be81032e0 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.038+0000 7f9be77fe640 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4be0b11de9bddbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.038+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 0x7f9bd00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.038+0000 7f9bcd7fa640 1 -- 172.21.15.45:0/768581019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0016070 con 0x559386e83580 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.038+0000 7f9bcd7fa640 1 -- 172.21.15.45:0/768581019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be0012450 con 0x559386e83580 2024-04-25T00:23:07.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.038+0000 7f9bcd7fa640 1 -- 172.21.15.45:0/768581019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be00116e0 con 0x559386e83580 2024-04-25T00:23:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- 172.21.15.45:0/768581019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 msgr2=0x559386e83960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 0x7f9bd00a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9be0011010 tx=0x7f9be0011db0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009f30 tx=0x7f9be0010800 comp rx=0 tx=0).stop 2024-04-25T00:23:07.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- 172.21.15.45:0/768581019 shutdown_connections 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e8e570 0x559386e90960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e83ea0 0x559386e8e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/768581019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e83960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- 172.21.15.45:0/768581019 >> 172.21.15.45:0/768581019 conn(0x559386e7f8a0 msgr2=0x559386e7fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- 172.21.15.45:0/768581019 shutdown_connections 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- 172.21.15.45:0/768581019 wait complete. 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 -- start start 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.039+0000 7f9bf03b1740 1 Processor -- start 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386e8b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e8e570 0x559386e8b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559386e91360 con 0x559386e83580 2024-04-25T00:23:07.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559386da4050 con 0x559386e83ea0 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559386e914e0 con 0x559386e8e570 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49632/0 (socket says 172.21.15.45:49632) 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 -- 172.21.15.45:0/1538576136 learned_addr learned my addr 172.21.15.45:0/1538576136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9bf03b1740 10 client.0 _ll_register_callbacks cb 0x559386afbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be67fc640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386e8b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be77fe640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e8e570 0x559386e8b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 -- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e8e570 msgr2=0x559386e8b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e8e570 0x559386e8b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 -- 172.21.15.45:0/1538576136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 msgr2=0x559386e8b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386e8b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.040+0000 7f9be6ffd640 1 -- 172.21.15.45:0/1538576136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559386d75dc0 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009900 tx=0x7f9bd8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b700602a175519b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8035040 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9bd8004450 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be0012450 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bd8005020 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bf03b1740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559386afc3f0 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be0012450 con 0x559386e83580 2024-04-25T00:23:07.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.041+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bd80035f0 con 0x559386e83580 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 20 client.16041 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e9e4f0 0x559386ea0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 4 client.16041 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559386df5bb0 con 0x559386e9e4f0 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9be67fc640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e9e4f0 0x559386ea0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9be67fc640 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e9e4f0 0x559386ea0980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63e8248cabd024d6 server_cookie=722eb1f0668f3e63 in_seq=0 out_seq=0 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.042+0000 7f9bcaff5640 10 client.16041 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bcaff5640 1 -- 172.21.15.45:0/1538576136 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559386df5bb0 con 0x559386e9e4f0 2024-04-25T00:23:07.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bcaff5640 10 client.16041 handle_command_reply: tid=0 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 client.16041 shutdown 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 msgr2=0x559386e8dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009900 tx=0x7f9bd8031850 comp rx=0 tx=0).stop 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 shutdown_connections 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e9e4f0 0x559386ea0980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bcaff5640 0 client.16041 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e8e570 0x559386e8b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386e8b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/1538576136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e8dea0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 >> 172.21.15.45:0/1538576136 conn(0x559386e7f8a0 msgr2=0x559386e7eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 shutdown_connections 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 1 -- 172.21.15.45:0/1538576136 wait complete. 2024-04-25T00:23:07.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.043+0000 7f9bf03b1740 20 client.16041 trim_cache size 0 max 16384 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.057+0000 7f9bf03b1740 1 Processor -- start 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.057+0000 7f9bf03b1740 1 -- start start 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e9a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83580 0x559386ea62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81b0d10 con 0x559386e83ea0 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81b2850 con 0x559386e83580 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81b13e0 con 0x559386e7f8a0 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49644/0 (socket says 172.21.15.45:49644) 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 -- 172.21.15.45:0/3119560630 learned_addr learned my addr 172.21.15.45:0/3119560630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be67fc640 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83580 0x559386ea62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e9a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 -- 172.21.15.45:0/3119560630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 msgr2=0x559386e9a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e9a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 -- 172.21.15.45:0/3119560630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83580 msgr2=0x559386ea62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83580 0x559386ea62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.058+0000 7f9be77fe640 1 -- 172.21.15.45:0/3119560630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559386bc67d0 con 0x559386e83ea0 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.059+0000 7f9be77fe640 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009740 tx=0x7f9be0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7c5cb02b643b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.059+0000 7f9bccff9640 1 -- 172.21.15.45:0/3119560630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be002d050 con 0x559386e83ea0 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.059+0000 7f9bccff9640 1 -- 172.21.15.45:0/3119560630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be0037070 con 0x559386e83ea0 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.059+0000 7f9bccff9640 1 -- 172.21.15.45:0/3119560630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0011620 con 0x559386e83ea0 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- 172.21.15.45:0/3119560630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 msgr2=0x559386ea67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009740 tx=0x7f9be0027000 comp rx=0 tx=0).stop 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- 172.21.15.45:0/3119560630 shutdown_connections 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83ea0 0x559386ea67e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83580 0x559386ea62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/3119560630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e9a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- 172.21.15.45:0/3119560630 >> 172.21.15.45:0/3119560630 conn(0x559386e8b830 msgr2=0x559386e7eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- 172.21.15.45:0/3119560630 shutdown_connections 2024-04-25T00:23:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- 172.21.15.45:0/3119560630 wait complete. 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 -- start start 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.060+0000 7f9bf03b1740 1 Processor -- start 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e94890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386f03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81b2180 con 0x559386e83580 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81b1ab0 con 0x559386e83ea0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81aecf0 con 0x559386e7f8a0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9bf03b1740 10 client.0 _ll_register_callbacks cb 0x7f9bd8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49660/0 (socket says 172.21.15.45:49660) 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be67fc640 1 -- 172.21.15.45:0/868568891 learned_addr learned my addr 172.21.15.45:0/868568891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386f03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e94890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 -- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 msgr2=0x559386e94890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e94890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 -- 172.21.15.45:0/868568891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 msgr2=0x559386e94dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be77fe640 1 -- 172.21.15.45:0/868568891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559386f04060 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.061+0000 7f9be67fc640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9be77fe640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386f03b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x559386e9d180 tx=0x7f9be001bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c74c6737b4ed6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0016040 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be0037070 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9be00113e0 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9be004f050 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bf03b1740 10 client.25607 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559386f05300 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.062+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be0037070 con 0x559386e83ea0 2024-04-25T00:23:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9be000f040 con 0x559386e83ea0 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 10 client.25607 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 10 client.25607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 20 client.25607 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e8e570 0x559386f07840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 4 client.25607 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559386deff00 con 0x559386e8e570 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e8e570 0x559386f07840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9bcaff5640 10 client.25607 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.063+0000 7f9be6ffd640 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e8e570 0x559386f07840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=184795aa2647dd58 server_cookie=541ff9622ddc9937 in_seq=0 out_seq=0 2024-04-25T00:23:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bcaff5640 1 -- 172.21.15.45:0/868568891 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559386deff00 con 0x559386e8e570 2024-04-25T00:23:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bcaff5640 10 client.25607 handle_command_reply: tid=0 2024-04-25T00:23:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 client.25607 shutdown 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 msgr2=0x559386f03b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386f03b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x559386e9d180 tx=0x7f9be001bbf0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 shutdown_connections 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559386e8e570 0x559386f07840 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559386e83ea0 0x559386f03b20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559386e83580 0x559386e94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bcaff5640 0 client.25607 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 --2- 172.21.15.45:0/868568891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559386e7f8a0 0x559386e94890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 >> 172.21.15.45:0/868568891 conn(0x559386e8b830 msgr2=0x559386e99210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 shutdown_connections 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 1 -- 172.21.15.45:0/868568891 wait complete. 2024-04-25T00:23:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.064+0000 7f9bf03b1740 20 client.25607 trim_cache size 0 max 16384 2024-04-25T00:23:07.070 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 msgr2=0x7f9bd00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 0x7f9bd00a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9be0011010 tx=0x7f9be0011db0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 msgr2=0x7f9be8100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e9f0 tx=0x7f9bdc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 shutdown_connections 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9bd00a2e20 0x7f9bd00a52e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be810bd00 0x7f9be8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be81032e0 0x7f9be8100b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 --2- 172.21.15.45:0/3811487240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9be81029c0 0x7f9be8100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 >> 172.21.15.45:0/3811487240 conn(0x7f9be8077740 msgr2=0x7f9be80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 shutdown_connections 2024-04-25T00:23:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.065+0000 7f9bed6a5640 1 -- 172.21.15.45:0/3811487240 wait complete. 2024-04-25T00:23:07.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:06 smithi045 ceph-mon[29194]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:23:07.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1047055176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:07.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:23:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 -- 172.21.15.45:0/3891191050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 msgr2=0x7f871810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3891191050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871810e1a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8710009ca0 tx=0x7f871002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 -- 172.21.15.45:0/3891191050 shutdown_connections 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3891191050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871810e1a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3891191050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 0x7f871810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3891191050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f8718102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 -- 172.21.15.45:0/3891191050 >> 172.21.15.45:0/3891191050 conn(0x7f8718077740 msgr2=0x7f8718077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 -- 172.21.15.45:0/3891191050 shutdown_connections 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 -- 172.21.15.45:0/3891191050 wait complete. 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.183+0000 7f871fe7b640 1 Processor -- start 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 -- start start 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f871819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 0x7f871819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111b00 con 0x7f871810bd00 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111980 con 0x7f87181032e0 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871fe7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111c80 con 0x7f87181029c0 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49678/0 (socket says 172.21.15.45:49678) 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 -- 172.21.15.45:0/3749551771 learned_addr learned my addr 172.21.15.45:0/3749551771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871dc17640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f871819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871d416640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 0x7f871819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 -- 172.21.15.45:0/3749551771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 msgr2=0x7f871819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f871819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 -- 172.21.15.45:0/3749551771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 msgr2=0x7f871819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 0x7f871819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871e418640 1 -- 172.21.15.45:0/3749551771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871819f230 con 0x7f871810bd00 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.184+0000 7f871dc17640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f871819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.185+0000 7f871e418640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8710006010 tx=0x7f8710003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.185+0000 7f870effd640 1 -- 172.21.15.45:0/3749551771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710047070 con 0x7f871810bd00 2024-04-25T00:23:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.185+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871819f4c0 con 0x7f871810bd00 2024-04-25T00:23:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.185+0000 7f870effd640 1 -- 172.21.15.45:0/3749551771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871002fb20 con 0x7f871810bd00 2024-04-25T00:23:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.185+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181a6db0 con 0x7f871810bd00 2024-04-25T00:23:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.186+0000 7f870effd640 1 -- 172.21.15.45:0/3749551771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710033050 con 0x7f871810bd00 2024-04-25T00:23:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.188+0000 7f870effd640 1 -- 172.21.15.45:0/3749551771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f871004a070 con 0x7f871810bd00 2024-04-25T00:23:07.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.188+0000 7f8722b81740 1 Processor -- start 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 -- start start 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b72960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5555f1aca670 con 0x5555f1b65580 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5555f1a72970 con 0x5555f1b65ea0 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f8722b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5555f1ac6630 con 0x5555f1b70570 2024-04-25T00:23:07.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f870effd640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 0x7f87040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f871d416640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f870effd640 1 -- 172.21.15.45:0/3749551771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f87100f0400 con 0x7f871810bd00 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f871dc17640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f871dc17640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60688/0 (socket says 172.21.15.45:60688) 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f871dc17640 1 -- 172.21.15.45:0/1434748653 learned_addr learned my addr 172.21.15.45:0/1434748653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.189+0000 7f871e418640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 0x7f87040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 -- 172.21.15.45:0/1434748653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 msgr2=0x5555f1b72960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b72960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 -- 172.21.15.45:0/1434748653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 msgr2=0x5555f1b65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b65960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 -- 172.21.15.45:0/1434748653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f1a96400 con 0x5555f1b65ea0 2024-04-25T00:23:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f871dc17640 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87140115c0 tx=0x7f8714011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d12c884440b8c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f8703fff640 1 -- 172.21.15.45:0/1434748653 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400f640 con 0x5555f1b65ea0 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f8703fff640 1 -- 172.21.15.45:0/1434748653 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871400fcc0 con 0x5555f1b65ea0 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.190+0000 7f8703fff640 1 -- 172.21.15.45:0/1434748653 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714019dc0 con 0x5555f1b65ea0 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 -- 172.21.15.45:0/1434748653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 msgr2=0x5555f1b70030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f871e418640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 0x7f87040a51c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87100ed6d0 tx=0x7f87100093d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87140115c0 tx=0x7f8714011a80 comp rx=0 tx=0).stop 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 -- 172.21.15.45:0/1434748653 shutdown_connections 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b72960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65ea0 0x5555f1b70030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 --2- 172.21.15.45:0/1434748653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 -- 172.21.15.45:0/1434748653 >> 172.21.15.45:0/1434748653 conn(0x5555f1b618a0 msgr2=0x5555f1b61c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 -- 172.21.15.45:0/1434748653 shutdown_connections 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.191+0000 7f8722b81740 1 -- 172.21.15.45:0/1434748653 wait complete. 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.192+0000 7f8722b81740 1 -- start start 2024-04-25T00:23:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.192+0000 7f8722b81740 1 Processor -- start 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.192+0000 7f8722b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 0x5555f1b6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.192+0000 7f8722b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871dc17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5555f1b73360 con 0x5555f1b65ea0 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871dc17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b6d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49704/0 (socket says 172.21.15.45:49704) 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871dc17640 1 -- 172.21.15.45:0/1519869282 learned_addr learned my addr 172.21.15.45:0/1519869282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5555f1a86050 con 0x5555f1b65580 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5555f1b734e0 con 0x5555f1b70570 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 0x5555f1b6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 10 client.0 _ll_register_callbacks cb 0x5555f17ddbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 -- 172.21.15.45:0/1519869282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 msgr2=0x5555f1b6d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b6d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 -- 172.21.15.45:0/1519869282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 msgr2=0x5555f1b6d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b6d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f1a57dc0 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f871d416640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 0x5555f1b6ff10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5555f1b62a30 tx=0x7f87080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a013db546f12a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f87017fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708012110 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.193+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f871400fcc0 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8708004260 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8708025050 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 10 client.25619 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555f17de3f0 con 0x5555f1b65580 2024-04-25T00:23:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871400fcc0 con 0x5555f1b65580 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8708028020 con 0x5555f1b65580 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 10 client.25619 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 20 client.25619 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1b80650 0x5555f1b82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 4 client.25619 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.194+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f1ad7bb0 con 0x5555f1b80650 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.195+0000 7f871dc17640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1b80650 0x5555f1b82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.195+0000 7f87017fa640 10 client.25619 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.195+0000 7f871dc17640 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1b80650 0x5555f1b82a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7fc6df7fb410f50 server_cookie=d2dfbfbe73cf515c in_seq=0 out_seq=0 2024-04-25T00:23:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f87017fa640 1 -- 172.21.15.45:0/1519869282 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555f1ad7bb0 con 0x5555f1b80650 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f87017fa640 10 client.25619 handle_command_reply: tid=0 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 client.25619 shutdown 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 msgr2=0x5555f1b6ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 0x5555f1b6ff10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5555f1b62a30 tx=0x7f87080046c0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 shutdown_connections 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b70570 0x5555f1b6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1b80650 0x5555f1b82a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b6d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f87017fa640 0 client.25619 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 --2- 172.21.15.45:0/1519869282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b65580 0x5555f1b6ff10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 >> 172.21.15.45:0/1519869282 conn(0x5555f1b618a0 msgr2=0x5555f1b60b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 shutdown_connections 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 1 -- 172.21.15.45:0/1519869282 wait complete. 2024-04-25T00:23:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.196+0000 7f8722b81740 20 client.25619 trim_cache size 0 max 16384 2024-04-25T00:23:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 Processor -- start 2024-04-25T00:23:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 -- start start 2024-04-25T00:23:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 0x5555f1b7c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b88890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87181b0a60 con 0x5555f1b65ea0 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87181b25a0 con 0x5555f1b70570 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f8722b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87181b1130 con 0x5555f1b65580 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f871dc17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f871dc17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b88350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49718/0 (socket says 172.21.15.45:49718) 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f871dc17640 1 -- 172.21.15.45:0/2594772863 learned_addr learned my addr 172.21.15.45:0/2594772863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.223+0000 7f871d416640 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 0x5555f1b7c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b88890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 -- 172.21.15.45:0/2594772863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 msgr2=0x5555f1b88890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b88890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 -- 172.21.15.45:0/2594772863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 msgr2=0x5555f1b88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 -- 172.21.15.45:0/2594772863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f18a87d0 con 0x5555f1b65580 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f871d416640 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 0x5555f1b7c220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87080059f0 tx=0x7f87080045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=834dc926426ce936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f87037fe640 1 -- 172.21.15.45:0/2594772863 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708005c50 con 0x5555f1b65580 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f87037fe640 1 -- 172.21.15.45:0/2594772863 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8708012110 con 0x5555f1b65580 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.224+0000 7f87037fe640 1 -- 172.21.15.45:0/2594772863 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708005070 con 0x5555f1b65580 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 -- 172.21.15.45:0/2594772863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 msgr2=0x5555f1b7c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 0x5555f1b7c220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87080059f0 tx=0x7f87080045c0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 -- 172.21.15.45:0/2594772863 shutdown_connections 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b88890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65ea0 0x5555f1b88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 --2- 172.21.15.45:0/2594772863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65580 0x5555f1b7c220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 -- 172.21.15.45:0/2594772863 >> 172.21.15.45:0/2594772863 conn(0x5555f1b618a0 msgr2=0x5555f1b60b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 -- 172.21.15.45:0/2594772863 shutdown_connections 2024-04-25T00:23:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.225+0000 7f8722b81740 1 -- 172.21.15.45:0/2594772863 wait complete. 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 -- start start 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 Processor -- start 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65ea0 0x5555f1b76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87181b1ed0 con 0x5555f1b65580 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87181b1800 con 0x5555f1b70570 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.226+0000 7f8722b81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87181aea40 con 0x5555f1b65ea0 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f8722b81740 10 client.0 _ll_register_callbacks cb 0x7f8708009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49728/0 (socket says 172.21.15.45:49728) 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 -- 172.21.15.45:0/4087150680 learned_addr learned my addr 172.21.15.45:0/4087150680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871dc17640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65ea0 0x5555f1b76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 -- 172.21.15.45:0/4087150680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65ea0 msgr2=0x5555f1b76830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65ea0 0x5555f1b76830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 -- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 msgr2=0x5555f1b76d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b76d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 -- 172.21.15.45:0/4087150680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f1be5ca0 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f871d416640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f87080123e0 tx=0x7f87080038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530c098aaa5dfd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f87017fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800f910 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8708032040 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.227+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8708012110 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8708040050 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555f1be6fa0 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8708012110 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8708028070 con 0x5555f1b65580 2024-04-25T00:23:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 20 client.16062 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1be7640 0x5555f1be9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 4 client.16062 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.228+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555f1ad1f00 con 0x5555f1be7640 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f871dc17640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1be7640 0x5555f1be9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f871dc17640 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1be7640 0x5555f1be9a60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9f761b3f8a5097 server_cookie=5897cf2ecae8520f in_seq=0 out_seq=0 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f87017fa640 10 client.16062 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f87017fa640 1 -- 172.21.15.45:0/4087150680 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555f1ad1f00 con 0x5555f1be7640 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f87017fa640 10 client.16062 handle_command_reply: tid=0 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f8722b81740 1 client.16062 shutdown 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 msgr2=0x5555f1b762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.229+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f87080123e0 tx=0x7f87080038e0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 shutdown_connections 2024-04-25T00:23:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555f1b70570 0x5555f1b76d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555f1b65ea0 0x5555f1b76830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5555f1be7640 0x5555f1be9a60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f87017fa640 0 client.16062 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 --2- 172.21.15.45:0/4087150680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555f1b65580 0x5555f1b762f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 >> 172.21.15.45:0/4087150680 conn(0x5555f1b618a0 msgr2=0x5555f1b7b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 shutdown_connections 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 1 -- 172.21.15.45:0/4087150680 wait complete. 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f8722b81740 20 client.16062 trim_cache size 0 max 16384 2024-04-25T00:23:07.237 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 msgr2=0x7f87040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 0x7f87040a51c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f87100ed6d0 tx=0x7f87100093d0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 msgr2=0x7f871819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.230+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8710006010 tx=0x7f8710003ba0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 shutdown_connections 2024-04-25T00:23:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87040a2d00 0x7f87040a51c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f871810bd00 0x7f871819eb00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87181032e0 0x7f871819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 --2- 172.21.15.45:0/3749551771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87181029c0 0x7f871819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 >> 172.21.15.45:0/3749551771 conn(0x7f8718077740 msgr2=0x7f8718107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 shutdown_connections 2024-04-25T00:23:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:07.231+0000 7f871fe7b640 1 -- 172.21.15.45:0/3749551771 wait complete. 2024-04-25T00:23:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:06 smithi122 ceph-mon[32959]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:23:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1047055176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:06 smithi071 ceph-mon[32667]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:23:07.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1047055176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:08.246 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:23:08.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:23:08.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:23:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 -- 172.21.15.45:0/454002925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1029c0 msgr2=0x7faffc102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 --2- 172.21.15.45:0/454002925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1029c0 0x7faffc102da0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7faff802a510 comp rx=0 tx=0).stop 2024-04-25T00:23:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 -- 172.21.15.45:0/454002925 shutdown_connections 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 --2- 172.21.15.45:0/454002925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 0x7faffc10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 --2- 172.21.15.45:0/454002925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1032e0 0x7faffc10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 --2- 172.21.15.45:0/454002925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1029c0 0x7faffc102da0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 -- 172.21.15.45:0/454002925 >> 172.21.15.45:0/454002925 conn(0x7faffc077740 msgr2=0x7faffc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 -- 172.21.15.45:0/454002925 shutdown_connections 2024-04-25T00:23:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.413+0000 7fb003af5640 1 -- 172.21.15.45:0/454002925 wait complete. 2024-04-25T00:23:08.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.414+0000 7fb003af5640 1 Processor -- start 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.414+0000 7fb003af5640 1 -- start start 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.414+0000 7fb003af5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1029c0 0x7faffc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb003af5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb003af5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 0x7faffc19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49752/0 (socket says 172.21.15.45:49752) 2024-04-25T00:23:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 -- 172.21.15.45:0/1335014585 learned_addr learned my addr 172.21.15.45:0/1335014585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faffc111b00 con 0x7faffc1032e0 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faffc111980 con 0x7faffc1029c0 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001891640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1029c0 0x7faffc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faffc111c80 con 0x7faffc10bd00 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb002092640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 0x7faffc19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 -- 172.21.15.45:0/1335014585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 msgr2=0x7faffc19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 0x7faffc19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 -- 172.21.15.45:0/1335014585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1029c0 msgr2=0x7faffc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1029c0 0x7faffc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc19cfd0 con 0x7faffc1032e0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.415+0000 7fb001090640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafec00bec0 tx=0x7fafec00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.416+0000 7faff2ffd640 1 -- 172.21.15.45:0/1335014585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec018070 con 0x7faffc1032e0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.416+0000 7faff2ffd640 1 -- 172.21.15.45:0/1335014585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafec016070 con 0x7faffc1032e0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.416+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc19d2c0 con 0x7faffc1032e0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.416+0000 7faff2ffd640 1 -- 172.21.15.45:0/1335014585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0047a0 con 0x7faffc1032e0 2024-04-25T00:23:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.416+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc100540 con 0x7faffc1032e0 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.419+0000 7fb0067ff740 1 Processor -- start 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 -- start start 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7faff2ffd640 1 -- 172.21.15.45:0/1335014585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fafec004020 con 0x7faffc1032e0 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5614395132c0 con 0x5614395af290 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5614394bb910 con 0x5614395c2820 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb0067ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56143950f280 con 0x5614395b0980 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb001891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb002092640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb001891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49754/0 (socket says 172.21.15.45:49754) 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb001891640 1 -- 172.21.15.45:0/1339310931 learned_addr learned my addr 172.21.15.45:0/1339310931 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.420+0000 7fb001090640 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 -- 172.21.15.45:0/1339310931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 msgr2=0x5614395aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 -- 172.21.15.45:0/1339310931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 msgr2=0x5614395af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 -- 172.21.15.45:0/1339310931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614394ccfb0 con 0x5614395c2820 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001891640 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb001090640 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395c4c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec0d0910 tx=0x7fafec0d5dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3764496e5491818f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7faff2ffd640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 0x7fafe80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fafe7fff640 1 -- 172.21.15.45:0/1339310931 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec09ea60 con 0x5614395c2820 2024-04-25T00:23:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fb002092640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 0x7fafe80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fafe7fff640 1 -- 172.21.15.45:0/1339310931 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafec078590 con 0x5614395c2820 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7fafe7fff640 1 -- 172.21.15.45:0/1339310931 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec09ed90 con 0x5614395c2820 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.421+0000 7faff2ffd640 1 -- 172.21.15.45:0/1335014585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fafec0d3590 con 0x7faffc1032e0 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 -- 172.21.15.45:0/1339310931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 msgr2=0x5614395c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb002092640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 0x7fafe80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff4005fd0 tx=0x7faff4009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395c4c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec0d0910 tx=0x7fafec0d5dc0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 -- 172.21.15.45:0/1339310931 shutdown_connections 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395c4c10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1339310931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.423+0000 7fb0067ff740 1 -- 172.21.15.45:0/1339310931 >> 172.21.15.45:0/1339310931 conn(0x561439577950 msgr2=0x5614395ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.424+0000 7fb0067ff740 1 -- 172.21.15.45:0/1339310931 shutdown_connections 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.424+0000 7fb0067ff740 1 -- 172.21.15.45:0/1339310931 wait complete. 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.424+0000 7fb0067ff740 1 -- start start 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.424+0000 7fb0067ff740 1 Processor -- start 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 0x5614395bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5614395b2320 con 0x5614395b0980 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5614394cee90 con 0x5614395c2820 2024-04-25T00:23:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5614395b24a0 con 0x5614395af290 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb0067ff740 10 client.0 _ll_register_callbacks cb 0x56143950a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49766/0 (socket says 172.21.15.45:49766) 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 -- 172.21.15.45:0/2204121977 learned_addr learned my addr 172.21.15.45:0/2204121977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb002092640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001891640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 0x5614395bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 -- 172.21.15.45:0/2204121977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 msgr2=0x5614395bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 0x5614395bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 -- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 msgr2=0x5614395bcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 -- 172.21.15.45:0/2204121977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614392ef7e0 con 0x5614395b0980 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001891640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 0x5614395bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fb001090640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fafec09f5a0 tx=0x7fafec005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a0398849c3ee57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.425+0000 7fafe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec021020 con 0x5614395b0980 2024-04-25T00:23:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fafec004310 con 0x5614395b0980 2024-04-25T00:23:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafec078590 con 0x5614395b0980 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafec09ef70 con 0x5614395b0980 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fb0067ff740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.426+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56143934ff70 con 0x5614395b0980 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.427+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafec078590 con 0x5614395b0980 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.427+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafec09bc80 con 0x5614395b0980 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 20 client.16074 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5614395dadc0 0x5614395dd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 4 client.16074 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614394e75a0 con 0x5614395dadc0 2024-04-25T00:23:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb001891640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5614395dadc0 0x5614395dd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fafe57fa640 10 client.16074 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.428+0000 7fb001891640 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5614395dadc0 0x5614395dd1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50279f1871dedf66 server_cookie=96db8895f25ca3a5 in_seq=0 out_seq=0 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fafe57fa640 1 -- 172.21.15.45:0/2204121977 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614394e75a0 con 0x5614395dadc0 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fafe57fa640 10 client.16074 handle_command_reply: tid=0 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 client.16074 shutdown 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 msgr2=0x5614395c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fafec09f5a0 tx=0x7fafec005bf0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 shutdown_connections 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5614395dadc0 0x5614395dd1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.429+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395b0980 0x5614395c2190 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fb0067ff740 1 --2- 172.21.15.45:0/2204121977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395af290 0x5614395bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 >> 172.21.15.45:0/2204121977 conn(0x561439577950 msgr2=0x5614395b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fafe57fa640 0 client.16074 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 shutdown_connections 2024-04-25T00:23:08.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fb0067ff740 1 -- 172.21.15.45:0/2204121977 wait complete. 2024-04-25T00:23:08.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.430+0000 7fb0067ff740 20 client.16074 trim_cache size 0 max 16384 2024-04-25T00:23:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 Processor -- start 2024-04-25T00:23:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 -- start start 2024-04-25T00:23:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1ac720 con 0x5614395af290 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1ae260 con 0x5614395c2820 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb001891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb001891640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49782/0 (socket says 172.21.15.45:49782) 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb001891640 1 -- 172.21.15.45:0/1319049379 learned_addr learned my addr 172.21.15.45:0/1319049379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb0067ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1acdf0 con 0x5614395b0980 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb002092640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.466+0000 7fb001090640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 -- 172.21.15.45:0/1319049379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 msgr2=0x5614395aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 -- 172.21.15.45:0/1319049379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 msgr2=0x5614395bc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 -- 172.21.15.45:0/1319049379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614391e7170 con 0x5614395af290 2024-04-25T00:23:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb002092640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:08.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fb001891640 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faff8005940 tx=0x7faff80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9cd357797e4237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fafe77fe640 1 -- 172.21.15.45:0/1319049379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff804b070 con 0x5614395af290 2024-04-25T00:23:08.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fafe77fe640 1 -- 172.21.15.45:0/1319049379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8046dc0 con 0x5614395af290 2024-04-25T00:23:08.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.467+0000 7fafe77fe640 1 -- 172.21.15.45:0/1319049379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8035040 con 0x5614395af290 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 -- 172.21.15.45:0/1319049379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 msgr2=0x5614395c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faff8005940 tx=0x7faff80058d0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 -- 172.21.15.45:0/1319049379 shutdown_connections 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395c2820 0x5614395bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395b0980 0x5614395aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1319049379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395af290 0x5614395c2100 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 -- 172.21.15.45:0/1319049379 >> 172.21.15.45:0/1319049379 conn(0x561439577950 msgr2=0x5614395aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 -- 172.21.15.45:0/1319049379 shutdown_connections 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.468+0000 7fb0067ff740 1 -- 172.21.15.45:0/1319049379 wait complete. 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 -- start start 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 Processor -- start 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 0x5614395dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 0x56143963ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1adb90 con 0x5614395c2820 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1ad4c0 con 0x5614395b0980 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1aa7b0 con 0x5614395de3b0 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.469+0000 7fb0067ff740 10 client.0 _ll_register_callbacks cb 0x7fafec0d5f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49792/0 (socket says 172.21.15.45:49792) 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 -- 172.21.15.45:0/1452501382 learned_addr learned my addr 172.21.15.45:0/1452501382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001891640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 0x5614395dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb002092640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 0x56143963ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 -- 172.21.15.45:0/1452501382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 msgr2=0x56143963ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 0x56143963ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 -- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 msgr2=0x5614395dd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 0x5614395dd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 -- 172.21.15.45:0/1452501382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561439640510 con 0x5614395c2820 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb002092640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 0x56143963ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001891640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 0x5614395dd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb001090640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fafec009610 tx=0x7fafec09f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d349fd1f6fedb2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fafe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec021040 con 0x5614395c2820 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fafec004310 con 0x5614395c2820 2024-04-25T00:23:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.470+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff8046dc0 con 0x5614395c2820 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafec09f150 con 0x5614395c2820 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614396417a0 con 0x5614395c2820 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8046dc0 con 0x5614395c2820 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafec09bc80 con 0x5614395c2820 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 20 client.16080 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561439641c10 0x561439644070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 4 client.16080 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.471+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561439520800 con 0x561439641c10 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.472+0000 7fb001891640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561439641c10 0x561439644070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.472+0000 7fafe57fa640 10 client.16080 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.472+0000 7fb001891640 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561439641c10 0x561439644070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a42ee920f6ddfc24 server_cookie=87958c2137f1bb28 in_seq=0 out_seq=0 2024-04-25T00:23:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.473+0000 7fafe57fa640 1 -- 172.21.15.45:0/1452501382 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561439520800 con 0x561439641c10 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.473+0000 7fafe57fa640 10 client.16080 handle_command_reply: tid=0 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.473+0000 7fb0067ff740 1 client.16080 shutdown 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 msgr2=0x5614395dde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fafec009610 tx=0x7fafec09f610 comp rx=0 tx=0).stop 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 shutdown_connections 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614395de3b0 0x56143963ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561439641c10 0x561439644070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614395c2820 0x5614395dde70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 --2- 172.21.15.45:0/1452501382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614395b0980 0x5614395dd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 >> 172.21.15.45:0/1452501382 conn(0x561439577950 msgr2=0x5614395aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fafe57fa640 0 client.16080 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 shutdown_connections 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 1 -- 172.21.15.45:0/1452501382 wait complete. 2024-04-25T00:23:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.474+0000 7fb0067ff740 20 client.16080 trim_cache size 0 max 16384 2024-04-25T00:23:08.479 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:08.479 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:23:08.479 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "7aeb0628-9639-4307-99c5-a8e98d9b4ee3", 2024-04-25T00:23:08.479 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:23:08.479 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 msgr2=0x7fafe80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 0x7fafe80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faff4005fd0 tx=0x7faff4009500 comp rx=0 tx=0).stop 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 msgr2=0x7faffc198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafec00bec0 tx=0x7fafec00dd20 comp rx=0 tx=0).stop 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 shutdown_connections 2024-04-25T00:23:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fafe80a2e20 0x7fafe80a52e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faffc10bd00 0x7faffc19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faffc1032e0 0x7faffc198520 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 --2- 172.21.15.45:0/1335014585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faffc1029c0 0x7faffc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 >> 172.21.15.45:0/1335014585 conn(0x7faffc077740 msgr2=0x7faffc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 shutdown_connections 2024-04-25T00:23:08.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.475+0000 7fb003af5640 1 -- 172.21.15.45:0/1335014585 wait complete. 2024-04-25T00:23:08.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:08.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 -- 172.21.15.45:0/2192949152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a010bbc0 msgr2=0x7f79a010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 --2- 172.21.15.45:0/2192949152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a010bbc0 0x7f79a010e060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ca0 tx=0x7f799802a950 comp rx=0 tx=0).stop 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 -- 172.21.15.45:0/2192949152 shutdown_connections 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 --2- 172.21.15.45:0/2192949152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a010bbc0 0x7f79a010e060 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 --2- 172.21.15.45:0/2192949152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01030c0 0x7f79a010b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 --2- 172.21.15.45:0/2192949152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a01027a0 0x7f79a0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 -- 172.21.15.45:0/2192949152 >> 172.21.15.45:0/2192949152 conn(0x7f79a00fbfc0 msgr2=0x7f79a00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 -- 172.21.15.45:0/2192949152 shutdown_connections 2024-04-25T00:23:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.603+0000 7f79a5da0640 1 -- 172.21.15.45:0/2192949152 wait complete. 2024-04-25T00:23:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 Processor -- start 2024-04-25T00:23:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 -- start start 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 0x7f79a0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 0x7f79a019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01119c0 con 0x7f79a01030c0 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0111840 con 0x7f79a010bbc0 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a5da0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0111b40 con 0x7f79a01027a0 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49800/0 (socket says 172.21.15.45:49800) 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 -- 172.21.15.45:0/92501252 learned_addr learned my addr 172.21.15.45:0/92501252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 0x7f79a0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a559f640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 0x7f79a019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 -- 172.21.15.45:0/92501252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 msgr2=0x7f79a0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 0x7f79a0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 -- 172.21.15.45:0/92501252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 msgr2=0x7f79a019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 0x7f79a019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f799ffff640 1 -- 172.21.15.45:0/92501252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a019ce60 con 0x7f79a01030c0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a559f640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 0x7f79a019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.604+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 0x7f79a0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f799ffff640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f798c00d740 tx=0x7f798c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f799dffb640 1 -- 172.21.15.45:0/92501252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c017070 con 0x7f79a01030c0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f799dffb640 1 -- 172.21.15.45:0/92501252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c00de50 con 0x7f79a01030c0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a019d150 con 0x7f79a01030c0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f799dffb640 1 -- 172.21.15.45:0/92501252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00f620 con 0x7f79a01030c0 2024-04-25T00:23:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.605+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a49e0 con 0x7f79a01030c0 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 Processor -- start 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 -- start start 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42152800 0x562f42152be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562f420b76b0 con 0x562f42153120 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562f4205f880 con 0x562f4215d7f0 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a9dea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562f420b3670 con 0x562f42152800 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49814/0 (socket says 172.21.15.45:49814) 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.608+0000 7f79a4d9e640 1 -- 172.21.15.45:0/186763708 learned_addr learned my addr 172.21.15.45:0/186763708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a559f640 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42152800 0x562f42152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 -- 172.21.15.45:0/186763708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42152800 msgr2=0x562f42152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42152800 0x562f42152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 -- 172.21.15.45:0/186763708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 msgr2=0x562f4215fbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 -- 172.21.15.45:0/186763708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f41e957d0 con 0x562f42153120 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f799400bd40 tx=0x7f799400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b127a1927d4706c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.609+0000 7f799dffb640 1 -- 172.21.15.45:0/92501252 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f798c009c30 con 0x7f79a01030c0 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.610+0000 7f799dffb640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 0x7f79900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.610+0000 7f79a559f640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 0x7f79900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.610+0000 7f799dffb640 1 -- 172.21.15.45:0/92501252 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f798c0cf460 con 0x7f79a01030c0 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.610+0000 7f798affd640 1 -- 172.21.15.45:0/186763708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994015070 con 0x562f42153120 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.611+0000 7f798affd640 1 -- 172.21.15.45:0/186763708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799400be30 con 0x562f42153120 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.611+0000 7f798affd640 1 -- 172.21.15.45:0/186763708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79940107f0 con 0x562f42153120 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.611+0000 7f79a9dea740 1 -- 172.21.15.45:0/186763708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 msgr2=0x562f4215d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.611+0000 7f79a559f640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 0x7f79900a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7998030040 tx=0x7f7998005d80 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.611+0000 7f79a9dea740 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f799400bd40 tx=0x7f799400f730 comp rx=0 tx=0).stop 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 -- 172.21.15.45:0/186763708 shutdown_connections 2024-04-25T00:23:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215d2b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 --2- 172.21.15.45:0/186763708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42152800 0x562f42152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 -- 172.21.15.45:0/186763708 >> 172.21.15.45:0/186763708 conn(0x562f4214eb20 msgr2=0x562f4214ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 -- 172.21.15.45:0/186763708 shutdown_connections 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 -- 172.21.15.45:0/186763708 wait complete. 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 -- start start 2024-04-25T00:23:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.612+0000 7f79a9dea740 1 Processor -- start 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42153120 0x562f4215ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562f421605e0 con 0x562f42152800 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562f42072f50 con 0x562f4215d7f0 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562f42160760 con 0x562f42153120 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49824/0 (socket says 172.21.15.45:49824) 2024-04-25T00:23:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 -- 172.21.15.45:0/2055638370 learned_addr learned my addr 172.21.15.45:0/2055638370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a9dea740 10 client.0 _ll_register_callbacks cb 0x562f420ae000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f799ffff640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42153120 0x562f4215ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a559f640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 -- 172.21.15.45:0/2055638370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42153120 msgr2=0x562f4215ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42153120 0x562f4215ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 -- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 msgr2=0x562f4215b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a4d9e640 1 -- 172.21.15.45:0/2055638370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f41dcb3f0 con 0x562f42152800 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.613+0000 7f79a559f640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7994011f90 tx=0x7f7994004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8a288968afdb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f797ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994015040 con 0x562f42152800 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7994011440 con 0x562f42152800 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799400be30 con 0x562f42152800 2024-04-25T00:23:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7994011640 con 0x562f42152800 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f79a9dea740 10 client.16092 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.614+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f41d8b0f0 con 0x562f42152800 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799400be30 con 0x562f42152800 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7994010570 con 0x562f42152800 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 20 client.16092 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f4216db10 0x562f4216ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 4 client.16092 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.615+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f420c4bf0 con 0x562f4216db10 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.616+0000 7f799ffff640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f4216db10 0x562f4216ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.616+0000 7f797ffff640 10 client.16092 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.616+0000 7f799ffff640 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f4216db10 0x562f4216ff00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b619c8ccd537ba3d server_cookie=2c7688f0825757ed in_seq=0 out_seq=0 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f797ffff640 1 -- 172.21.15.45:0/2055638370 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f420c4bf0 con 0x562f4216db10 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f797ffff640 10 client.16092 handle_command_reply: tid=0 2024-04-25T00:23:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 client.16092 shutdown 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 msgr2=0x562f4215a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7994011f90 tx=0x7f7994004270 comp rx=0 tx=0).stop 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 shutdown_connections 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f4215b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f42153120 0x562f4215ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f4216db10 0x562f4216ff00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 --2- 172.21.15.45:0/2055638370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42152800 0x562f4215a7d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 >> 172.21.15.45:0/2055638370 conn(0x562f4214eb20 msgr2=0x562f42154920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f797ffff640 0 client.16092 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.617+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 shutdown_connections 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.618+0000 7f79a9dea740 1 -- 172.21.15.45:0/2055638370 wait complete. 2024-04-25T00:23:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.618+0000 7f79a9dea740 20 client.16092 trim_cache size 0 max 16384 2024-04-25T00:23:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.652+0000 7f79a9dea740 1 Processor -- start 2024-04-25T00:23:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.652+0000 7f79a9dea740 1 -- start start 2024-04-25T00:23:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 0x562f421694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 0x562f421755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49838/0 (socket says 172.21.15.45:49838) 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 -- 172.21.15.45:0/3528132985 learned_addr learned my addr 172.21.15.45:0/3528132985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01aea40 con 0x562f42153120 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01b0580 con 0x562f4214da20 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01af110 con 0x562f4215d7f0 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f799ffff640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 0x562f421694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 -- 172.21.15.45:0/3528132985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 msgr2=0x562f421694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a559f640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 0x562f421755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 0x562f421694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 -- 172.21.15.45:0/3528132985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 msgr2=0x562f421755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 0x562f421755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.653+0000 7f79a4d9e640 1 -- 172.21.15.45:0/3528132985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f42152600 con 0x562f42153120 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f799ffff640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 0x562f421694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f79a559f640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 0x562f421755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7994010970 tx=0x7f799400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125920f8e9a34773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f798a7fc640 1 -- 172.21.15.45:0/3528132985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799403b070 con 0x562f42153120 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f798a7fc640 1 -- 172.21.15.45:0/3528132985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994028070 con 0x562f42153120 2024-04-25T00:23:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.654+0000 7f798a7fc640 1 -- 172.21.15.45:0/3528132985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994015070 con 0x562f42153120 2024-04-25T00:23:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 msgr2=0x562f4215cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7994010970 tx=0x7f799400b480 comp rx=0 tx=0).stop 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 shutdown_connections 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4214da20 0x562f421755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4215d7f0 0x562f421694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 --2- 172.21.15.45:0/3528132985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42153120 0x562f4215cf00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 >> 172.21.15.45:0/3528132985 conn(0x562f4214eb20 msgr2=0x562f4214d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 shutdown_connections 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- 172.21.15.45:0/3528132985 wait complete. 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 -- start start 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.655+0000 7f79a9dea740 1 Processor -- start 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4214da20 0x562f42165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f42157100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01afeb0 con 0x562f42157640 2024-04-25T00:23:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01af7e0 con 0x562f4215d7f0 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01aca20 con 0x562f4214da20 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a9dea740 10 client.0 _ll_register_callbacks cb 0x7f799400bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49846/0 (socket says 172.21.15.45:49846) 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 -- 172.21.15.45:0/1215885468 learned_addr learned my addr 172.21.15.45:0/1215885468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4214da20 0x562f42165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f799ffff640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f42157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 -- 172.21.15.45:0/1215885468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4214da20 msgr2=0x562f42165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4214da20 0x562f42165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 -- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 msgr2=0x562f42157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f42157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.656+0000 7f79a559f640 1 -- 172.21.15.45:0/1215885468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f421d2f50 con 0x562f42157640 2024-04-25T00:23:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f79a559f640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fe0 tx=0x7f799802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e23a98af63bd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f797ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998034040 con 0x562f42157640 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7998040030 con 0x562f42157640 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7994028070 con 0x562f42157640 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7998004040 con 0x562f42157640 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f79a9dea740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f421d4190 con 0x562f42157640 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.657+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994028070 con 0x562f42157640 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7998030910 con 0x562f42157640 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 20 client.16098 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f421d48e0 0x562f421d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 4 client.16098 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f420bef40 con 0x562f421d48e0 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f421d48e0 0x562f421d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f797ffff640 10 client.16098 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.658+0000 7f79a4d9e640 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f421d48e0 0x562f421d6d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf93b4a616268f65 server_cookie=6ab78aea45259b59 in_seq=0 out_seq=0 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f797ffff640 1 -- 172.21.15.45:0/1215885468 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562f420bef40 con 0x562f421d48e0 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f797ffff640 10 client.16098 handle_command_reply: tid=0 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 client.16098 shutdown 2024-04-25T00:23:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 msgr2=0x562f42157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fe0 tx=0x7f799802a950 comp rx=0 tx=0).stop 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 shutdown_connections 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562f421d48e0 0x562f421d6d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562f42157640 0x562f42157aa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f797ffff640 0 client.16098 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f4215d7f0 0x562f42157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 --2- 172.21.15.45:0/1215885468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562f4214da20 0x562f42165dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 >> 172.21.15.45:0/1215885468 conn(0x562f4214eb20 msgr2=0x562f4215a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 shutdown_connections 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 1 -- 172.21.15.45:0/1215885468 wait complete. 2024-04-25T00:23:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.659+0000 7f79a9dea740 20 client.16098 trim_cache size 0 max 16384 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "7aeb0628-9639-4307-99c5-a8e98d9b4ee3": { 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "7aeb0628-9639-4307-99c5-a8e98d9b4ee3", 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:23:08.669 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 msgr2=0x7f79900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 0x7f79900a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7998030040 tx=0x7f7998005d80 comp rx=0 tx=0).stop 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 msgr2=0x7f79a0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f798c00d740 tx=0x7f798c00dc00 comp rx=0 tx=0).stop 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 shutdown_connections 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79900a2f20 0x7f79900a53e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79a010bbc0 0x7f79a019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79a01030c0 0x7f79a0198400 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 --2- 172.21.15.45:0/92501252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a01027a0 0x7f79a0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:08.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 >> 172.21.15.45:0/92501252 conn(0x7f79a00fbfc0 msgr2=0x7f79a00fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:08.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 shutdown_connections 2024-04-25T00:23:08.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:08.660+0000 7f79a5da0640 1 -- 172.21.15.45:0/92501252 wait complete. 2024-04-25T00:23:08.675 INFO:tasks.cephfs.filesystem:scrub status for tag:7aeb0628-9639-4307-99c5-a8e98d9b4ee3 - {'path': '/', 'tag': '7aeb0628-9639-4307-99c5-a8e98d9b4ee3', 'options': 'recursive,force'} 2024-04-25T00:23:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:08 smithi122 ceph-mon[32959]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:08 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:23:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:08 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:08 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:23:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:08 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:08 smithi071 ceph-mon[32667]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:08 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:23:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:08 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:08 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:23:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:08 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:08 smithi045 ceph-mon[29194]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:08 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:23:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:08 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:08 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:23:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:08 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:10 smithi122 ceph-mon[32959]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T00:23:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:10 smithi071 ceph-mon[32667]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T00:23:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:10 smithi045 ceph-mon[29194]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T00:23:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:12 smithi122 ceph-mon[32959]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:12 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:23:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:12 smithi071 ceph-mon[32667]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:12 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:23:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:12 smithi045 ceph-mon[29194]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:23:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:12 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:23:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:14 smithi122 ceph-mon[32959]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:23:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:14 smithi071 ceph-mon[32667]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:23:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:14 smithi045 ceph-mon[29194]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:23:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:16 smithi122 ceph-mon[32959]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T00:23:17.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:16 smithi071 ceph-mon[32667]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T00:23:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:16 smithi045 ceph-mon[29194]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T00:23:18.379 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 94% (11828/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T00:23:18.923 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:18.924 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T00:23:18.927 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T00:23:18.965 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T00:23:18.967 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:23:18.967 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T00:23:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:18 smithi122 ceph-mon[32959]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:23:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:18 smithi071 ceph-mon[32667]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:23:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:18 smithi045 ceph-mon[29194]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:23:19.563 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T00:23:19.624 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T00:23:19.665 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:23:19.667 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:23:19.667 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T00:23:19.707 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:23:19.710 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:23:19.710 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T00:23:19.766 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-04-25T00:23:19.768 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-04-25T00:23:19.770 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-25T00:23:19.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-04-25T00:23:19.854 INFO:tasks.workunit.client.0.smithi045.stderr:--2024-04-25 00:23:19-- http://download.ceph.com/qa/fsync-tester.c 2024-04-25T00:23:19.854 INFO:tasks.workunit.client.0.smithi045.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-25T00:23:19.891 INFO:tasks.workunit.client.0.smithi045.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-25T00:23:19.928 INFO:tasks.workunit.client.0.smithi045.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-25T00:23:19.928 INFO:tasks.workunit.client.0.smithi045.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-04-25T00:23:19.929 INFO:tasks.workunit.client.0.smithi045.stderr:Saving to: ‘fsync-tester.c’ 2024-04-25T00:23:19.932 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:19.932 INFO:tasks.workunit.client.0.smithi045.stderr: 0K .. 100% 556M=0s 2024-04-25T00:23:19.932 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:19.933 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-25 00:23:19 (556 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-04-25T00:23:19.933 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T00:23:19.933 INFO:tasks.workunit.client.0.smithi045.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-04-25T00:23:20.008 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./fsync-tester 2024-04-25T00:23:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:20 smithi122 ceph-mon[32959]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:23:21.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:20 smithi071 ceph-mon[32667]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:23:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:20 smithi045 ceph-mon[29194]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:23:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:23 smithi122 ceph-mon[32959]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:23:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:23 smithi071 ceph-mon[32667]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:23:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:23 smithi045 ceph-mon[29194]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:23:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:25 smithi122 ceph-mon[32959]: pgmap v46: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 348 KiB/s wr, 14 op/s 2024-04-25T00:23:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:25 smithi071 ceph-mon[32667]: pgmap v46: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 348 KiB/s wr, 14 op/s 2024-04-25T00:23:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:25 smithi045 ceph-mon[29194]: pgmap v46: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 348 KiB/s wr, 14 op/s 2024-04-25T00:23:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:27 smithi122 ceph-mon[32959]: pgmap v47: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:27 smithi071 ceph-mon[32667]: pgmap v47: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:27 smithi045 ceph-mon[29194]: pgmap v47: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:29 smithi122 ceph-mon[32959]: pgmap v48: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:29 smithi071 ceph-mon[32667]: pgmap v48: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:29 smithi045 ceph-mon[29194]: pgmap v48: 129 pgs: 129 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 348 KiB/s wr, 10 op/s 2024-04-25T00:23:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:31 smithi122 ceph-mon[32959]: pgmap v49: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.4 MiB/s wr, 17 op/s 2024-04-25T00:23:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:31 smithi122 ceph-mon[32959]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-25T00:23:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:31 smithi071 ceph-mon[32667]: pgmap v49: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.4 MiB/s wr, 17 op/s 2024-04-25T00:23:31.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:31 smithi071 ceph-mon[32667]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-25T00:23:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:31 smithi045 ceph-mon[29194]: pgmap v49: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.4 MiB/s wr, 17 op/s 2024-04-25T00:23:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:31 smithi045 ceph-mon[29194]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-25T00:23:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:33 smithi122 ceph-mon[32959]: pgmap v50: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 MiB/s wr, 13 op/s 2024-04-25T00:23:33.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:33 smithi071 ceph-mon[32667]: pgmap v50: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 MiB/s wr, 13 op/s 2024-04-25T00:23:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:33 smithi045 ceph-mon[29194]: pgmap v50: 129 pgs: 129 active+clean; 34 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 MiB/s wr, 13 op/s 2024-04-25T00:23:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:35 smithi122 ceph-mon[32959]: pgmap v51: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 23 op/s 2024-04-25T00:23:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:35 smithi071 ceph-mon[32667]: pgmap v51: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 23 op/s 2024-04-25T00:23:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:35 smithi045 ceph-mon[29194]: pgmap v51: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 23 op/s 2024-04-25T00:23:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:37 smithi122 ceph-mon[32959]: pgmap v52: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:37 smithi071 ceph-mon[32667]: pgmap v52: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:37.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:37 smithi045 ceph-mon[29194]: pgmap v52: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:38.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:38.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 -- 172.21.15.45:0/3615042694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc041077a0 msgr2=0x7efc04107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 --2- 172.21.15.45:0/3615042694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc041077a0 0x7efc04107b80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009930 tx=0x7efbf402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 -- 172.21.15.45:0/3615042694 shutdown_connections 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 --2- 172.21.15.45:0/3615042694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc04102150 0x7efc0410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 --2- 172.21.15.45:0/3615042694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04101790 0x7efc04101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 --2- 172.21.15.45:0/3615042694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc041077a0 0x7efc04107b80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 -- 172.21.15.45:0/3615042694 >> 172.21.15.45:0/3615042694 conn(0x7efc040fd2a0 msgr2=0x7efc040ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 -- 172.21.15.45:0/3615042694 shutdown_connections 2024-04-25T00:23:38.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.789+0000 7efc088cd640 1 -- 172.21.15.45:0/3615042694 wait complete. 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 Processor -- start 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 -- start start 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102150 0x7efc04198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc041077a0 0x7efc0419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111b70 con 0x7efc04101790 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc041119f0 con 0x7efc04102150 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc088cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111cf0 con 0x7efc041077a0 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34330/0 (socket says 172.21.15.45:34330) 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.790+0000 7efc0259c640 1 -- 172.21.15.45:0/1033813758 learned_addr learned my addr 172.21.15.45:0/1033813758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc02d9d640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc041077a0 0x7efc0419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102150 0x7efc04198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 -- 172.21.15.45:0/1033813758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc041077a0 msgr2=0x7efc0419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc041077a0 0x7efc0419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 -- 172.21.15.45:0/1033813758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102150 msgr2=0x7efc04198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102150 0x7efc04198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 -- 172.21.15.45:0/1033813758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0419d0f0 con 0x7efc04101790 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc0259c640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009230 tx=0x7efbf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efbfb7fe640 1 -- 172.21.15.45:0/1033813758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf403f030 con 0x7efc04101790 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0419d380 con 0x7efc04101790 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efbfb7fe640 1 -- 172.21.15.45:0/1033813758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4003ad0 con 0x7efc04101790 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efbfb7fe640 1 -- 172.21.15.45:0/1033813758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4035070 con 0x7efc04101790 2024-04-25T00:23:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.791+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a4c70 con 0x7efc04101790 2024-04-25T00:23:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.793+0000 7efbfb7fe640 1 -- 172.21.15.45:0/1033813758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efbf4042030 con 0x7efc04101790 2024-04-25T00:23:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.794+0000 7efbfb7fe640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 0x7efbf00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.794+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 0x7efbf00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.794+0000 7efbfb7fe640 1 -- 172.21.15.45:0/1033813758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7efbf40efc00 con 0x7efc04101790 2024-04-25T00:23:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.795+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 0x7efbf00a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc040ff9e0 tx=0x7efbec00b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.795+0000 7efc0b5da740 1 Processor -- start 2024-04-25T00:23:38.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 -- start start 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f73da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f74aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56308f6a26b0 con 0x56308f73df50 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56308f64a880 con 0x56308f73d630 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0b5da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56308f69e670 con 0x56308f748620 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34340/0 (socket says 172.21.15.45:34340) 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 -- 172.21.15.45:0/629343917 learned_addr learned my addr 172.21.15.45:0/629343917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0259c640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f73da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc02d9d640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f74aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 -- 172.21.15.45:0/629343917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 msgr2=0x56308f74aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f74aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 -- 172.21.15.45:0/629343917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 msgr2=0x56308f73da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f73da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efbfbfff640 1 -- 172.21.15.45:0/629343917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308f4807d0 con 0x56308f73df50 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.796+0000 7efc0259c640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f73da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.797+0000 7efbfbfff640 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b6e0 tx=0x7efbec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154978123fae8456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.797+0000 7efbf8ff9640 1 -- 172.21.15.45:0/629343917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec005220 con 0x56308f73df50 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.797+0000 7efbf8ff9640 1 -- 172.21.15.45:0/629343917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec004c80 con 0x56308f73df50 2024-04-25T00:23:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.797+0000 7efbf8ff9640 1 -- 172.21.15.45:0/629343917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec01e960 con 0x56308f73df50 2024-04-25T00:23:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- 172.21.15.45:0/629343917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 msgr2=0x56308f7480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b6e0 tx=0x7efbec013000 comp rx=0 tx=0).stop 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- 172.21.15.45:0/629343917 shutdown_connections 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f74aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f7480e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 --2- 172.21.15.45:0/629343917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f73da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- 172.21.15.45:0/629343917 >> 172.21.15.45:0/629343917 conn(0x56308f739950 msgr2=0x56308f739d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- 172.21.15.45:0/629343917 shutdown_connections 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- 172.21.15.45:0/629343917 wait complete. 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.798+0000 7efc0b5da740 1 -- start start 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 Processor -- start 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f745b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f746080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56308f74b410 con 0x56308f73d630 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56308f65df50 con 0x56308f73df50 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56308f74b590 con 0x56308f748620 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0b5da740 10 client.0 _ll_register_callbacks cb 0x56308f699000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34344/0 (socket says 172.21.15.45:34344) 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 -- 172.21.15.45:0/3682013501 learned_addr learned my addr 172.21.15.45:0/3682013501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efbfbfff640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f745b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc02d9d640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f746080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 -- 172.21.15.45:0/3682013501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 msgr2=0x56308f746080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f746080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 -- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 msgr2=0x56308f745b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f745b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 -- 172.21.15.45:0/3682013501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308f3b63f0 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efbfbfff640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f745b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.799+0000 7efc0259c640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efbf40b8840 tx=0x7efbf40bbe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9e23bebc02db4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4049040 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7efbf4003c80 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbec004c80 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efbf40c4a30 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56308f3760f0 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbec004c80 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efbf403bb90 con 0x56308f73d630 2024-04-25T00:23:38.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 20 client.16110 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f758860 0x56308f75acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 4 client.16110 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.800+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308f6afbf0 con 0x56308f758860 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.801+0000 7efbfbfff640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f758860 0x56308f75acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.801+0000 7efbfbfff640 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f758860 0x56308f75acb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9e31f0a349331c server_cookie=c4b4b5a4d533094f in_seq=0 out_seq=0 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.801+0000 7efbe9ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efbe9ffb640 1 -- 172.21.15.45:0/3682013501 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56308f6afbf0 con 0x56308f758860 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efbe9ffb640 10 client.16110 handle_command_reply: tid=0 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 client.16110 shutdown 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 msgr2=0x56308f745600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efbf40b8840 tx=0x7efbf40bbe70 comp rx=0 tx=0).stop 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 shutdown_connections 2024-04-25T00:23:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f748620 0x56308f746080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f745b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f758860 0x56308f75acb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 --2- 172.21.15.45:0/3682013501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73d630 0x56308f745600 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efbe9ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 >> 172.21.15.45:0/3682013501 conn(0x56308f739950 msgr2=0x56308f73f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 shutdown_connections 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.802+0000 7efc0b5da740 1 -- 172.21.15.45:0/3682013501 wait complete. 2024-04-25T00:23:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.803+0000 7efc0b5da740 20 client.16110 trim_cache size 0 max 16384 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 Processor -- start 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 -- start start 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f739950 0x56308f739270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f754300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc041aec10 con 0x56308f73df50 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b0750 con 0x56308f73d630 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.821+0000 7efc0b5da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc041af2e0 con 0x56308f739950 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34350/0 (socket says 172.21.15.45:34350) 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 -- 172.21.15.45:0/2659350673 learned_addr learned my addr 172.21.15.45:0/2659350673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc0259c640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f739950 0x56308f739270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efbfbfff640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f754300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 -- 172.21.15.45:0/2659350673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f739950 msgr2=0x56308f739270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f739950 0x56308f739270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 -- 172.21.15.45:0/2659350673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 msgr2=0x56308f754300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f754300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 -- 172.21.15.45:0/2659350673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308f746560 con 0x56308f73df50 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efbfbfff640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f754300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efc02d9d640 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009ee0 tx=0x7efbfc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155916adc25befd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efbebfff640 1 -- 172.21.15.45:0/2659350673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc003970 con 0x56308f73df50 2024-04-25T00:23:38.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efbebfff640 1 -- 172.21.15.45:0/2659350673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc004020 con 0x56308f73df50 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.822+0000 7efbebfff640 1 -- 172.21.15.45:0/2659350673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc01b7b0 con 0x56308f73df50 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- 172.21.15.45:0/2659350673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 msgr2=0x56308f760550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009ee0 tx=0x7efbfc0127c0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- 172.21.15.45:0/2659350673 shutdown_connections 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f73df50 0x56308f760550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73d630 0x56308f754300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 --2- 172.21.15.45:0/2659350673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f739950 0x56308f739270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- 172.21.15.45:0/2659350673 >> 172.21.15.45:0/2659350673 conn(0x56308f7442a0 msgr2=0x56308f745e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- 172.21.15.45:0/2659350673 shutdown_connections 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- 172.21.15.45:0/2659350673 wait complete. 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.823+0000 7efc0b5da740 1 -- start start 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 Processor -- start 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 0x56308f750d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f74d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b0080 con 0x56308f739950 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc041af9b0 con 0x56308f73df50 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc041acbf0 con 0x56308f73d630 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0b5da740 10 client.0 _ll_register_callbacks cb 0x7efbf4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34360/0 (socket says 172.21.15.45:34360) 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 -- 172.21.15.45:0/1082253131 learned_addr learned my addr 172.21.15.45:0/1082253131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 0x56308f750d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc02d9d640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f74d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 -- 172.21.15.45:0/1082253131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 msgr2=0x56308f750d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 0x56308f750d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 -- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 msgr2=0x56308f74d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f74d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc0259c640 1 -- 172.21.15.45:0/1082253131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308f74ddd0 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 0x56308f750d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.824+0000 7efc02d9d640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f74d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0259c640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efbf40bbb70 tx=0x7efbf40ed520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff54ac3622a5b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4049040 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7efbf40c45a0 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc004020 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efbf40a5020 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56308f7bf6c0 con 0x56308f739950 2024-04-25T00:23:38.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc004020 con 0x56308f739950 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efbf4031bd0 con 0x56308f739950 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 20 client.16116 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f748620 0x56308f7c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 4 client.16116 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.825+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56308f6a9f40 con 0x56308f748620 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f748620 0x56308f7c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efbe9ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efbfbfff640 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f748620 0x56308f7c1cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac132e918e9a4e0 server_cookie=e8082ae2c0520d1d in_seq=0 out_seq=0 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efbe9ffb640 1 -- 172.21.15.45:0/1082253131 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56308f6a9f40 con 0x56308f748620 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efbe9ffb640 10 client.16116 handle_command_reply: tid=0 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 client.16116 shutdown 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 msgr2=0x56308f750810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efbf40bbb70 tx=0x7efbf40ed520 comp rx=0 tx=0).stop 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 shutdown_connections 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308f73df50 0x56308f74d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56308f73d630 0x56308f750d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56308f748620 0x56308f7c1cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.826+0000 7efc0b5da740 1 --2- 172.21.15.45:0/1082253131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308f739950 0x56308f750810 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 >> 172.21.15.45:0/1082253131 conn(0x56308f7442a0 msgr2=0x56308f743b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efbe9ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 shutdown_connections 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc0b5da740 1 -- 172.21.15.45:0/1082253131 wait complete. 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc0b5da740 20 client.16116 trim_cache size 0 max 16384 2024-04-25T00:23:38.835 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:38.836 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:23:38.836 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:23:38.836 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:38.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 msgr2=0x7efbf00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 0x7efbf00a51c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc040ff9e0 tx=0x7efbec00b390 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 msgr2=0x7efc04198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.827+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009230 tx=0x7efbf4004270 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 shutdown_connections 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efbf00a2d00 0x7efbf00a51c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc041077a0 0x7efc0419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102150 0x7efc04198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 --2- 172.21.15.45:0/1033813758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc04101790 0x7efc04198100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 >> 172.21.15.45:0/1033813758 conn(0x7efc040fd2a0 msgr2=0x7efc040fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 shutdown_connections 2024-04-25T00:23:38.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.828+0000 7efc088cd640 1 -- 172.21.15.45:0/1033813758 wait complete. 2024-04-25T00:23:38.840 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:23:38.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:23:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 -- 172.21.15.45:0/788739857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 msgr2=0x7f480c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 --2- 172.21.15.45:0/788739857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c107b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009930 tx=0x7f47fc02a420 comp rx=0 tx=0).stop 2024-04-25T00:23:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 -- 172.21.15.45:0/788739857 shutdown_connections 2024-04-25T00:23:38.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 --2- 172.21.15.45:0/788739857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 0x7f480c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 --2- 172.21.15.45:0/788739857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 0x7f480c069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 --2- 172.21.15.45:0/788739857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c107b10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 -- 172.21.15.45:0/788739857 >> 172.21.15.45:0/788739857 conn(0x7f480c076710 msgr2=0x7f480c076b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 -- 172.21.15.45:0/788739857 shutdown_connections 2024-04-25T00:23:38.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.947+0000 7f4812a00640 1 -- 172.21.15.45:0/788739857 wait complete. 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 Processor -- start 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 -- start start 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 0x7f480c100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 0x7f480c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111b70 con 0x7f480c107730 2024-04-25T00:23:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1119f0 con 0x7f480c069b10 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4812a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111cf0 con 0x7f480c06a4b0 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4810f9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4810f9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34368/0 (socket says 172.21.15.45:34368) 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.948+0000 7f4810f9d640 1 -- 172.21.15.45:0/2620995658 learned_addr learned my addr 172.21.15.45:0/2620995658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f480bfff640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 0x7f480c100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4803fff640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 0x7f480c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 -- 172.21.15.45:0/2620995658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 msgr2=0x7f480c100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 0x7f480c100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 -- 172.21.15.45:0/2620995658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 msgr2=0x7f480c100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 0x7f480c100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 -- 172.21.15.45:0/2620995658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1a6590 con 0x7f480c107730 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4810f9d640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f480400eaa0 tx=0x7f480400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480400cdb0 con 0x7f480c107730 2024-04-25T00:23:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4804005020 con 0x7f480c107730 2024-04-25T00:23:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1a6880 con 0x7f480c107730 2024-04-25T00:23:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040116b0 con 0x7f480c107730 2024-04-25T00:23:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.949+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1a6dd0 con 0x7f480c107730 2024-04-25T00:23:38.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.950+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c067cd0 con 0x7f480c107730 2024-04-25T00:23:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.952+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4804005560 con 0x7f480c107730 2024-04-25T00:23:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.953+0000 7f4809ffb640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 0x7f47f80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.953+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f48040cfc50 con 0x7f480c107730 2024-04-25T00:23:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.953+0000 7f480bfff640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 0x7f47f80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.954+0000 7f480bfff640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 0x7f47f80a51c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009900 tx=0x7f47fc0058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:38.956+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4804016030 con 0x7f480c107730 2024-04-25T00:23:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.184+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f480c02d170 con 0x7f480c107730 2024-04-25T00:23:39.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.185+0000 7f4809ffb640 1 -- 172.21.15.45:0/2620995658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f480409bec0 con 0x7f480c107730 2024-04-25T00:23:39.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:23:39.186 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:23:39.187 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.187+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 msgr2=0x7f47f80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.187+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 0x7f47f80a51c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009900 tx=0x7f47fc0058d0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.187+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 msgr2=0x7f480c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.187+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f480400eaa0 tx=0x7f480400ef60 comp rx=0 tx=0).stop 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 shutdown_connections 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f47f80a2d00 0x7f47f80a51c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f480c107730 0x7f480c1011f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f480c06a4b0 0x7f480c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 --2- 172.21.15.45:0/2620995658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c069b10 0x7f480c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 >> 172.21.15.45:0/2620995658 conn(0x7f480c076710 msgr2=0x7f480c0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 shutdown_connections 2024-04-25T00:23:39.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.188+0000 7f4812a00640 1 -- 172.21.15.45:0/2620995658 wait complete. 2024-04-25T00:23:39.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:23:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:39 smithi122 ceph-mon[32959]: pgmap v53: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 -- 172.21.15.45:0/2703518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 msgr2=0x7f84dc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 --2- 172.21.15.45:0/2703518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc100aa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009930 tx=0x7f84d002a420 comp rx=0 tx=0).stop 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 -- 172.21.15.45:0/2703518944 shutdown_connections 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 --2- 172.21.15.45:0/2703518944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 0x7f84dc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 --2- 172.21.15.45:0/2703518944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc100aa0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 --2- 172.21.15.45:0/2703518944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 0x7f84dc106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 -- 172.21.15.45:0/2703518944 >> 172.21.15.45:0/2703518944 conn(0x7f84dc0769c0 msgr2=0x7f84dc076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 -- 172.21.15.45:0/2703518944 shutdown_connections 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.311+0000 7f84e2a88640 1 -- 172.21.15.45:0/2703518944 wait complete. 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 Processor -- start 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 -- start start 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 0x7f84dc10e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 0x7f84dc10ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc111b40 con 0x7f84dc100620 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1119c0 con 0x7f84dc100fe0 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e2a88640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc111cc0 con 0x7f84dc106630 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34390/0 (socket says 172.21.15.45:34390) 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 -- 172.21.15.45:0/3070552544 learned_addr learned my addr 172.21.15.45:0/3070552544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84dbfff640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 0x7f84dc10e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e1025640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 0x7f84dc10ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 -- 172.21.15.45:0/3070552544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 msgr2=0x7f84dc10ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 0x7f84dc10ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 -- 172.21.15.45:0/3070552544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 msgr2=0x7f84dc10e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 0x7f84dc10e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.312+0000 7f84e0824640 1 -- 172.21.15.45:0/3070552544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc1a6520 con 0x7f84dc100620 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84e0824640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9f0 tx=0x7f84cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84d9ffb640 1 -- 172.21.15.45:0/3070552544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc00cd80 con 0x7f84dc100620 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84d9ffb640 1 -- 172.21.15.45:0/3070552544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84cc010910 con 0x7f84dc100620 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc1a6810 con 0x7f84dc100620 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84d9ffb640 1 -- 172.21.15.45:0/3070552544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc018990 con 0x7f84dc100620 2024-04-25T00:23:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.313+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a6d30 con 0x7f84dc100620 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.315+0000 7f84d9ffb640 1 -- 172.21.15.45:0/3070552544 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84cc018b30 con 0x7f84dc100620 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 Processor -- start 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 -- start start 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84d9ffb640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 0x7f84c40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x56125243e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5612523a3670 con 0x56125243eea0 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56125234b970 con 0x561252449570 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e5793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56125239f630 con 0x56125243e580 2024-04-25T00:23:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84d9ffb640 1 -- 172.21.15.45:0/3070552544 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f84cc0cfa50 con 0x7f84dc100620 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e1025640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 0x7f84c40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34402/0 (socket says 172.21.15.45:34402) 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 -- 172.21.15.45:0/2115397139 learned_addr learned my addr 172.21.15.45:0/2115397139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e0824640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x56125243e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e1025640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 -- 172.21.15.45:0/2115397139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 msgr2=0x56125243e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x56125243e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 -- 172.21.15.45:0/2115397139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 msgr2=0x56125244b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84dbfff640 1 -- 172.21.15.45:0/2115397139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125236f400 con 0x56125243eea0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.316+0000 7f84e0824640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x56125243e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.317+0000 7f84e1025640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 0x7f84c40a51f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005fd0 tx=0x7f84d40149d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.317+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f84d002a940 tx=0x7f84d0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8250b444f795bc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.317+0000 7f84c2ffd640 1 -- 172.21.15.45:0/2115397139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0040070 con 0x56125243eea0 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.317+0000 7f84c2ffd640 1 -- 172.21.15.45:0/2115397139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d0031870 con 0x56125243eea0 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.317+0000 7f84c2ffd640 1 -- 172.21.15.45:0/2115397139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0035040 con 0x56125243eea0 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- 172.21.15.45:0/2115397139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 msgr2=0x561252449030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f84d002a940 tx=0x7f84d0005c00 comp rx=0 tx=0).stop 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- 172.21.15.45:0/2115397139 shutdown_connections 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x561252449030 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 --2- 172.21.15.45:0/2115397139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x56125243e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- 172.21.15.45:0/2115397139 >> 172.21.15.45:0/2115397139 conn(0x56125243a8a0 msgr2=0x56125243ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- 172.21.15.45:0/2115397139 shutdown_connections 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- 172.21.15.45:0/2115397139 wait complete. 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 -- start start 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.318+0000 7f84e5793740 1 Processor -- start 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243e580 0x561252448ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 0x561252446730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56125244c300 con 0x56125243eea0 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56125235f050 con 0x56125243e580 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56125244c480 con 0x561252449570 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e5793740 10 client.0 _ll_register_callbacks cb 0x5612520b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34406/0 (socket says 172.21.15.45:34406) 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 -- 172.21.15.45:0/1963222945 learned_addr learned my addr 172.21.15.45:0/1963222945 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e0824640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243e580 0x561252448ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e1025640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 0x561252446730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 -- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 msgr2=0x561252446730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 0x561252446730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 -- 172.21.15.45:0/1963222945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243e580 msgr2=0x561252448ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243e580 0x561252448ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 -- 172.21.15.45:0/1963222945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561252330dc0 con 0x56125243eea0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84e1025640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 0x561252446730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84dbfff640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccce37933a0fac0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.319+0000 7f84bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0040030 con 0x56125243eea0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f84d0004260 con 0x56125243eea0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d0031870 con 0x56125243eea0 2024-04-25T00:23:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84d0056050 con 0x56125243eea0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5612520b73f0 con 0x56125243eea0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d0031870 con 0x56125243eea0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84d00038d0 con 0x56125243eea0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 10 client.16134 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 20 client.16134 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524593f0 0x56125245b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 4 client.16134 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.320+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612523b0bb0 con 0x5612524593f0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.321+0000 7f84e0824640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524593f0 0x56125245b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.321+0000 7f84bbfff640 10 client.16134 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.321+0000 7f84e0824640 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524593f0 0x56125245b7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c735d4f1218a3d7e server_cookie=5bb22305a96b9664 in_seq=0 out_seq=0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84bbfff640 1 -- 172.21.15.45:0/1963222945 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612523b0bb0 con 0x5612524593f0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84bbfff640 10 client.16134 handle_command_reply: tid=0 2024-04-25T00:23:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 client.16134 shutdown 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 msgr2=0x5612524461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d0005250 comp rx=0 tx=0).stop 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 shutdown_connections 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524593f0 0x56125245b7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561252449570 0x561252446730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x5612524461d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 --2- 172.21.15.45:0/1963222945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243e580 0x561252448ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84bbfff640 0 client.16134 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 >> 172.21.15.45:0/1963222945 conn(0x56125243a8a0 msgr2=0x5612524406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 shutdown_connections 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 1 -- 172.21.15.45:0/1963222945 wait complete. 2024-04-25T00:23:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.322+0000 7f84e5793740 20 client.16134 trim_cache size 0 max 16384 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 Processor -- start 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 -- start start 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x5612524551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243eea0 0x561252448a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1b0ce0 con 0x561252449570 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1b2820 con 0x56125243eea0 2024-04-25T00:23:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e5793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1b13b0 con 0x56125243e580 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e1025640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e1025640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34420/0 (socket says 172.21.15.45:34420) 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e0824640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x5612524551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84e1025640 1 -- 172.21.15.45:0/2928684659 learned_addr learned my addr 172.21.15.45:0/2928684659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.336+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243eea0 0x561252448a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 -- 172.21.15.45:0/2928684659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 msgr2=0x5612524551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x5612524551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 -- 172.21.15.45:0/2928684659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243eea0 msgr2=0x561252448a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243eea0 0x561252448a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 -- 172.21.15.45:0/2928684659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612521817d0 con 0x561252449570 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e0824640 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x5612524551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84e1025640 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f84d40126b0 tx=0x7f84d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa19ed436feadbaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84c27fc640 1 -- 172.21.15.45:0/2928684659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4003b70 con 0x561252449570 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84c27fc640 1 -- 172.21.15.45:0/2928684659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4003d10 con 0x561252449570 2024-04-25T00:23:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.337+0000 7f84c27fc640 1 -- 172.21.15.45:0/2928684659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4020b10 con 0x561252449570 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- 172.21.15.45:0/2928684659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 msgr2=0x561252460de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f84d40126b0 tx=0x7f84d4017000 comp rx=0 tx=0).stop 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- 172.21.15.45:0/2928684659 shutdown_connections 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561252449570 0x561252460de0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56125243eea0 0x561252448a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 --2- 172.21.15.45:0/2928684659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x5612524551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- 172.21.15.45:0/2928684659 >> 172.21.15.45:0/2928684659 conn(0x56125243a8a0 msgr2=0x561252441860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- 172.21.15.45:0/2928684659 shutdown_connections 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- 172.21.15.45:0/2928684659 wait complete. 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 -- start start 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.338+0000 7f84e5793740 1 Processor -- start 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x561252451a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1b2150 con 0x56125243eea0 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1b1a80 con 0x561252449570 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1aecc0 con 0x56125243e580 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e5793740 10 client.0 _ll_register_callbacks cb 0x7f84d0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34436/0 (socket says 172.21.15.45:34436) 2024-04-25T00:23:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 -- 172.21.15.45:0/2800495099 learned_addr learned my addr 172.21.15.45:0/2800495099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e1025640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84e0824640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x561252451a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 -- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 msgr2=0x561252451a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x561252451a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 -- 172.21.15.45:0/2800495099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 msgr2=0x56125244d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 -- 172.21.15.45:0/2800495099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125244dbc0 con 0x56125243eea0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84dbfff640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f84d0003e90 tx=0x7f84d0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16951d3ed5120a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.339+0000 7f84bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0040070 con 0x56125243eea0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f84d0004460 con 0x56125243eea0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d4003d10 con 0x56125243eea0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84d0068050 con 0x56125243eea0 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5612524bf620 con 0x56125243eea0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d4003d10 con 0x56125243eea0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84d0004890 con 0x56125243eea0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 10 client.16140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 20 client.16140 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524bfdc0 0x5612524c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 4 client.16140 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.340+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612523aaf00 con 0x5612524bfdc0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e0824640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524bfdc0 0x5612524c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e0824640 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524bfdc0 0x5612524c21e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2df24852c30bd1c6 server_cookie=398a408de100115e in_seq=0 out_seq=0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84bbfff640 10 client.16140 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84bbfff640 1 -- 172.21.15.45:0/2800495099 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612523aaf00 con 0x5612524bfdc0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84bbfff640 10 client.16140 handle_command_reply: tid=0 2024-04-25T00:23:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 client.16140 shutdown 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 msgr2=0x56125244ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f84d0003e90 tx=0x7f84d0005d50 comp rx=0 tx=0).stop 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 shutdown_connections 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5612524bfdc0 0x5612524c21e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561252449570 0x56125244d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56125243eea0 0x56125244ce50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84bbfff640 0 client.16140 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 --2- 172.21.15.45:0/2800495099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56125243e580 0x561252451a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.341+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 >> 172.21.15.45:0/2800495099 conn(0x56125243a8a0 msgr2=0x561252440f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 shutdown_connections 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e5793740 1 -- 172.21.15.45:0/2800495099 wait complete. 2024-04-25T00:23:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e5793740 20 client.16140 trim_cache size 0 max 16384 2024-04-25T00:23:39.349 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 msgr2=0x7f84c40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 0x7f84c40a51f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005fd0 tx=0x7f84d40149d0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 msgr2=0x7f84dc10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9f0 tx=0x7f84cc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 shutdown_connections 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f84c40a2d30 0x7f84c40a51f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc106630 0x7f84dc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84dc100fe0 0x7f84dc10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 --2- 172.21.15.45:0/3070552544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84dc100620 0x7f84dc10df80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 >> 172.21.15.45:0/3070552544 conn(0x7f84dc0769c0 msgr2=0x7f84dc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 shutdown_connections 2024-04-25T00:23:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.342+0000 7f84e2a88640 1 -- 172.21.15.45:0/3070552544 wait complete. 2024-04-25T00:23:39.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:39 smithi045 ceph-mon[29194]: pgmap v53: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:39.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:23:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:39 smithi071 ceph-mon[32667]: pgmap v53: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 MiB/s wr, 16 op/s 2024-04-25T00:23:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 -- 172.21.15.45:0/2146568825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c06a4d0 msgr2=0x7f899c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 --2- 172.21.15.45:0/2146568825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c06a4d0 0x7f899c10e5f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8994009cd0 tx=0x7f899402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 -- 172.21.15.45:0/2146568825 shutdown_connections 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 --2- 172.21.15.45:0/2146568825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c06a4d0 0x7f899c10e5f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 --2- 172.21.15.45:0/2146568825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c069b50 0x7f899c069f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 --2- 172.21.15.45:0/2146568825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 0x7f899c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 -- 172.21.15.45:0/2146568825 >> 172.21.15.45:0/2146568825 conn(0x7f899c0776e0 msgr2=0x7f899c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 -- 172.21.15.45:0/2146568825 shutdown_connections 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.463+0000 7f89a2810640 1 -- 172.21.15.45:0/2146568825 wait complete. 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 Processor -- start 2024-04-25T00:23:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 -- start start 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 0x7f899c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 0x7f899c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f899c111b30 con 0x7f899c069b50 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1119b0 con 0x7f899c06a4d0 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a2810640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f899c111cb0 con 0x7f899c107730 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34446/0 (socket says 172.21.15.45:34446) 2024-04-25T00:23:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f899bfff640 1 -- 172.21.15.45:0/2301856965 learned_addr learned my addr 172.21.15.45:0/2301856965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.464+0000 7f89a0dad640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 0x7f899c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899b7fe640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 0x7f899c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 -- 172.21.15.45:0/2301856965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 msgr2=0x7f899c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 0x7f899c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 -- 172.21.15.45:0/2301856965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 msgr2=0x7f899c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 0x7f899c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 -- 172.21.15.45:0/2301856965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c19cfe0 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899b7fe640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 0x7f899c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f899bfff640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f899000e9f0 tx=0x7f899000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f89997fa640 1 -- 172.21.15.45:0/2301856965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899000cd80 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c19d2d0 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f89997fa640 1 -- 172.21.15.45:0/2301856965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990005500 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f89997fa640 1 -- 172.21.15.45:0/2301856965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990010690 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.465+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a4b60 con 0x7f899c069b50 2024-04-25T00:23:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.467+0000 7f89997fa640 1 -- 172.21.15.45:0/2301856965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8990010830 con 0x7f899c069b50 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 Processor -- start 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 -- start start 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89997fa640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 0x7f89840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 0x55ceaac54030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ceaabae670 con 0x55ceaac49580 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ceaab56970 con 0x55ceaac49ea0 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a55f5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ceaabaa630 con 0x55ceaac54570 2024-04-25T00:23:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a0dad640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 0x7f89840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89997fa640 1 -- 172.21.15.45:0/2301856965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f89900cfc20 con 0x7f899c069b50 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34448/0 (socket says 172.21.15.45:34448) 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f899bfff640 1 -- 172.21.15.45:0/1732061255 learned_addr learned my addr 172.21.15.45:0/1732061255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f89a0dad640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.468+0000 7f899b7fe640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 0x55ceaac54030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 -- 172.21.15.45:0/1732061255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 msgr2=0x55ceaac56960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac56960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 -- 172.21.15.45:0/1732061255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 msgr2=0x55ceaac54030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 0x55ceaac54030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 -- 172.21.15.45:0/1732061255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaab7a400 con 0x55ceaac49580 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899b7fe640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 0x55ceaac54030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f89a0dad640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 0x7f89840a51c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8994004ab0 tx=0x7f89940316c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f899bfff640 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89900a44c0 tx=0x7f8990098150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e58b1f588b5686d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f89827fc640 1 -- 172.21.15.45:0/1732061255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899009bd80 con 0x55ceaac49580 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f89827fc640 1 -- 172.21.15.45:0/1732061255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89900a0130 con 0x55ceaac49580 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.469+0000 7f89827fc640 1 -- 172.21.15.45:0/1732061255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900cb180 con 0x55ceaac49580 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- 172.21.15.45:0/1732061255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 msgr2=0x55ceaac49960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89900a44c0 tx=0x7f8990098150 comp rx=0 tx=0).stop 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- 172.21.15.45:0/1732061255 shutdown_connections 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac56960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49ea0 0x55ceaac54030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 --2- 172.21.15.45:0/1732061255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac49960 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- 172.21.15.45:0/1732061255 >> 172.21.15.45:0/1732061255 conn(0x55ceaac458a0 msgr2=0x55ceaac45c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- 172.21.15.45:0/1732061255 shutdown_connections 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- 172.21.15.45:0/1732061255 wait complete. 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.470+0000 7f89a55f5740 1 -- start start 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 Processor -- start 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ceaac5b770 con 0x55ceaac49ea0 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ceaab6a050 con 0x55ceaac49580 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ceaac5b8f0 con 0x55ceaac54570 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f89a55f5740 10 client.0 _ll_register_callbacks cb 0x55ceaa8c1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f899bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34462/0 (socket says 172.21.15.45:34462) 2024-04-25T00:23:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f899bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:47432/0 (socket says 172.21.15.45:47432) 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.471+0000 7f899b7fe640 1 -- 172.21.15.45:0/4283296081 learned_addr learned my addr 172.21.15.45:0/4283296081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f89a0dad640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 -- 172.21.15.45:0/4283296081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 msgr2=0x55ceaac51840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac51840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 -- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 msgr2=0x55ceaac53f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 -- 172.21.15.45:0/4283296081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaab3bdc0 con 0x55ceaac49ea0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899bfff640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f899b7fe640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f898c013790 tx=0x7f898c013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcb6c976961abce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f897b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c020060 con 0x55ceaac49ea0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f898c011dd0 con 0x55ceaac49ea0 2024-04-25T00:23:39.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89900a0130 con 0x55ceaac49ea0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f898c027050 con 0x55ceaac49ea0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f89a55f5740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceaa8c23f0 con 0x55ceaac49ea0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.472+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89900a0130 con 0x55ceaac49ea0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f898c01a030 con 0x55ceaac49ea0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 10 client.16152 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 20 client.16152 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaac68a90 0x55ceaac6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 4 client.16152 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceaabbbbb0 con 0x55ceaac68a90 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f899bfff640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaac68a90 0x55ceaac6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f897b7fe640 10 client.16152 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:39.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.473+0000 7f899bfff640 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaac68a90 0x55ceaac6aec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbcc655580f36cc0 server_cookie=1b6022e01ba35421 in_seq=0 out_seq=0 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f897b7fe640 1 -- 172.21.15.45:0/4283296081 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ceaabbbbb0 con 0x55ceaac68a90 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f897b7fe640 10 client.16152 handle_command_reply: tid=0 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 client.16152 shutdown 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 msgr2=0x55ceaac51230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f898c013790 tx=0x7f898c013c50 comp rx=0 tx=0).stop 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 shutdown_connections 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaac68a90 0x55ceaac6aec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac54570 0x55ceaac51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac51230 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f897b7fe640 0 client.16152 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.474+0000 7f89a55f5740 1 --2- 172.21.15.45:0/4283296081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac49580 0x55ceaac53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.475+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 >> 172.21.15.45:0/4283296081 conn(0x55ceaac458a0 msgr2=0x55ceaac44b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.475+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 shutdown_connections 2024-04-25T00:23:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.475+0000 7f89a55f5740 1 -- 172.21.15.45:0/4283296081 wait complete. 2024-04-25T00:23:39.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.475+0000 7f89a55f5740 20 client.16152 trim_cache size 0 max 16384 2024-04-25T00:23:39.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.488+0000 7f89a55f5740 1 Processor -- start 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.488+0000 7f89a55f5740 1 -- start start 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49580 0x55ceaac4b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaac70790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1ac910 con 0x55ceaac49ea0 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1ae450 con 0x55ceaac54570 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a55f5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1acfe0 con 0x55ceaac49580 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34478/0 (socket says 172.21.15.45:34478) 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 -- 172.21.15.45:0/2161760792 learned_addr learned my addr 172.21.15.45:0/2161760792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899bfff640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49580 0x55ceaac4b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a0dad640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaac70790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 -- 172.21.15.45:0/2161760792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49580 msgr2=0x55ceaac4b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49580 0x55ceaac4b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 -- 172.21.15.45:0/2161760792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 msgr2=0x55ceaac70790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaac70790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 -- 172.21.15.45:0/2161760792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaa98c7d0 con 0x55ceaac49ea0 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f89a0dad640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaac70790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f899b7fe640 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e6e0 tx=0x7f898c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858c2e3c563065df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.489+0000 7f8981ffb640 1 -- 172.21.15.45:0/2161760792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c000e60 con 0x55ceaac49ea0 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.490+0000 7f8981ffb640 1 -- 172.21.15.45:0/2161760792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c0055f0 con 0x55ceaac49ea0 2024-04-25T00:23:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.490+0000 7f8981ffb640 1 -- 172.21.15.45:0/2161760792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0035f0 con 0x55ceaac49ea0 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- 172.21.15.45:0/2161760792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 msgr2=0x55ceaac53a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e6e0 tx=0x7f898c009e20 comp rx=0 tx=0).stop 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- 172.21.15.45:0/2161760792 shutdown_connections 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaac70790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49ea0 0x55ceaac53a70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 --2- 172.21.15.45:0/2161760792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49580 0x55ceaac4b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- 172.21.15.45:0/2161760792 >> 172.21.15.45:0/2161760792 conn(0x55ceaac458a0 msgr2=0x55ceaac4c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- 172.21.15.45:0/2161760792 shutdown_connections 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- 172.21.15.45:0/2161760792 wait complete. 2024-04-25T00:23:39.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 -- start start 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.491+0000 7f89a55f5740 1 Processor -- start 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49ea0 0x55ceaac5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaacce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1add80 con 0x55ceaac49580 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1ad6b0 con 0x55ceaac54570 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1aa8f0 con 0x55ceaac49ea0 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a55f5740 10 client.0 _ll_register_callbacks cb 0x7f898c009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34490/0 (socket says 172.21.15.45:34490) 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 -- 172.21.15.45:0/888350374 learned_addr learned my addr 172.21.15.45:0/888350374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f89a0dad640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaacce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899b7fe640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49ea0 0x55ceaac5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 -- 172.21.15.45:0/888350374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49ea0 msgr2=0x55ceaac5cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49ea0 0x55ceaac5cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 -- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 msgr2=0x55ceaacce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaacce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 -- 172.21.15.45:0/888350374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaacce540 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f899bfff640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89900049d0 tx=0x7f899009c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c721be7154383c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.492+0000 7f897b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900a49f0 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f89900ba670 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898c0055f0 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f89900a0130 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 10 client.16158 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceaaccf700 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c0055f0 con 0x55ceaac49580 2024-04-25T00:23:39.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f89900babe0 con 0x55ceaac49580 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 10 client.16158 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 20 client.16158 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaaccfaa0 0x55ceaacd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 4 client.16158 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.493+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ceaabb5f00 con 0x55ceaaccfaa0 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f899b7fe640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaaccfaa0 0x55ceaacd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f899b7fe640 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaaccfaa0 0x55ceaacd1e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e733233d7fd46791 server_cookie=2f731f4a204a9d5c in_seq=0 out_seq=0 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f897b7fe640 10 client.16158 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f897b7fe640 1 -- 172.21.15.45:0/888350374 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ceaabb5f00 con 0x55ceaaccfaa0 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f897b7fe640 10 client.16158 handle_command_reply: tid=0 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 client.16158 shutdown 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 msgr2=0x55ceaac5c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89900049d0 tx=0x7f899009c480 comp rx=0 tx=0).stop 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 shutdown_connections 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55ceaaccfaa0 0x55ceaacd1e90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ceaac54570 0x55ceaacce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ceaac49ea0 0x55ceaac5cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 --2- 172.21.15.45:0/888350374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ceaac49580 0x55ceaac5c980 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f897b7fe640 0 client.16158 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.494+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 >> 172.21.15.45:0/888350374 conn(0x55ceaac458a0 msgr2=0x55ceaac442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 shutdown_connections 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a55f5740 1 -- 172.21.15.45:0/888350374 wait complete. 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a55f5740 20 client.16158 trim_cache size 0 max 16384 2024-04-25T00:23:39.501 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 msgr2=0x7f89840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 0x7f89840a51c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8994004ab0 tx=0x7f89940316c0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 msgr2=0x7f899c197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f899000e9f0 tx=0x7f899000edf0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 shutdown_connections 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f89840a2d00 0x7f89840a51c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f899c107730 0x7f899c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f899c06a4d0 0x7f899c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 --2- 172.21.15.45:0/2301856965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f899c069b50 0x7f899c197ff0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 >> 172.21.15.45:0/2301856965 conn(0x7f899c0776e0 msgr2=0x7f899c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 shutdown_connections 2024-04-25T00:23:39.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.495+0000 7f89a2810640 1 -- 172.21.15.45:0/2301856965 wait complete. 2024-04-25T00:23:39.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/996302435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 msgr2=0x7fbc1410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/996302435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1410e1a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009ca0 tx=0x7fbc0c02a9e0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/996302435 shutdown_connections 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/996302435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1410e1a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/996302435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141032e0 0x7fbc1410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/996302435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141029c0 0x7fbc14102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/996302435 >> 172.21.15.45:0/996302435 conn(0x7fbc14077740 msgr2=0x7fbc14077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/996302435 shutdown_connections 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.615+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/996302435 wait complete. 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 Processor -- start 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 -- start start 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141029c0 0x7fbc1419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141032e0 0x7fbc1419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14111b00 con 0x7fbc1410bd00 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14111980 con 0x7fbc141029c0 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1c7a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc14111c80 con 0x7fbc141032e0 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1ad3f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1ad3f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34510/0 (socket says 172.21.15.45:34510) 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1ad3f640 1 -- 172.21.15.45:0/2341213751 learned_addr learned my addr 172.21.15.45:0/2341213751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141032e0 0x7fbc1419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.616+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141029c0 0x7fbc1419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 -- 172.21.15.45:0/2341213751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141032e0 msgr2=0x7fbc1419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141032e0 0x7fbc1419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 -- 172.21.15.45:0/2341213751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141029c0 msgr2=0x7fbc1419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141029c0 0x7fbc1419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 -- 172.21.15.45:0/2341213751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1419f1a0 con 0x7fbc1410bd00 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1ad3f640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc0b7fe640 1 -- 172.21.15.45:0/2341213751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c047070 con 0x7fbc1410bd00 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1419f430 con 0x7fbc1410bd00 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc0b7fe640 1 -- 172.21.15.45:0/2341213751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc0c003e50 con 0x7fbc1410bd00 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc141a6d20 con 0x7fbc1410bd00 2024-04-25T00:23:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.617+0000 7fbc0b7fe640 1 -- 172.21.15.45:0/2341213751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c051060 con 0x7fbc1410bd00 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc0b7fe640 1 -- 172.21.15.45:0/2341213751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc0c04a070 con 0x7fbc1410bd00 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 Processor -- start 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 -- start start 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602580 0x55ec41602960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 0x55ec4160d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ec41567670 con 0x55ec4160d570 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ec4150f970 con 0x55ec41602ea0 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.620+0000 7fbc1f594740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec41563630 con 0x55ec41602580 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc0b7fe640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 0x7fbc000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34518/0 (socket says 172.21.15.45:34518) 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc19d3d640 1 -- 172.21.15.45:0/1430726299 learned_addr learned my addr 172.21.15.45:0/1430726299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 0x55ec4160d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1ad3f640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 0x7fbc000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602580 0x55ec41602960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc0b7fe640 1 -- 172.21.15.45:0/2341213751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc0c0f0350 con 0x7fbc1410bd00 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 -- 172.21.15.45:0/1430726299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602580 msgr2=0x55ec41602960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602580 0x55ec41602960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 -- 172.21.15.45:0/1430726299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 msgr2=0x55ec4160f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 -- 172.21.15.45:0/1430726299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec41533400 con 0x55ec41602ea0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1ad3f640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 0x7fbc000a51c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0ed7f0 tx=0x7fbc0c08c420 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 0x55ec4160d030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc10004630 tx=0x7fbc1000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc17decd55ed0227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.621+0000 7fbc08ff9640 1 -- 172.21.15.45:0/1430726299 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10003bd0 con 0x55ec41602ea0 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc08ff9640 1 -- 172.21.15.45:0/1430726299 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc10015070 con 0x55ec41602ea0 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc08ff9640 1 -- 172.21.15.45:0/1430726299 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10004d20 con 0x55ec41602ea0 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 -- 172.21.15.45:0/1430726299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 msgr2=0x55ec4160d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 0x55ec4160d030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc10004630 tx=0x7fbc1000bb20 comp rx=0 tx=0).stop 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 -- 172.21.15.45:0/1430726299 shutdown_connections 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602ea0 0x55ec4160d030 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1430726299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602580 0x55ec41602960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 -- 172.21.15.45:0/1430726299 >> 172.21.15.45:0/1430726299 conn(0x55ec415fe8a0 msgr2=0x55ec415fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 -- 172.21.15.45:0/1430726299 shutdown_connections 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.622+0000 7fbc1f594740 1 -- 172.21.15.45:0/1430726299 wait complete. 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 -- start start 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 Processor -- start 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec4160ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec4160a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ec41610360 con 0x55ec4160d570 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ec41523050 con 0x55ec41602580 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec416104e0 con 0x55ec41602ea0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1f594740 10 client.0 _ll_register_callbacks cb 0x55ec4127abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34522/0 (socket says 172.21.15.45:34522) 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc19d3d640 1 -- 172.21.15.45:0/1754803179 learned_addr learned my addr 172.21.15.45:0/1754803179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec4160ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.623+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec4160a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 -- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 msgr2=0x55ec4160a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec4160a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 -- 172.21.15.45:0/1754803179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 msgr2=0x55ec4160ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec4160ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 -- 172.21.15.45:0/1754803179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec414f4dc0 con 0x55ec4160d570 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbc04013d80 tx=0x7fbc04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589f424f9cf93afb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04013230 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc04012480 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc10015070 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc0402b050 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc1f594740 10 client.16170 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec4127b3f0 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.624+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc10015070 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc04024060 con 0x55ec4160d570 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 20 client.16170 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec4161d650 0x55ec4161fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 4 client.16170 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec41574bb0 con 0x55ec4161d650 2024-04-25T00:23:39.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec4161d650 0x55ec4161fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec4161d650 0x55ec4161fab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b585577bb452853b server_cookie=6a0e7a5f10cfb17e in_seq=0 out_seq=0 2024-04-25T00:23:39.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.625+0000 7fbbfdffb640 10 client.16170 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbbfdffb640 1 -- 172.21.15.45:0/1754803179 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec41574bb0 con 0x55ec4161d650 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbbfdffb640 10 client.16170 handle_command_reply: tid=0 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 client.16170 shutdown 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 msgr2=0x55ec4160a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbc04013d80 tx=0x7fbc04004730 comp rx=0 tx=0).stop 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 shutdown_connections 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec4161d650 0x55ec4161fab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec4160d570 0x55ec4160a770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec4160a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbbfdffb640 0 client.16170 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 --2- 172.21.15.45:0/1754803179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec4160ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.626+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 >> 172.21.15.45:0/1754803179 conn(0x55ec415fe8a0 msgr2=0x55ec416046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.627+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 shutdown_connections 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.627+0000 7fbc1f594740 1 -- 172.21.15.45:0/1754803179 wait complete. 2024-04-25T00:23:39.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.627+0000 7fbc1f594740 20 client.16170 trim_cache size 0 max 16384 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 Processor -- start 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 -- start start 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec41619250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 0x55ec416055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141b0ca0 con 0x55ec41602ea0 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141b27e0 con 0x55ec41602580 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.640+0000 7fbc1f594740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141b1370 con 0x55ec4160d570 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34532/0 (socket says 172.21.15.45:34532) 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 -- 172.21.15.45:0/4034454037 learned_addr learned my addr 172.21.15.45:0/4034454037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec41619250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 0x55ec416055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 -- 172.21.15.45:0/4034454037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 msgr2=0x55ec416055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 0x55ec416055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 -- 172.21.15.45:0/4034454037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 msgr2=0x55ec41619250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec41619250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 -- 172.21.15.45:0/4034454037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec413457d0 con 0x55ec41602ea0 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 0x55ec416055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbc10004580 tx=0x7fbc10005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916e3fa27470ed31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbbfffff640 1 -- 172.21.15.45:0/4034454037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10030070 con 0x55ec41602ea0 2024-04-25T00:23:39.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbbfffff640 1 -- 172.21.15.45:0/4034454037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc10015040 con 0x55ec41602ea0 2024-04-25T00:23:39.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.641+0000 7fbbfffff640 1 -- 172.21.15.45:0/4034454037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100117c0 con 0x55ec41602ea0 2024-04-25T00:23:39.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 -- 172.21.15.45:0/4034454037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 msgr2=0x55ec41609ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbc10004580 tx=0x7fbc10005c70 comp rx=0 tx=0).stop 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 -- 172.21.15.45:0/4034454037 shutdown_connections 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec4160d570 0x55ec416055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602ea0 0x55ec41609ad0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 --2- 172.21.15.45:0/4034454037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec41602580 0x55ec41619250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 -- 172.21.15.45:0/4034454037 >> 172.21.15.45:0/4034454037 conn(0x55ec415fe8a0 msgr2=0x55ec41604eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 -- 172.21.15.45:0/4034454037 shutdown_connections 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.642+0000 7fbc1f594740 1 -- 172.21.15.45:0/4034454037 wait complete. 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 -- start start 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 Processor -- start 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec41615cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec4160d570 0x55ec41610cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141b2110 con 0x55ec41602580 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141b1a40 con 0x55ec4160d570 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc141aec80 con 0x55ec41602ea0 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1f594740 10 client.0 _ll_register_callbacks cb 0x7fbc04005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34542/0 (socket says 172.21.15.45:34542) 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 -- 172.21.15.45:0/2521913911 learned_addr learned my addr 172.21.15.45:0/2521913911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec41615cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec4160d570 0x55ec41610cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 -- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 msgr2=0x55ec41615cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec41615cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 -- 172.21.15.45:0/2521913911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec4160d570 msgr2=0x55ec41610cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec4160d570 0x55ec41610cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.643+0000 7fbc19d3d640 1 -- 172.21.15.45:0/2521913911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec416114c0 con 0x55ec41602580 2024-04-25T00:23:39.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec41615cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbc19d3d640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ec415ff6e0 tx=0x7fbc0400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6d60ca3026ff71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04013220 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc10015040 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc04004040 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc0403c050 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbc1f594740 10 client.16176 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec41683850 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.644+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc10015040 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc0401c070 con 0x55ec41602580 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 10 client.16176 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 20 client.16176 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec41683f40 0x55ec41686360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 4 client.16176 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec4156ef00 con 0x55ec41683f40 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec41683f40 0x55ec41686360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbc1a53e640 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec41683f40 0x55ec41686360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c4db74b13455c5 server_cookie=7c54ca55d40e46a3 in_seq=0 out_seq=0 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.645+0000 7fbbfdffb640 10 client.16176 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbbfdffb640 1 -- 172.21.15.45:0/2521913911 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec4156ef00 con 0x55ec41683f40 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbbfdffb640 10 client.16176 handle_command_reply: tid=0 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 client.16176 shutdown 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 msgr2=0x55ec41615780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ec415ff6e0 tx=0x7fbc0400be80 comp rx=0 tx=0).stop 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 shutdown_connections 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ec41683f40 0x55ec41686360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec4160d570 0x55ec41610cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec41602ea0 0x55ec41615cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbbfdffb640 0 client.16176 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 --2- 172.21.15.45:0/2521913911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ec41602580 0x55ec41615780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 >> 172.21.15.45:0/2521913911 conn(0x55ec415fe8a0 msgr2=0x55ec415fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 shutdown_connections 2024-04-25T00:23:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 1 -- 172.21.15.45:0/2521913911 wait complete. 2024-04-25T00:23:39.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1f594740 20 client.16176 trim_cache size 0 max 16384 2024-04-25T00:23:39.653 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 msgr2=0x7fbc000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 0x7fbc000a51c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0ed7f0 tx=0x7fbc0c08c420 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 msgr2=0x7fbc1419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.646+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006010 tx=0x7fbc0c003ce0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 shutdown_connections 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc000a2d00 0x7fbc000a51c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1410bd00 0x7fbc1419ea70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc141032e0 0x7fbc1419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 --2- 172.21.15.45:0/2341213751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc141029c0 0x7fbc1419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 >> 172.21.15.45:0/2341213751 conn(0x7fbc14077740 msgr2=0x7fbc14101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 shutdown_connections 2024-04-25T00:23:39.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.647+0000 7fbc1c7a2640 1 -- 172.21.15.45:0/2341213751 wait complete. 2024-04-25T00:23:39.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:23:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 -- 172.21.15.45:0/281240682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac107730 msgr2=0x7fecac107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 --2- 172.21.15.45:0/281240682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac107730 0x7fecac107b10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7feca0009930 tx=0x7feca002a420 comp rx=0 tx=0).stop 2024-04-25T00:23:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 -- 172.21.15.45:0/281240682 shutdown_connections 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 --2- 172.21.15.45:0/281240682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 0x7fecac10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 --2- 172.21.15.45:0/281240682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac101720 0x7fecac101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 --2- 172.21.15.45:0/281240682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac107730 0x7fecac107b10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 -- 172.21.15.45:0/281240682 >> 172.21.15.45:0/281240682 conn(0x7fecac078b40 msgr2=0x7fecac078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 -- 172.21.15.45:0/281240682 shutdown_connections 2024-04-25T00:23:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.767+0000 7fecb33b0640 1 -- 172.21.15.45:0/281240682 wait complete. 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.768+0000 7fecb33b0640 1 Processor -- start 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.768+0000 7fecb33b0640 1 -- start start 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 0x7fecac198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac107730 0x7fecac19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecac111b40 con 0x7fecac101720 2024-04-25T00:23:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1119c0 con 0x7fecac107730 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb33b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fecac111cc0 con 0x7fecac1020e0 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34560/0 (socket says 172.21.15.45:34560) 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 -- 172.21.15.45:0/3491272056 learned_addr learned my addr 172.21.15.45:0/3491272056 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb094b640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 0x7fecac198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb194d640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac107730 0x7fecac19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 -- 172.21.15.45:0/3491272056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 msgr2=0x7fecac198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 0x7fecac198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 -- 172.21.15.45:0/3491272056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac107730 msgr2=0x7fecac19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac107730 0x7fecac19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 -- 172.21.15.45:0/3491272056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac19d010 con 0x7fecac101720 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.769+0000 7fecb114c640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7feca0009230 tx=0x7feca0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.770+0000 7fec9e7fc640 1 -- 172.21.15.45:0/3491272056 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca003f030 con 0x7fecac101720 2024-04-25T00:23:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.770+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac19d2a0 con 0x7fecac101720 2024-04-25T00:23:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.770+0000 7fec9e7fc640 1 -- 172.21.15.45:0/3491272056 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca0003ad0 con 0x7fecac101720 2024-04-25T00:23:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.770+0000 7fec9e7fc640 1 -- 172.21.15.45:0/3491272056 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0035070 con 0x7fecac101720 2024-04-25T00:23:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.770+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a4b90 con 0x7fecac101720 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.772+0000 7fec9e7fc640 1 -- 172.21.15.45:0/3491272056 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feca0042030 con 0x7fecac101720 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.772+0000 7fecb619d740 1 Processor -- start 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.772+0000 7fecb619d740 1 -- start start 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642480 0x55f68b642860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642da0 0x55f68b651440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f68b5a7670 con 0x55f68b651980 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f68b54f970 con 0x55f68b642da0 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb619d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f68b5a3630 con 0x55f68b642480 2024-04-25T00:23:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34572/0 (socket says 172.21.15.45:34572) 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 -- 172.21.15.45:0/4064120087 learned_addr learned my addr 172.21.15.45:0/4064120087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb114c640 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642da0 0x55f68b651440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb194d640 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642480 0x55f68b642860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 -- 172.21.15.45:0/4064120087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642480 msgr2=0x55f68b642860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642480 0x55f68b642860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 -- 172.21.15.45:0/4064120087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642da0 msgr2=0x55f68b651440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642da0 0x55f68b651440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 -- 172.21.15.45:0/4064120087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68b573400 con 0x55f68b651980 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fec9e7fc640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 0x7fec980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb094b640 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feca8009c90 tx=0x7feca8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b947b18f3ebcb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.773+0000 7fecb194d640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 0x7fec980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fec977fe640 1 -- 172.21.15.45:0/4064120087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca801e070 con 0x55f68b651980 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fec9e7fc640 1 -- 172.21.15.45:0/3491272056 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feca00efd40 con 0x7fecac101720 2024-04-25T00:23:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fec977fe640 1 -- 172.21.15.45:0/4064120087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca8019b10 con 0x55f68b651980 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fec977fe640 1 -- 172.21.15.45:0/4064120087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca800eae0 con 0x55f68b651980 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 -- 172.21.15.45:0/4064120087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 msgr2=0x55f68b653d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb194d640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 0x7fec980a51c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feca4005fd0 tx=0x7feca4009600 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feca8009c90 tx=0x7feca8011810 comp rx=0 tx=0).stop 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 -- 172.21.15.45:0/4064120087 shutdown_connections 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b653d70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642da0 0x55f68b651440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 --2- 172.21.15.45:0/4064120087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642480 0x55f68b642860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.774+0000 7fecb619d740 1 -- 172.21.15.45:0/4064120087 >> 172.21.15.45:0/4064120087 conn(0x55f68b63e8a0 msgr2=0x55f68b63ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- 172.21.15.45:0/4064120087 shutdown_connections 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- 172.21.15.45:0/4064120087 wait complete. 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- start start 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 Processor -- start 2024-04-25T00:23:39.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b651200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f68b654740 con 0x55f68b651980 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f68b563050 con 0x55f68b642480 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f68b6548c0 con 0x55f68b642da0 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.775+0000 7fecb619d740 10 client.0 _ll_register_callbacks cb 0x55f68b2babe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb094b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34578/0 (socket says 172.21.15.45:34578) 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 -- 172.21.15.45:0/951038125 learned_addr learned my addr 172.21.15.45:0/951038125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb094b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:60574/0 (socket says 172.21.15.45:60574) 2024-04-25T00:23:39.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb114c640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b651200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 -- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 msgr2=0x55f68b64e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 -- 172.21.15.45:0/951038125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 msgr2=0x55f68b651200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b651200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 -- 172.21.15.45:0/951038125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68b534dc0 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb194d640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feca4011d40 tx=0x7feca4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b2535a8633bf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fec94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4004d20 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feca4005020 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca8019b10 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.776+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feca4005260 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 10 client.16188 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f68b2bb3f0 con 0x55f68b651980 2024-04-25T00:23:39.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca8019b10 con 0x55f68b651980 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feca400f040 con 0x55f68b651980 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 10 client.16188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 20 client.16188 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6619e0 0x55f68b663e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 4 client.16188 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f68b5b4bb0 con 0x55f68b6619e0 2024-04-25T00:23:39.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.777+0000 7fecb114c640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6619e0 0x55f68b663e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.778+0000 7fec94ff9640 10 client.16188 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.778+0000 7fecb114c640 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6619e0 0x55f68b663e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17f6fba35029c1ff server_cookie=515059e29f3a3cfa in_seq=0 out_seq=0 2024-04-25T00:23:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.778+0000 7fec94ff9640 1 -- 172.21.15.45:0/951038125 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f68b5b4bb0 con 0x55f68b6619e0 2024-04-25T00:23:39.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.778+0000 7fec94ff9640 10 client.16188 handle_command_reply: tid=0 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.778+0000 7fecb619d740 1 client.16188 shutdown 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 msgr2=0x55f68b64ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feca4011d40 tx=0x7feca4004a40 comp rx=0 tx=0).stop 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 shutdown_connections 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6619e0 0x55f68b663e30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64ea30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fec94ff9640 0 client.16188 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 --2- 172.21.15.45:0/951038125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b651200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 >> 172.21.15.45:0/951038125 conn(0x55f68b63e8a0 msgr2=0x55f68b6446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 shutdown_connections 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 1 -- 172.21.15.45:0/951038125 wait complete. 2024-04-25T00:23:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.779+0000 7fecb619d740 20 client.16188 trim_cache size 0 max 16384 2024-04-25T00:23:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.792+0000 7fecb619d740 1 Processor -- start 2024-04-25T00:23:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.792+0000 7fecb619d740 1 -- start start 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b651980 0x55f68b669700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ac960 con 0x55f68b642da0 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ae4a0 con 0x55f68b642480 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb619d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ad030 con 0x55f68b651980 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34590/0 (socket says 172.21.15.45:34590) 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 -- 172.21.15.45:0/2372496910 learned_addr learned my addr 172.21.15.45:0/2372496910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb114c640 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb194d640 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b651980 0x55f68b669700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 -- 172.21.15.45:0/2372496910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b651980 msgr2=0x55f68b669700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b651980 0x55f68b669700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 -- 172.21.15.45:0/2372496910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 msgr2=0x55f68b64d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 -- 172.21.15.45:0/2372496910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68b3857d0 con 0x55f68b642da0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.793+0000 7fecb094b640 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feca8009250 tx=0x7feca80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3334bfa4cb88f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fec96ffd640 1 -- 172.21.15.45:0/2372496910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca802a020 con 0x55f68b642da0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fec96ffd640 1 -- 172.21.15.45:0/2372496910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca801f040 con 0x55f68b642da0 2024-04-25T00:23:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fec96ffd640 1 -- 172.21.15.45:0/2372496910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca801e040 con 0x55f68b642da0 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 -- 172.21.15.45:0/2372496910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 msgr2=0x55f68b650d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7feca8009250 tx=0x7feca80099e0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 -- 172.21.15.45:0/2372496910 shutdown_connections 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b651980 0x55f68b669700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b642da0 0x55f68b650d30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 --2- 172.21.15.45:0/2372496910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 -- 172.21.15.45:0/2372496910 >> 172.21.15.45:0/2372496910 conn(0x55f68b63e8a0 msgr2=0x55f68b63d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.794+0000 7fecb619d740 1 -- 172.21.15.45:0/2372496910 shutdown_connections 2024-04-25T00:23:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.795+0000 7fecb619d740 1 -- 172.21.15.45:0/2372496910 wait complete. 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.795+0000 7fecb619d740 1 -- start start 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.795+0000 7fecb619d740 1 Processor -- start 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1addd0 con 0x55f68b651980 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1ad700 con 0x55f68b642480 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fecac1aa940 con 0x55f68b642da0 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb619d740 10 client.0 _ll_register_callbacks cb 0x7feca4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34600/0 (socket says 172.21.15.45:34600) 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 -- 172.21.15.45:0/3861247539 learned_addr learned my addr 172.21.15.45:0/3861247539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb094b640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb114c640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 -- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 msgr2=0x55f68b64b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 -- 172.21.15.45:0/3861247539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 msgr2=0x55f68b64b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 -- 172.21.15.45:0/3861247539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68b6c71c0 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb094b640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb114c640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fecb194d640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feca4011d40 tx=0x7feca401fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d784150c62f6e8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.796+0000 7fec94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca403a070 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feca4003970 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca801f040 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feca4046050 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 10 client.16194 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f68b6c8480 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca801f040 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feca400f040 con 0x55f68b651980 2024-04-25T00:23:39.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 20 client.16194 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6c8ba0 0x55f68b6cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 4 client.16194 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f68b64edc0 con 0x55f68b6c8ba0 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.797+0000 7fecb114c640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6c8ba0 0x55f68b6cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb114c640 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6c8ba0 0x55f68b6cafc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d5790b66a5ef0a8 server_cookie=4d1ec29e087c7160 in_seq=0 out_seq=0 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fec94ff9640 10 client.16194 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fec94ff9640 1 -- 172.21.15.45:0/3861247539 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f68b64edc0 con 0x55f68b6c8ba0 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fec94ff9640 10 client.16194 handle_command_reply: tid=0 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 client.16194 shutdown 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 msgr2=0x55f68b64be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7feca4011d40 tx=0x7feca401fc90 comp rx=0 tx=0).stop 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 shutdown_connections 2024-04-25T00:23:39.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55f68b6c8ba0 0x55f68b6cafc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b651980 0x55f68b64be50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fec94ff9640 0 client.16194 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f68b642da0 0x55f68b64b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 --2- 172.21.15.45:0/3861247539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f68b642480 0x55f68b64b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.798+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 >> 172.21.15.45:0/3861247539 conn(0x55f68b63e8a0 msgr2=0x55f68b63d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 shutdown_connections 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb619d740 1 -- 172.21.15.45:0/3861247539 wait complete. 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb619d740 20 client.16194 trim_cache size 0 max 16384 2024-04-25T00:23:39.807 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 msgr2=0x7fec980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 0x7fec980a51c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feca4005fd0 tx=0x7feca4009600 comp rx=0 tx=0).stop 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 msgr2=0x7fecac198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7feca0009230 tx=0x7feca0004270 comp rx=0 tx=0).stop 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 shutdown_connections 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fec980a2d00 0x7fec980a51c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac107730 0x7fecac19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.799+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fecac1020e0 0x7fecac198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.800+0000 7fecb33b0640 1 --2- 172.21.15.45:0/3491272056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fecac101720 0x7fecac198020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.800+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 >> 172.21.15.45:0/3491272056 conn(0x7fecac078b40 msgr2=0x7fecac104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.800+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 shutdown_connections 2024-04-25T00:23:39.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.800+0000 7fecb33b0640 1 -- 172.21.15.45:0/3491272056 wait complete. 2024-04-25T00:23:39.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 -- 172.21.15.45:0/2766896669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 msgr2=0x7feffc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 --2- 172.21.15.45:0/2766896669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc100aa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009930 tx=0x7fefe402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 -- 172.21.15.45:0/2766896669 shutdown_connections 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 --2- 172.21.15.45:0/2766896669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 0x7feffc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 --2- 172.21.15.45:0/2766896669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc100aa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 --2- 172.21.15.45:0/2766896669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 -- 172.21.15.45:0/2766896669 >> 172.21.15.45:0/2766896669 conn(0x7feffc0776a0 msgr2=0x7feffc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 -- 172.21.15.45:0/2766896669 shutdown_connections 2024-04-25T00:23:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.921+0000 7ff00097e640 1 -- 172.21.15.45:0/2766896669 wait complete. 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 Processor -- start 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 -- start start 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 0x7feffc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feffc111b40 con 0x7feffc100620 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1119c0 con 0x7feffc106630 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.922+0000 7ff00097e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feffc111cc0 con 0x7feffc100fe0 2024-04-25T00:23:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34618/0 (socket says 172.21.15.45:34618) 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 -- 172.21.15.45:0/3802918562 learned_addr learned my addr 172.21.15.45:0/3802918562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffad9d640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feff3fff640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 0x7feffc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 -- 172.21.15.45:0/3802918562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 msgr2=0x7feffc1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 0x7feffc1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 -- 172.21.15.45:0/3802918562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 msgr2=0x7feffc19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 -- 172.21.15.45:0/3802918562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc19d020 con 0x7feffc100620 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffad9d640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feffa59c640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e9f0 tx=0x7fefec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feff37fe640 1 -- 172.21.15.45:0/3802918562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec00cd80 con 0x7feffc100620 2024-04-25T00:23:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feff37fe640 1 -- 172.21.15.45:0/3802918562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefec010910 con 0x7feffc100620 2024-04-25T00:23:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7feff37fe640 1 -- 172.21.15.45:0/3802918562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec018990 con 0x7feffc100620 2024-04-25T00:23:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc19d310 con 0x7feffc100620 2024-04-25T00:23:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.923+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc1a4ba0 con 0x7feffc100620 2024-04-25T00:23:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.926+0000 7feff37fe640 1 -- 172.21.15.45:0/3802918562 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fefec010ac0 con 0x7feffc100620 2024-04-25T00:23:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 Processor -- start 2024-04-25T00:23:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 -- start start 2024-04-25T00:23:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbaf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbb1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f70bb09670 con 0x55f70bba4580 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f70bab1970 con 0x55f70bba4ea0 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7ff00376b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f70bb05630 con 0x55f70bbaf570 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34628/0 (socket says 172.21.15.45:34628) 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffad9d640 1 -- 172.21.15.45:0/2722993121 learned_addr learned my addr 172.21.15.45:0/2722993121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffa59c640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbaf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feff3fff640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbb1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffad9d640 1 -- 172.21.15.45:0/2722993121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 msgr2=0x55f70bbb1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.927+0000 7feffad9d640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbb1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffad9d640 1 -- 172.21.15.45:0/2722993121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 msgr2=0x55f70bbaf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feff37fe640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 0x7fefe80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffad9d640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbaf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffad9d640 1 -- 172.21.15.45:0/2722993121 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70bad5400 con 0x55f70bba4580 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffa59c640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbaf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffad9d640 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feff40117e0 tx=0x7feff4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ca412b81d0700d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feff37fe640 1 -- 172.21.15.45:0/3802918562 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fefec0cfa30 con 0x7feffc100620 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feffad9d640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 0x7fefe80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feff0ff9640 1 -- 172.21.15.45:0/2722993121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4003a10 con 0x55f70bba4580 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feff0ff9640 1 -- 172.21.15.45:0/2722993121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff400fe80 con 0x55f70bba4580 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.928+0000 7feff0ff9640 1 -- 172.21.15.45:0/2722993121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4010690 con 0x55f70bba4580 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- 172.21.15.45:0/2722993121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 msgr2=0x55f70bba4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7feffad9d640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 0x7fefe80a5240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feff401b000 tx=0x7feff4010970 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feff40117e0 tx=0x7feff4011ca0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- 172.21.15.45:0/2722993121 shutdown_connections 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbb1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbaf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 --2- 172.21.15.45:0/2722993121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba4960 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- 172.21.15.45:0/2722993121 >> 172.21.15.45:0/2722993121 conn(0x55f70bba08a0 msgr2=0x55f70bba0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- 172.21.15.45:0/2722993121 shutdown_connections 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- 172.21.15.45:0/2722993121 wait complete. 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 -- start start 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.929+0000 7ff00376b740 1 Processor -- start 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 0x55f70bbaee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4ea0 0x55f70bbac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f70bbb2360 con 0x55f70bbaf570 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f70bac5050 con 0x55f70bba4580 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f70bbb24e0 con 0x55f70bba4ea0 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7ff00376b740 10 client.0 _ll_register_callbacks cb 0x55f70b81cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34634/0 (socket says 172.21.15.45:34634) 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 -- 172.21.15.45:0/2087377476 learned_addr learned my addr 172.21.15.45:0/2087377476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feff3fff640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4ea0 0x55f70bbac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffa59c640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 0x55f70bbaee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 -- 172.21.15.45:0/2087377476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4ea0 msgr2=0x55f70bbac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4ea0 0x55f70bbac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 -- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 msgr2=0x55f70bbaee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 0x55f70bbaee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 -- 172.21.15.45:0/2087377476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70ba96dc0 con 0x55f70bbaf570 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffa59c640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 0x55f70bbaee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:39.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7feffad9d640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feff400a010 tx=0x7feff4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5517bda2bee89800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7fefe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.930+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4037060 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feff4010640 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f70bb16bb0 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feff403e050 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 10 client.16206 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f70b81d3f0 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f70bb16bb0 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feff4018070 con 0x55f70bbaf570 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 10 client.16206 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 20 client.16206 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bbbf4a0 0x55f70bbc1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 4 client.16206 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f70bb10f00 con 0x55f70bbbf4a0 2024-04-25T00:23:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.931+0000 7feffa59c640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bbbf4a0 0x55f70bbc1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.932+0000 7feffa59c640 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bbbf4a0 0x55f70bbc1890 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f8e4d1d90d0399 server_cookie=6eb53d86bee4ed5 in_seq=0 out_seq=0 2024-04-25T00:23:39.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.932+0000 7fefe1ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.932+0000 7fefe1ffb640 1 -- 172.21.15.45:0/2087377476 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f70bb10f00 con 0x55f70bbbf4a0 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.932+0000 7fefe1ffb640 10 client.16206 handle_command_reply: tid=0 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 client.16206 shutdown 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 msgr2=0x55f70bbac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feff400a010 tx=0x7feff4018000 comp rx=0 tx=0).stop 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 shutdown_connections 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bbbf4a0 0x55f70bbc1890 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bbac650 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4ea0 0x55f70bbac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 --2- 172.21.15.45:0/2087377476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4580 0x55f70bbaee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 >> 172.21.15.45:0/2087377476 conn(0x55f70bba08a0 msgr2=0x55f70bb9fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7fefe1ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 shutdown_connections 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 1 -- 172.21.15.45:0/2087377476 wait complete. 2024-04-25T00:23:39.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.933+0000 7ff00376b740 20 client.16206 trim_cache size 0 max 16384 2024-04-25T00:23:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.946+0000 7ff00376b740 1 Processor -- start 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 -- start start 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bba02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbbb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1aebc0 con 0x55f70bba4580 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1b0700 con 0x55f70bba4ea0 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7ff00376b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1af290 con 0x55f70bbaf570 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34640/0 (socket says 172.21.15.45:34640) 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 -- 172.21.15.45:0/4212408170 learned_addr learned my addr 172.21.15.45:0/4212408170 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feff3fff640 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bba02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffad9d640 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbbb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 -- 172.21.15.45:0/4212408170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 msgr2=0x55f70bbbb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbbb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 -- 172.21.15.45:0/4212408170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 msgr2=0x55f70bba02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bba02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 -- 172.21.15.45:0/4212408170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70b8e77d0 con 0x55f70bba4580 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.947+0000 7feffa59c640 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fefec098050 tx=0x7fefec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48d9c44e7e9e576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7fefe3fff640 1 -- 172.21.15.45:0/4212408170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec09bbf0 con 0x55f70bba4580 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7fefe3fff640 1 -- 172.21.15.45:0/4212408170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefec09ffa0 con 0x55f70bba4580 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7fefe3fff640 1 -- 172.21.15.45:0/4212408170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec018e20 con 0x55f70bba4580 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 -- 172.21.15.45:0/4212408170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 msgr2=0x55f70bba67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fefec098050 tx=0x7fefec000bc0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 -- 172.21.15.45:0/4212408170 shutdown_connections 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bbaf570 0x55f70bbbb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bba02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 --2- 172.21.15.45:0/4212408170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bba4580 0x55f70bba67a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 -- 172.21.15.45:0/4212408170 >> 172.21.15.45:0/4212408170 conn(0x55f70bba08a0 msgr2=0x55f70bbbaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.948+0000 7ff00376b740 1 -- 172.21.15.45:0/4212408170 shutdown_connections 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 -- 172.21.15.45:0/4212408170 wait complete. 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 -- start start 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 Processor -- start 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4580 0x55f70bbb7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbb7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1b0030 con 0x55f70bbaf570 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1af960 con 0x55f70bba4ea0 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1acba0 con 0x55f70bba4580 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7ff00376b740 10 client.0 _ll_register_callbacks cb 0x7feff40116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7feffad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34648/0 (socket says 172.21.15.45:34648) 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7feffad9d640 1 -- 172.21.15.45:0/902276312 learned_addr learned my addr 172.21.15.45:0/902276312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7feffa59c640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4580 0x55f70bbb7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.949+0000 7feff3fff640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbb7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 -- 172.21.15.45:0/902276312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4580 msgr2=0x55f70bbb7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4580 0x55f70bbb7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 -- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 msgr2=0x55f70bbb7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbb7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 -- 172.21.15.45:0/902276312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70bba8960 con 0x55f70bbaf570 2024-04-25T00:23:39.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feff3fff640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbb7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7feffad9d640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feff40049c0 tx=0x7feff40115e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f962d3c5a906139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7fefe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4037040 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefec09ffa0 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feff4010640 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feff4052050 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7ff00376b740 10 client.16212 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f70bc251a0 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.950+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff4046020 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feff4004250 con 0x55f70bbaf570 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 20 client.16212 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bc258e0 0x55f70bc27d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:39.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 4 client.16212 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f70ba8e820 con 0x55f70bc258e0 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7feffa59c640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bc258e0 0x55f70bc27d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7fefe1ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7feffa59c640 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bc258e0 0x55f70bc27d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ff79ea973fa948 server_cookie=b0a967b9410eaa86 in_seq=0 out_seq=0 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7fefe1ffb640 1 -- 172.21.15.45:0/902276312 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f70ba8e820 con 0x55f70bc258e0 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7fefe1ffb640 10 client.16212 handle_command_reply: tid=0 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.951+0000 7ff00376b740 1 client.16212 shutdown 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 msgr2=0x55f70bba83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feff40049c0 tx=0x7feff40115e0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 shutdown_connections 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f70bc258e0 0x55f70bc27d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f70bbaf570 0x55f70bba83a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f70bba4ea0 0x55f70bbb7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7fefe1ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:39.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 --2- 172.21.15.45:0/902276312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f70bba4580 0x55f70bbb7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 >> 172.21.15.45:0/902276312 conn(0x55f70bba08a0 msgr2=0x55f70bbba210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 shutdown_connections 2024-04-25T00:23:39.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 1 -- 172.21.15.45:0/902276312 wait complete. 2024-04-25T00:23:39.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00376b740 20 client.16212 trim_cache size 0 max 16384 2024-04-25T00:23:39.959 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 msgr2=0x7fefe80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 0x7fefe80a5240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feff401b000 tx=0x7feff4010970 comp rx=0 tx=0).stop 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 msgr2=0x7feffc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.952+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e9f0 tx=0x7fefec00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 shutdown_connections 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe80a2d80 0x7fefe80a5240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feffc106630 0x7feffc19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feffc100fe0 0x7feffc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 --2- 172.21.15.45:0/3802918562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feffc100620 0x7feffc198080 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 >> 172.21.15.45:0/3802918562 conn(0x7feffc0776a0 msgr2=0x7feffc103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 shutdown_connections 2024-04-25T00:23:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:39.953+0000 7ff00097e640 1 -- 172.21.15.45:0/3802918562 wait complete. 2024-04-25T00:23:40.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:40 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2620995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:40.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:40 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2620995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:40.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:40 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2620995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:40.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:23:40.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:23:40.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.077+0000 7f43494f5640 1 -- 172.21.15.45:0/642906673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 msgr2=0x7f4344102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.077+0000 7f43494f5640 1 --2- 172.21.15.45:0/642906673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344102da0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f432c009900 tx=0x7f432c02a510 comp rx=0 tx=0).stop 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 -- 172.21.15.45:0/642906673 shutdown_connections 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 --2- 172.21.15.45:0/642906673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f434410bd00 0x7f434410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 --2- 172.21.15.45:0/642906673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43441032e0 0x7f434410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 --2- 172.21.15.45:0/642906673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344102da0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 -- 172.21.15.45:0/642906673 >> 172.21.15.45:0/642906673 conn(0x7f4344077740 msgr2=0x7f4344077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 -- 172.21.15.45:0/642906673 shutdown_connections 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 -- 172.21.15.45:0/642906673 wait complete. 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 Processor -- start 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.078+0000 7f43494f5640 1 -- start start 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 0x7f4344198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f434410bd00 0x7f434419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111b00 con 0x7f43441029c0 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111980 con 0x7f43441032e0 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43494f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111c80 con 0x7f434410bd00 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f4342ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f4342ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34662/0 (socket says 172.21.15.45:34662) 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f4342ffd640 1 -- 172.21.15.45:0/1034750663 learned_addr learned my addr 172.21.15.45:0/1034750663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 0x7f4344198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43437fe640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f434410bd00 0x7f434419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 -- 172.21.15.45:0/1034750663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f434410bd00 msgr2=0x7f434419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f434410bd00 0x7f434419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 -- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 msgr2=0x7f4344198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 -- 172.21.15.45:0/1034750663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419d010 con 0x7f43441032e0 2024-04-25T00:23:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f43427fc640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 0x7f4344198560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f433400d6d0 tx=0x7f433400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.079+0000 7f433bfff640 1 -- 172.21.15.45:0/1034750663 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334017070 con 0x7f43441032e0 2024-04-25T00:23:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.080+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434419d300 con 0x7f43441032e0 2024-04-25T00:23:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.080+0000 7f433bfff640 1 -- 172.21.15.45:0/1034750663 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334009820 con 0x7f43441032e0 2024-04-25T00:23:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.080+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a4b90 con 0x7f43441032e0 2024-04-25T00:23:41.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.080+0000 7f433bfff640 1 -- 172.21.15.45:0/1034750663 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334004cd0 con 0x7f43441032e0 2024-04-25T00:23:41.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.082+0000 7f433bfff640 1 -- 172.21.15.45:0/1034750663 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f43340099d0 con 0x7f43441032e0 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.082+0000 7f433bfff640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 0x7f43300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.083+0000 7f4342ffd640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 0x7f43300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.083+0000 7f433bfff640 1 -- 172.21.15.45:0/1034750663 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f43340cf120 con 0x7f43441032e0 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.083+0000 7f4342ffd640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 0x7f43300a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f432c0096c0 tx=0x7f432c005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.083+0000 7f434c1fe740 1 Processor -- start 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 -- start start 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c48cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c54010 0x55b000c56400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b000bad2c0 con 0x55b000c49290 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b000b55910 con 0x55b000c4a980 2024-04-25T00:23:41.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f434c1fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b000ba9280 con 0x55b000c54010 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f43427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f43427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34668/0 (socket says 172.21.15.45:34668) 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f43427fc640 1 -- 172.21.15.45:0/684924182 learned_addr learned my addr 172.21.15.45:0/684924182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f4342ffd640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c48cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f43437fe640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c54010 0x55b000c56400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.084+0000 7f43427fc640 1 -- 172.21.15.45:0/684924182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c54010 msgr2=0x55b000c56400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43427fc640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c54010 0x55b000c56400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43427fc640 1 -- 172.21.15.45:0/684924182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 msgr2=0x55b000c48cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43427fc640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c48cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43427fc640 1 -- 172.21.15.45:0/684924182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b000b66fb0 con 0x55b000c49290 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f4342ffd640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c48cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43427fc640 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4334097740 tx=0x7f433409b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6dd9e062413a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43397fa640 1 -- 172.21.15.45:0/684924182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334020070 con 0x55b000c49290 2024-04-25T00:23:41.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43397fa640 1 -- 172.21.15.45:0/684924182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334095e50 con 0x55b000c49290 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.085+0000 7f43397fa640 1 -- 172.21.15.45:0/684924182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433409f3d0 con 0x55b000c49290 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 -- 172.21.15.45:0/684924182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 msgr2=0x55b000c49710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4334097740 tx=0x7f433409b160 comp rx=0 tx=0).stop 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 -- 172.21.15.45:0/684924182 shutdown_connections 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c54010 0x55b000c56400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c49290 0x55b000c49710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 --2- 172.21.15.45:0/684924182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c48cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 -- 172.21.15.45:0/684924182 >> 172.21.15.45:0/684924182 conn(0x55b000c11950 msgr2=0x55b000c457f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 -- 172.21.15.45:0/684924182 shutdown_connections 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.086+0000 7f434c1fe740 1 -- 172.21.15.45:0/684924182 wait complete. 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 -- start start 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 Processor -- start 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c50f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 0x55b000c519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b000c56e40 con 0x55b000c4a980 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b000b68e90 con 0x55b000c54010 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b000c56fc0 con 0x55b000c49290 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f434c1fe740 10 client.0 _ll_register_callbacks cb 0x55b000ba4970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34682/0 (socket says 172.21.15.45:34682) 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 -- 172.21.15.45:0/1472417118 learned_addr learned my addr 172.21.15.45:0/1472417118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f4342ffd640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c50f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43437fe640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 0x55b000c519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 -- 172.21.15.45:0/1472417118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 msgr2=0x55b000c50f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c50f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 -- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 msgr2=0x55b000c519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 0x55b000c519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43427fc640 1 -- 172.21.15.45:0/1472417118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0009897e0 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.087+0000 7f43437fe640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 0x55b000c519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f43427fc640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f43340cf440 tx=0x7f4334097b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe60a5a87cf222e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334020040 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f43340cc790 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4334095e50 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4334082050 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 10 client.16224 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0009e9f70 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4334095e50 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43340bb0b0 con 0x55b000c4a980 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 10 client.16224 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:41.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 20 client.16224 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000c63f40 0x55b000c66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 4 client.16224 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.088+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b000b815a0 con 0x55b000c63f40 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.089+0000 7f4342ffd640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000c63f40 0x55b000c66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.089+0000 7f432a7fc640 10 client.16224 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.089+0000 7f4342ffd640 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000c63f40 0x55b000c66330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f45cf80e35eb8ac server_cookie=4f3f502f4e56d41e in_seq=0 out_seq=0 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f432a7fc640 1 -- 172.21.15.45:0/1472417118 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b000b815a0 con 0x55b000c63f40 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f432a7fc640 10 client.16224 handle_command_reply: tid=0 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 client.16224 shutdown 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 msgr2=0x55b000c51480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f43340cf440 tx=0x7f4334097b80 comp rx=0 tx=0).stop 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 shutdown_connections 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c54010 0x55b000c519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000c63f40 0x55b000c66330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c4a980 0x55b000c51480 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 --2- 172.21.15.45:0/1472417118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c50f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f432a7fc640 0 client.16224 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 >> 172.21.15.45:0/1472417118 conn(0x55b000c11950 msgr2=0x55b000c4b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 shutdown_connections 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 1 -- 172.21.15.45:0/1472417118 wait complete. 2024-04-25T00:23:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.090+0000 7f434c1fe740 20 client.16224 trim_cache size 0 max 16384 2024-04-25T00:23:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.108+0000 7f434c1fe740 1 Processor -- start 2024-04-25T00:23:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.108+0000 7f434c1fe740 1 -- start start 2024-04-25T00:23:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c5fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43441aec30 con 0x55b000c54010 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b0770 con 0x55b000c4a980 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f434c1fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43441af300 con 0x55b000c49290 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34684/0 (socket says 172.21.15.45:34684) 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 -- 172.21.15.45:0/2734600846 learned_addr learned my addr 172.21.15.45:0/2734600846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f4342ffd640 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c5fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43427fc640 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 -- 172.21.15.45:0/2734600846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 msgr2=0x55b000c5fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c5fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 -- 172.21.15.45:0/2734600846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 msgr2=0x55b000c517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 -- 172.21.15.45:0/2734600846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b000881170 con 0x55b000c54010 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f43437fe640 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f433c003a50 tx=0x7f433c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fada3e583117b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.109+0000 7f4338ff9640 1 -- 172.21.15.45:0/2734600846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c004680 con 0x55b000c54010 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f4338ff9640 1 -- 172.21.15.45:0/2734600846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c004820 con 0x55b000c54010 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f4338ff9640 1 -- 172.21.15.45:0/2734600846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0116a0 con 0x55b000c54010 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 -- 172.21.15.45:0/2734600846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 msgr2=0x55b000c6b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f433c003a50 tx=0x7f433c003f10 comp rx=0 tx=0).stop 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 -- 172.21.15.45:0/2734600846 shutdown_connections 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c6b990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c4a980 0x55b000c517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2734600846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c49290 0x55b000c5fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 -- 172.21.15.45:0/2734600846 >> 172.21.15.45:0/2734600846 conn(0x55b000c11950 msgr2=0x55b000c44950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 -- 172.21.15.45:0/2734600846 shutdown_connections 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.110+0000 7f434c1fe740 1 -- 172.21.15.45:0/2734600846 wait complete. 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 -- start start 2024-04-25T00:23:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 Processor -- start 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c49290 0x55b000c4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c4a980 0x55b000c4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43441b00a0 con 0x55b000c54010 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43441af9d0 con 0x55b000c49290 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43441acc10 con 0x55b000c4a980 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f434c1fe740 10 client.0 _ll_register_callbacks cb 0x7f433409ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f43437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f43437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34700/0 (socket says 172.21.15.45:34700) 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f43437fe640 1 -- 172.21.15.45:0/2206331552 learned_addr learned my addr 172.21.15.45:0/2206331552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f43427fc640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c4a980 0x55b000c4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.111+0000 7f4342ffd640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c49290 0x55b000c4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 -- 172.21.15.45:0/2206331552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c4a980 msgr2=0x55b000c4e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c4a980 0x55b000c4e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 -- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c49290 msgr2=0x55b000c4de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c49290 0x55b000c4de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 -- 172.21.15.45:0/2206331552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b000cc9450 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f43437fe640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f433c01cc10 tx=0x7f433c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd7f4d3d6b90970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c004680 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f433c018040 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433c004820 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f433c011440 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b000cca750 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c004820 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f433c0276b0 con 0x55b000c54010 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:41.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 20 client.16230 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000ccade0 0x55b000ccd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 4 client.16230 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.112+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b000c45d50 con 0x55b000ccade0 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.113+0000 7f4342ffd640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000ccade0 0x55b000ccd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.113+0000 7f4342ffd640 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000ccade0 0x55b000ccd200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3022a82a2612ced0 server_cookie=95cfef9e35bbc488 in_seq=0 out_seq=0 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.113+0000 7f432a7fc640 10 client.16230 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f432a7fc640 1 -- 172.21.15.45:0/2206331552 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b000c45d50 con 0x55b000ccade0 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f432a7fc640 10 client.16230 handle_command_reply: tid=0 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 client.16230 shutdown 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 msgr2=0x55b000c4e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f433c01cc10 tx=0x7f433c01c6a0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 shutdown_connections 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b000ccade0 0x55b000ccd200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b000c54010 0x55b000c4e8d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b000c4a980 0x55b000c4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 --2- 172.21.15.45:0/2206331552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b000c49290 0x55b000c4de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 >> 172.21.15.45:0/2206331552 conn(0x55b000c11950 msgr2=0x55b000c440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.114+0000 7f432a7fc640 0 client.16230 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 shutdown_connections 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f434c1fe740 1 -- 172.21.15.45:0/2206331552 wait complete. 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f434c1fe740 20 client.16230 trim_cache size 0 max 16384 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "85de2c98-5b93-4d1e-a9fe-a139c4998ab4", 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:23:41.121 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 msgr2=0x7f43300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 0x7f43300a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f432c0096c0 tx=0x7f432c005a20 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 msgr2=0x7f4344198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 0x7f4344198560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f433400d6d0 tx=0x7f433400db90 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 shutdown_connections 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43300a2d00 0x7f43300a51c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f434410bd00 0x7f434419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43441032e0 0x7f4344198560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 --2- 172.21.15.45:0/1034750663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43441029c0 0x7f4344198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 >> 172.21.15.45:0/1034750663 conn(0x7f4344077740 msgr2=0x7f43440fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 shutdown_connections 2024-04-25T00:23:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.115+0000 7f43494f5640 1 -- 172.21.15.45:0/1034750663 wait complete. 2024-04-25T00:23:41.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:41.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 -- 172.21.15.45:0/3852779611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67701066b0 msgr2=0x7f6770106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 --2- 172.21.15.45:0/3852779611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67701066b0 0x7f6770106a90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6760009930 tx=0x7f676002a420 comp rx=0 tx=0).stop 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 -- 172.21.15.45:0/3852779611 shutdown_connections 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 --2- 172.21.15.45:0/3852779611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 0x7f677010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 --2- 172.21.15.45:0/3852779611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6770100610 0x7f6770100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 --2- 172.21.15.45:0/3852779611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67701066b0 0x7f6770106a90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 -- 172.21.15.45:0/3852779611 >> 172.21.15.45:0/3852779611 conn(0x7f6770077740 msgr2=0x7f6770077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 -- 172.21.15.45:0/3852779611 shutdown_connections 2024-04-25T00:23:41.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.237+0000 7f67764a3640 1 -- 172.21.15.45:0/3852779611 wait complete. 2024-04-25T00:23:41.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 Processor -- start 2024-04-25T00:23:41.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 -- start start 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 0x7f6770199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67701066b0 0x7f67701a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6770110a30 con 0x7f6770100610 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f67701108b0 con 0x7f67701066b0 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f67764a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6770110bb0 con 0x7f6770100fd0 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34716/0 (socket says 172.21.15.45:34716) 2024-04-25T00:23:41.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f676ffff640 1 -- 172.21.15.45:0/105857678 learned_addr learned my addr 172.21.15.45:0/105857678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f6774a40640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67701066b0 0x7f67701a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.238+0000 7f676f7fe640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 0x7f6770199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 -- 172.21.15.45:0/105857678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 msgr2=0x7f6770199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 0x7f6770199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 -- 172.21.15.45:0/105857678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67701066b0 msgr2=0x7f67701a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67701066b0 0x7f67701a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 -- 172.21.15.45:0/105857678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67701a4390 con 0x7f6770100610 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676ffff640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6760005940 tx=0x7f6760004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676d7fa640 1 -- 172.21.15.45:0/105857678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676003f030 con 0x7f6770100610 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67701a4560 con 0x7f6770100610 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676d7fa640 1 -- 172.21.15.45:0/105857678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6760003ad0 con 0x7f6770100610 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f676d7fa640 1 -- 172.21.15.45:0/105857678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760035070 con 0x7f6770100610 2024-04-25T00:23:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.239+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701a49c0 con 0x7f6770100610 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.241+0000 7f676d7fa640 1 -- 172.21.15.45:0/105857678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6760042030 con 0x7f6770100610 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 Processor -- start 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 -- start start 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31a360 0x5631bd31a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd32be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5631bd27f6b0 con 0x5631bd31ad10 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5631bd227880 con 0x5631bd31a360 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f67791a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631bd27b670 con 0x5631bd329a30 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34722/0 (socket says 172.21.15.45:34722) 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 -- 172.21.15.45:0/1767484455 learned_addr learned my addr 172.21.15.45:0/1767484455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676d7fa640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 0x7f67580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f6774a40640 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31a360 0x5631bd31a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676f7fe640 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd32be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f6774a40640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 0x7f67580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676d7fa640 1 -- 172.21.15.45:0/105857678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f67600efc00 con 0x7f6770100610 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 -- 172.21.15.45:0/1767484455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 msgr2=0x5631bd32be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd32be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 -- 172.21.15.45:0/1767484455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31a360 msgr2=0x5631bd31a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31a360 0x5631bd31a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.242+0000 7f676ffff640 1 -- 172.21.15.45:0/1767484455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bd05d7d0 con 0x5631bd31ad10 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.243+0000 7f676ffff640 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f67600bbd50 tx=0x7f676002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b082dd79fd43cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.243+0000 7f67567fc640 1 -- 172.21.15.45:0/1767484455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760049070 con 0x5631bd31ad10 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.243+0000 7f6774a40640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 0x7f67580a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6768005fd0 tx=0x7f6768015040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.243+0000 7f67567fc640 1 -- 172.21.15.45:0/1767484455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67600bc4c0 con 0x5631bd31ad10 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.243+0000 7f67567fc640 1 -- 172.21.15.45:0/1767484455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760003c80 con 0x5631bd31ad10 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- 172.21.15.45:0/1767484455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 msgr2=0x5631bd3294f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f67600bbd50 tx=0x7f676002a420 comp rx=0 tx=0).stop 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- 172.21.15.45:0/1767484455 shutdown_connections 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd32be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31ad10 0x5631bd3294f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 --2- 172.21.15.45:0/1767484455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31a360 0x5631bd31a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- 172.21.15.45:0/1767484455 >> 172.21.15.45:0/1767484455 conn(0x5631bd316950 msgr2=0x5631bd316d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- 172.21.15.45:0/1767484455 shutdown_connections 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- 172.21.15.45:0/1767484455 wait complete. 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 -- start start 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.244+0000 7f67791a8740 1 Processor -- start 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd326f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd327490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5631bd27f6b0 con 0x5631bd31a360 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5631bd23af50 con 0x5631bd31ad10 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631bd32c5b0 con 0x5631bd329a30 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 10 client.0 _ll_register_callbacks cb 0x5631bd276000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34736/0 (socket says 172.21.15.45:34736) 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 -- 172.21.15.45:0/3304836644 learned_addr learned my addr 172.21.15.45:0/3304836644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676f7fe640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd326f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f6774a40640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd327490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 -- 172.21.15.45:0/3304836644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 msgr2=0x5631bd327490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd327490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 -- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 msgr2=0x5631bd326f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd326f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 -- 172.21.15.45:0/3304836644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bcf933f0 con 0x5631bd31a360 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f6774a40640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd327490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f676ffff640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f67600b8790 tx=0x7f67600c4bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fafba65d555911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f674f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760049040 con 0x5631bd31a360 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f67600ed270 con 0x5631bd31a360 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.245+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67600bc4c0 con 0x5631bd31a360 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67600bb900 con 0x5631bd31a360 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 10 client.16242 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:41.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631bcf530f0 con 0x5631bd31a360 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67600bc4c0 con 0x5631bd31a360 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f676009abe0 con 0x5631bd31a360 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 10 client.16242 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 20 client.16242 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd339830 0x5631bd33bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 4 client.16242 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631bd28cbf0 con 0x5631bd339830 2024-04-25T00:23:41.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.246+0000 7f676f7fe640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd339830 0x5631bd33bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.247+0000 7f674f7fe640 10 client.16242 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.247+0000 7f676f7fe640 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd339830 0x5631bd33bc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1158e6210175bbfc server_cookie=f043e59b32227864 in_seq=0 out_seq=0 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.247+0000 7f674f7fe640 1 -- 172.21.15.45:0/3304836644 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631bd28cbf0 con 0x5631bd339830 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.247+0000 7f674f7fe640 10 client.16242 handle_command_reply: tid=0 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.247+0000 7f67791a8740 1 client.16242 shutdown 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 msgr2=0x5631bd326a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f67600b8790 tx=0x7f67600c4bb0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 shutdown_connections 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd329a30 0x5631bd327490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd326f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd339830 0x5631bd33bc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 --2- 172.21.15.45:0/3304836644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd31a360 0x5631bd326a10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f674f7fe640 0 client.16242 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 >> 172.21.15.45:0/3304836644 conn(0x5631bd316950 msgr2=0x5631bd31c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 shutdown_connections 2024-04-25T00:23:41.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 1 -- 172.21.15.45:0/3304836644 wait complete. 2024-04-25T00:23:41.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.248+0000 7f67791a8740 20 client.16242 trim_cache size 0 max 16384 2024-04-25T00:23:41.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.265+0000 7f67791a8740 1 Processor -- start 2024-04-25T00:23:41.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.265+0000 7f67791a8740 1 -- start start 2024-04-25T00:23:41.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd335330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd315970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f67701ac8e0 con 0x5631bd329a30 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f67701ae420 con 0x5631bd31ad10 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f67791a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67701acfb0 con 0x5631bd31a360 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34752/0 (socket says 172.21.15.45:34752) 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 -- 172.21.15.45:0/4133382805 learned_addr learned my addr 172.21.15.45:0/4133382805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f676f7fe640 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd315970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f676ffff640 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd335330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 -- 172.21.15.45:0/4133382805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 msgr2=0x5631bd335330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd335330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 -- 172.21.15.45:0/4133382805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 msgr2=0x5631bd315970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd315970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 -- 172.21.15.45:0/4133382805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631bd31b1e0 con 0x5631bd329a30 2024-04-25T00:23:41.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.266+0000 7f6774a40640 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6768013430 tx=0x7f6768009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fb4a32f19df121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f6755ffb640 1 -- 172.21.15.45:0/4133382805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67680129d0 con 0x5631bd329a30 2024-04-25T00:23:41.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f6755ffb640 1 -- 172.21.15.45:0/4133382805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768026030 con 0x5631bd329a30 2024-04-25T00:23:41.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f6755ffb640 1 -- 172.21.15.45:0/4133382805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768010dc0 con 0x5631bd329a30 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 -- 172.21.15.45:0/4133382805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 msgr2=0x5631bd325f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6768013430 tx=0x7f6768009500 comp rx=0 tx=0).stop 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 -- 172.21.15.45:0/4133382805 shutdown_connections 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd325f70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd315970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 --2- 172.21.15.45:0/4133382805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd335330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 -- 172.21.15.45:0/4133382805 >> 172.21.15.45:0/4133382805 conn(0x5631bd316950 msgr2=0x5631bd315260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 -- 172.21.15.45:0/4133382805 shutdown_connections 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.267+0000 7f67791a8740 1 -- 172.21.15.45:0/4133382805 wait complete. 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 -- start start 2024-04-25T00:23:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 Processor -- start 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd3980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd3985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f67701add50 con 0x5631bd329a30 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f67701ad680 con 0x5631bd31ad10 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67701aa8c0 con 0x5631bd31a360 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f67791a8740 10 client.0 _ll_register_callbacks cb 0x7f6768009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f6774a40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f6774a40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34756/0 (socket says 172.21.15.45:34756) 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f6774a40640 1 -- 172.21.15.45:0/3995649375 learned_addr learned my addr 172.21.15.45:0/3995649375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f676f7fe640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd3985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.268+0000 7f676ffff640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd3980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 -- 172.21.15.45:0/3995649375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 msgr2=0x5631bd3980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd3980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 -- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 msgr2=0x5631bd3985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd3985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 -- 172.21.15.45:0/3995649375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631bd32d0a0 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f676f7fe640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd3985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f6774a40640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6768010a20 tx=0x7f6768009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da263f06a15b78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f674f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768039060 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6768026030 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6768010b60 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6768040050 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f67791a8740 10 client.16248 fetch_fsmap learned FSMap version 28 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.269+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631bd39f7e0 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768026030 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f676802b420 con 0x5631bd329a30 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:23:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 20 client.16248 populate_metadata read hostname 'smithi045' 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd39fe70 0x5631bd3a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 4 client.16248 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631bd286f40 con 0x5631bd39fe70 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f676ffff640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd39fe70 0x5631bd3a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f674f7fe640 10 client.16248 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.270+0000 7f676ffff640 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd39fe70 0x5631bd3a2290 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801d40a20089e3c9 server_cookie=d31cacc4b2b5e756 in_seq=0 out_seq=0 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f674f7fe640 1 -- 172.21.15.45:0/3995649375 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5631bd286f40 con 0x5631bd39fe70 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f674f7fe640 10 client.16248 handle_command_reply: tid=0 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 client.16248 shutdown 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 msgr2=0x5631bd32c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6768010a20 tx=0x7f6768009d00 comp rx=0 tx=0).stop 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 shutdown_connections 2024-04-25T00:23:41.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5631bd39fe70 0x5631bd3a2290 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631bd329a30 0x5631bd32c9e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631bd31ad10 0x5631bd3985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f674f7fe640 0 client.16248 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 --2- 172.21.15.45:0/3995649375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631bd31a360 0x5631bd3980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 >> 172.21.15.45:0/3995649375 conn(0x5631bd316950 msgr2=0x5631bd326f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 shutdown_connections 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 1 -- 172.21.15.45:0/3995649375 wait complete. 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.271+0000 7f67791a8740 20 client.16248 trim_cache size 0 max 16384 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:23:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "85de2c98-5b93-4d1e-a9fe-a139c4998ab4": { 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "85de2c98-5b93-4d1e-a9fe-a139c4998ab4", 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:23:41.279 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 msgr2=0x7f67580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 0x7f67580a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6768005fd0 tx=0x7f6768015040 comp rx=0 tx=0).stop 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 msgr2=0x7f6770198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6760005940 tx=0x7f6760004270 comp rx=0 tx=0).stop 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 shutdown_connections 2024-04-25T00:23:41.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f67580a2d00 0x7f67580a51c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67701066b0 0x7f67701a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6770100fd0 0x7f6770199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 --2- 172.21.15.45:0/105857678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6770100610 0x7f6770198dd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 >> 172.21.15.45:0/105857678 conn(0x7f6770077740 msgr2=0x7f67700ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 shutdown_connections 2024-04-25T00:23:41.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:23:41.272+0000 7f67764a3640 1 -- 172.21.15.45:0/105857678 wait complete. 2024-04-25T00:23:41.282 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:41 smithi045 ceph-mon[29194]: pgmap v54: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-04-25T00:23:41.284 INFO:tasks.cephfs.filesystem:scrub status for tag:85de2c98-5b93-4d1e-a9fe-a139c4998ab4 - {'path': '/', 'tag': '85de2c98-5b93-4d1e-a9fe-a139c4998ab4', 'options': 'recursive,force'} 2024-04-25T00:23:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:41 smithi122 ceph-mon[32959]: pgmap v54: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-04-25T00:23:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:41 smithi071 ceph-mon[32667]: pgmap v54: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-04-25T00:23:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:42 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:23:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:42 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:42 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:23:42.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:42 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:42 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:23:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:42 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:42 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:23:42.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:42 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:42.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:42 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:23:42.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:42 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:42.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:42 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:23:42.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:42 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:43 smithi122 ceph-mon[32959]: pgmap v55: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-04-25T00:23:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:43 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:23:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:43 smithi071 ceph-mon[32667]: pgmap v55: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-04-25T00:23:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:43 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:23:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:43 smithi045 ceph-mon[29194]: pgmap v55: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-04-25T00:23:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:43 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:23:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:45 smithi122 ceph-mon[32959]: pgmap v56: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 20 op/s 2024-04-25T00:23:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:45 smithi071 ceph-mon[32667]: pgmap v56: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 20 op/s 2024-04-25T00:23:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:45 smithi045 ceph-mon[29194]: pgmap v56: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 20 op/s 2024-04-25T00:23:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:47 smithi122 ceph-mon[32959]: pgmap v57: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:47.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:47 smithi071 ceph-mon[32667]: pgmap v57: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:47 smithi045 ceph-mon[29194]: pgmap v57: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:49 smithi122 ceph-mon[32959]: pgmap v58: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:49 smithi071 ceph-mon[32667]: pgmap v58: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:49 smithi045 ceph-mon[29194]: pgmap v58: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:51.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:51 smithi122 ceph-mon[32959]: pgmap v59: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:51 smithi071 ceph-mon[32667]: pgmap v59: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:51 smithi045 ceph-mon[29194]: pgmap v59: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:53 smithi122 ceph-mon[32959]: pgmap v60: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:53 smithi071 ceph-mon[32667]: pgmap v60: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:53 smithi045 ceph-mon[29194]: pgmap v60: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:55.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:55 smithi122 ceph-mon[32959]: pgmap v61: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:55.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:55 smithi071 ceph-mon[32667]: pgmap v61: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:55 smithi045 ceph-mon[29194]: pgmap v61: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-04-25T00:23:57.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:57 smithi071 ceph-mon[32667]: pgmap v62: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:57 smithi045 ceph-mon[29194]: pgmap v62: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:57.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:57 smithi122 ceph-mon[32959]: pgmap v62: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:23:59 smithi071 ceph-mon[32667]: pgmap v63: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:23:59 smithi045 ceph-mon[29194]: pgmap v63: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:23:59.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:23:59 smithi122 ceph-mon[32959]: pgmap v63: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:24:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:01 smithi071 ceph-mon[32667]: pgmap v64: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-25T00:24:01.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:01 smithi045 ceph-mon[29194]: pgmap v64: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-25T00:24:01.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:01 smithi122 ceph-mon[32959]: pgmap v64: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-04-25T00:24:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:03 smithi071 ceph-mon[32667]: pgmap v65: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 776 KiB/s wr, 10 op/s 2024-04-25T00:24:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:03 smithi045 ceph-mon[29194]: pgmap v65: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 776 KiB/s wr, 10 op/s 2024-04-25T00:24:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:03 smithi122 ceph-mon[32959]: pgmap v65: 129 pgs: 129 active+clean; 227 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 776 KiB/s wr, 10 op/s 2024-04-25T00:24:04.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:04 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:04 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:04 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:05 smithi071 ceph-mon[32667]: pgmap v66: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 130 op/s 2024-04-25T00:24:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:05 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:05 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:05 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:24:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:05 smithi045 ceph-mon[29194]: pgmap v66: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 130 op/s 2024-04-25T00:24:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:05 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:05 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:05 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:24:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:05 smithi122 ceph-mon[32959]: pgmap v66: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 130 op/s 2024-04-25T00:24:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:05 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:05 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:05 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:24:06.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:06 smithi071 ceph-mon[32667]: pgmap v67: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:06 smithi045 ceph-mon[29194]: pgmap v67: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:06.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:06 smithi122 ceph-mon[32959]: pgmap v67: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:08.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:08 smithi122 ceph-mon[32959]: pgmap v68: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:08.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:08 smithi071 ceph-mon[32667]: pgmap v68: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:08.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:08 smithi045 ceph-mon[29194]: pgmap v68: 129 pgs: 129 active+clean; 260 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 125 op/s 2024-04-25T00:24:10.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:10 smithi122 ceph-mon[32959]: pgmap v69: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 345 op/s 2024-04-25T00:24:10.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:10 smithi071 ceph-mon[32667]: pgmap v69: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 345 op/s 2024-04-25T00:24:10.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:10 smithi045 ceph-mon[29194]: pgmap v69: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 345 op/s 2024-04-25T00:24:11.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:11.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:11.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 -- 172.21.15.45:0/2426759507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b81066b0 msgr2=0x7f95b8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/2426759507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b81066b0 0x7f95b8106a90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009900 tx=0x7f95b402a410 comp rx=0 tx=0).stop 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 -- 172.21.15.45:0/2426759507 shutdown_connections 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/2426759507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b8100fd0 0x7f95b810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/2426759507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/2426759507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b81066b0 0x7f95b8106a90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 -- 172.21.15.45:0/2426759507 >> 172.21.15.45:0/2426759507 conn(0x7f95b8077740 msgr2=0x7f95b8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 -- 172.21.15.45:0/2426759507 shutdown_connections 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.414+0000 7f95bf6f5640 1 -- 172.21.15.45:0/2426759507 wait complete. 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 Processor -- start 2024-04-25T00:24:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 -- start start 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 0x7f95b8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 0x7f95b819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81109f0 con 0x7f95b81066b0 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8110870 con 0x7f95b8100fd0 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.415+0000 7f95bf6f5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8110b70 con 0x7f95b8100610 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49546/0 (socket says 172.21.15.45:49546) 2024-04-25T00:24:11.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 -- 172.21.15.45:0/3576385660 learned_addr learned my addr 172.21.15.45:0/3576385660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bdc92640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 0x7f95b819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bcc90640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 0x7f95b8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 -- 172.21.15.45:0/3576385660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 msgr2=0x7f95b8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 0x7f95b8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 -- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 msgr2=0x7f95b819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 0x7f95b819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 -- 172.21.15.45:0/3576385660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b819cfc0 con 0x7f95b8100610 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bdc92640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 0x7f95b819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bcc90640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 0x7f95b8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.416+0000 7f95bd491640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95b4005940 tx=0x7f95b4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.417+0000 7f95ae7fc640 1 -- 172.21.15.45:0/3576385660 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4050070 con 0x7f95b8100610 2024-04-25T00:24:11.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.417+0000 7f95ae7fc640 1 -- 172.21.15.45:0/3576385660 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b4031c30 con 0x7f95b8100610 2024-04-25T00:24:11.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.417+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b819d250 con 0x7f95b8100610 2024-04-25T00:24:11.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.417+0000 7f95ae7fc640 1 -- 172.21.15.45:0/3576385660 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4035030 con 0x7f95b8100610 2024-04-25T00:24:11.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.417+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b806b160 con 0x7f95b8100610 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95ae7fc640 1 -- 172.21.15.45:0/3576385660 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95b4053070 con 0x7f95b8100610 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95c2404740 1 Processor -- start 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95c2404740 1 -- start start 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95c2404740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95c2404740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.420+0000 7f95c2404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95c2404740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba243296b0 con 0x55ba243c4f50 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95c2404740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ba242d1880 con 0x55ba243c4630 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95c2404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ba24325670 con 0x55ba243cf620 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95ae7fc640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 0x7f95a40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bdc92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bdc92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60548/0 (socket says 172.21.15.45:60548) 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bdc92640 1 -- 172.21.15.45:0/530384959 learned_addr learned my addr 172.21.15.45:0/530384959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bd491640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bdc92640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 0x7f95a40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 -- 172.21.15.45:0/530384959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 msgr2=0x55ba243c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 -- 172.21.15.45:0/530384959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 msgr2=0x55ba243cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bcc90640 1 -- 172.21.15.45:0/530384959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba241077d0 con 0x55ba243cf620 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95bdc92640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.421+0000 7f95ae7fc640 1 -- 172.21.15.45:0/3576385660 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f95b4100ec0 con 0x7f95b8100610 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.422+0000 7f95bcc90640 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243d1a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009fa0 tx=0x7f95a80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe51b42152e6219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.422+0000 7f95bdc92640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 0x7f95a40a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b0004ab0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.422+0000 7f95a37fe640 1 -- 172.21.15.45:0/530384959 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004030 con 0x55ba243cf620 2024-04-25T00:24:11.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.422+0000 7f95a37fe640 1 -- 172.21.15.45:0/530384959 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a80041d0 con 0x55ba243cf620 2024-04-25T00:24:11.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.422+0000 7f95a37fe640 1 -- 172.21.15.45:0/530384959 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a800f910 con 0x55ba243cf620 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- 172.21.15.45:0/530384959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 msgr2=0x55ba243d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243d1a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009fa0 tx=0x7f95a80118d0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- 172.21.15.45:0/530384959 shutdown_connections 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243d1a10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 --2- 172.21.15.45:0/530384959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- 172.21.15.45:0/530384959 >> 172.21.15.45:0/530384959 conn(0x55ba243c0950 msgr2=0x55ba243c0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- 172.21.15.45:0/530384959 shutdown_connections 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- 172.21.15.45:0/530384959 wait complete. 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 -- start start 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.423+0000 7f95c2404740 1 Processor -- start 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba243d2410 con 0x55ba243cf620 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ba242e4f50 con 0x55ba243c4630 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ba243d2590 con 0x55ba243c4f50 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95bdc92640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95bd491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95c2404740 10 client.0 _ll_register_callbacks cb 0x55ba24320000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49568/0 (socket says 172.21.15.45:49568) 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.424+0000 7f95bcc90640 1 -- 172.21.15.45:0/690166550 learned_addr learned my addr 172.21.15.45:0/690166550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 -- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 msgr2=0x55ba243cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 -- 172.21.15.45:0/690166550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 msgr2=0x55ba243cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 -- 172.21.15.45:0/690166550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2403d3f0 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95bcc90640 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95a800a010 tx=0x7f95a8005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6831d61f9f30071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8027070 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a80041d0 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f95a8004380 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.425+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95a8033050 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 10 client.25789 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba23ffd0f0 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a80041d0 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f95a8004bb0 con 0x55ba243c4f50 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 10 client.25789 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 20 client.25789 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:11.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba243dfd10 0x55ba243e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 4 client.25789 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba24336bf0 con 0x55ba243dfd10 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.426+0000 7f95bd491640 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba243dfd10 0x55ba243e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.427+0000 7f95a0ff9640 10 client.25789 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.427+0000 7f95bd491640 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba243dfd10 0x55ba243e2190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdaefec761137a server_cookie=aaf4ca742e5e9e14 in_seq=0 out_seq=0 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.427+0000 7f95a0ff9640 1 -- 172.21.15.45:0/690166550 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba24336bf0 con 0x55ba243dfd10 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.427+0000 7f95a0ff9640 10 client.25789 handle_command_reply: tid=0 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.427+0000 7f95c2404740 1 client.25789 shutdown 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 msgr2=0x55ba243ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95a800a010 tx=0x7f95a8005650 comp rx=0 tx=0).stop 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 shutdown_connections 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba243dfd10 0x55ba243e2190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243ccae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 --2- 172.21.15.45:0/690166550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95a0ff9640 0 client.25789 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:11.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 >> 172.21.15.45:0/690166550 conn(0x55ba243c0950 msgr2=0x55ba243c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 shutdown_connections 2024-04-25T00:24:11.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 1 -- 172.21.15.45:0/690166550 wait complete. 2024-04-25T00:24:11.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.428+0000 7f95c2404740 20 client.25789 trim_cache size 0 max 16384 2024-04-25T00:24:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 Processor -- start 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 -- start start 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8072f50 con 0x55ba243c4f50 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81a5c70 con 0x55ba243c4630 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95c2404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8073620 con 0x55ba243cf620 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bd491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45350/0 (socket says 172.21.15.45:45350) 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 -- 172.21.15.45:0/1428120706 learned_addr learned my addr 172.21.15.45:0/1428120706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bdc92640 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 -- 172.21.15.45:0/1428120706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 msgr2=0x55ba243c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 -- 172.21.15.45:0/1428120706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 msgr2=0x55ba243cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bcc90640 1 -- 172.21.15.45:0/1428120706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba243caaf0 con 0x55ba243c4f50 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.453+0000 7f95bd491640 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95bcc90640 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009380 tx=0x7f95a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726db7f7b8dcd34f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95a2ffd640 1 -- 172.21.15.45:0/1428120706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a803c070 con 0x55ba243c4f50 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95a2ffd640 1 -- 172.21.15.45:0/1428120706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a800fd30 con 0x55ba243c4f50 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95a2ffd640 1 -- 172.21.15.45:0/1428120706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8027040 con 0x55ba243c4f50 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95c2404740 1 -- 172.21.15.45:0/1428120706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 msgr2=0x55ba243db2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95c2404740 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009380 tx=0x7f95a800f690 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95c2404740 1 -- 172.21.15.45:0/1428120706 shutdown_connections 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.454+0000 7f95c2404740 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243cf620 0x55ba243c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243c4f50 0x55ba243db2d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 --2- 172.21.15.45:0/1428120706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- 172.21.15.45:0/1428120706 >> 172.21.15.45:0/1428120706 conn(0x55ba243c0950 msgr2=0x55ba243cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- 172.21.15.45:0/1428120706 shutdown_connections 2024-04-25T00:24:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- 172.21.15.45:0/1428120706 wait complete. 2024-04-25T00:24:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- start start 2024-04-25T00:24:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 Processor -- start 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81a55a0 con 0x55ba243cf620 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95b81a4ed0 con 0x55ba243c4630 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.455+0000 7f95c2404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8070f30 con 0x55ba243c4f50 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95c2404740 10 client.0 _ll_register_callbacks cb 0x7f95b40cce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bd491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bcc90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49582/0 (socket says 172.21.15.45:49582) 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bcc90640 1 -- 172.21.15.45:0/1887358491 learned_addr learned my addr 172.21.15.45:0/1887358491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 -- 172.21.15.45:0/1887358491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 msgr2=0x55ba243d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 -- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 msgr2=0x55ba243d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243d7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 -- 172.21.15.45:0/1887358491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba243d3570 con 0x55ba243cf620 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bcc90640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bd491640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95bdc92640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243d2d40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005fd0 tx=0x7f95b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85da17f7cb50e8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.456+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0004b20 con 0x55ba243cf620 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f95b0004cc0 con 0x55ba243cf620 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a800fd30 con 0x55ba243cf620 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95b0010ad0 con 0x55ba243cf620 2024-04-25T00:24:11.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95c2404740 10 client.16266 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba244453e0 con 0x55ba243cf620 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.457+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a800fd30 con 0x55ba243cf620 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f95b002c470 con 0x55ba243cf620 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 20 client.16266 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba24445be0 0x55ba24448000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 4 client.16266 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba24330f40 con 0x55ba24445be0 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.458+0000 7f95bd491640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba24445be0 0x55ba24448000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95bd491640 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba24445be0 0x55ba24448000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f98ce3db3b66e85 server_cookie=c9170b46eb71f648 in_seq=0 out_seq=0 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95a0ff9640 10 client.16266 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95a0ff9640 1 -- 172.21.15.45:0/1887358491 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba24330f40 con 0x55ba24445be0 2024-04-25T00:24:11.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95a0ff9640 10 client.16266 handle_command_reply: tid=0 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 client.16266 shutdown 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 msgr2=0x55ba243d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243d2d40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f95b0005fd0 tx=0x7f95b0004a40 comp rx=0 tx=0).stop 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 shutdown_connections 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55ba24445be0 0x55ba24448000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba243cf620 0x55ba243d2d40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ba243c4f50 0x55ba243d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 --2- 172.21.15.45:0/1887358491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba243c4630 0x55ba243d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 >> 172.21.15.45:0/1887358491 conn(0x55ba243c0950 msgr2=0x55ba243bf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95a0ff9640 0 client.16266 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 shutdown_connections 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.459+0000 7f95c2404740 1 -- 172.21.15.45:0/1887358491 wait complete. 2024-04-25T00:24:11.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95c2404740 20 client.16266 trim_cache size 0 max 16384 2024-04-25T00:24:11.466 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:11.466 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:24:11.466 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:24:11.466 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 msgr2=0x7f95a40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 0x7f95a40a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b80 tx=0x7f95b0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 msgr2=0x7f95b8198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95b4005940 tx=0x7f95b4003c10 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 shutdown_connections 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95a40a2d00 0x7f95a40a51c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95b81066b0 0x7f95b819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8100fd0 0x7f95b8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 --2- 172.21.15.45:0/3576385660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8100610 0x7f95b8198020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 >> 172.21.15.45:0/3576385660 conn(0x7f95b8077740 msgr2=0x7f95b80ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 shutdown_connections 2024-04-25T00:24:11.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.460+0000 7f95bf6f5640 1 -- 172.21.15.45:0/3576385660 wait complete. 2024-04-25T00:24:11.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:24:11.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:24:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 -- 172.21.15.45:0/861377164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100620 msgr2=0x7fa914100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 --2- 172.21.15.45:0/861377164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100620 0x7fa914100aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa908009900 tx=0x7fa90802a410 comp rx=0 tx=0).stop 2024-04-25T00:24:11.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 -- 172.21.15.45:0/861377164 shutdown_connections 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 --2- 172.21.15.45:0/861377164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100fe0 0x7fa91410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 --2- 172.21.15.45:0/861377164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100620 0x7fa914100aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 --2- 172.21.15.45:0/861377164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 0x7fa914106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 -- 172.21.15.45:0/861377164 >> 172.21.15.45:0/861377164 conn(0x7fa914077740 msgr2=0x7fa914077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 -- 172.21.15.45:0/861377164 shutdown_connections 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.601+0000 7fa9194da640 1 -- 172.21.15.45:0/861377164 wait complete. 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.602+0000 7fa9194da640 1 Processor -- start 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.602+0000 7fa9194da640 1 -- start start 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.602+0000 7fa9194da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 0x7fa914198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9194da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9194da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 0x7fa91419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9194da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa914110a40 con 0x7fa914106630 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9194da640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141108c0 con 0x7fa914100fe0 2024-04-25T00:24:11.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9194da640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa914110bc0 con 0x7fa914100620 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 0x7fa914198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60578/0 (socket says 172.21.15.45:60578) 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa912ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 0x7fa914198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49612/0 (socket says 172.21.15.45:49612) 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 -- 172.21.15.45:0/2055548735 learned_addr learned my addr 172.21.15.45:0/2055548735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9137fe640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 0x7fa91419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 -- 172.21.15.45:0/2055548735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 msgr2=0x7fa914198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 0x7fa914198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 -- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 msgr2=0x7fa91419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 0x7fa91419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 -- 172.21.15.45:0/2055548735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419d080 con 0x7fa914100fe0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.603+0000 7fa9127fc640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa908003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.604+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90803f070 con 0x7fa914100fe0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.604+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908031c30 con 0x7fa914100fe0 2024-04-25T00:24:11.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.604+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908035030 con 0x7fa914100fe0 2024-04-25T00:24:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.604+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419d310 con 0x7fa914100fe0 2024-04-25T00:24:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.604+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a4c00 con 0x7fa914100fe0 2024-04-25T00:24:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.605+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa914068560 con 0x7fa914100fe0 2024-04-25T00:24:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.606+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa908042070 con 0x7fa914100fe0 2024-04-25T00:24:11.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.607+0000 7fa8fbfff640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 0x7fa8fc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.607+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa9080efdc0 con 0x7fa914100fe0 2024-04-25T00:24:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.607+0000 7fa912ffd640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 0x7fa8fc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:11.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.608+0000 7fa912ffd640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 0x7fa8fc0a5240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9040095f0 tx=0x7fa904005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:11.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.612+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa908031de0 con 0x7fa914100fe0 2024-04-25T00:24:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.910+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa914109bc0 con 0x7fa914100fe0 2024-04-25T00:24:11.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.911+0000 7fa8fbfff640 1 -- 172.21.15.45:0/2055548735 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fa9080bbfd0 con 0x7fa914100fe0 2024-04-25T00:24:11.912 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:24:11.912 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:24:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:24:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 msgr2=0x7fa8fc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 0x7fa8fc0a5240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa9040095f0 tx=0x7fa904005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 msgr2=0x7fa9141985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa908003c10 comp rx=0 tx=0).stop 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 shutdown_connections 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa8fc0a2d80 0x7fa8fc0a5240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa914106630 0x7fa91419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa914100fe0 0x7fa9141985d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 --2- 172.21.15.45:0/2055548735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa914100620 0x7fa914198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.913+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 >> 172.21.15.45:0/2055548735 conn(0x7fa914077740 msgr2=0x7fa914068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.914+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 shutdown_connections 2024-04-25T00:24:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:11.914+0000 7fa9194da640 1 -- 172.21.15.45:0/2055548735 wait complete. 2024-04-25T00:24:11.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:24:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 -- 172.21.15.45:0/755821295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7998100fd0 msgr2=0x7f799810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 --2- 172.21.15.45:0/755821295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7998100fd0 0x7f799810d4e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7990009cd0 tx=0x7f799002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 -- 172.21.15.45:0/755821295 shutdown_connections 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 --2- 172.21.15.45:0/755821295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7998100fd0 0x7f799810d4e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 --2- 172.21.15.45:0/755821295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100610 0x7f7998100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 --2- 172.21.15.45:0/755821295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79981066b0 0x7f7998106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 -- 172.21.15.45:0/755821295 >> 172.21.15.45:0/755821295 conn(0x7f7998077740 msgr2=0x7f7998077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 -- 172.21.15.45:0/755821295 shutdown_connections 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.051+0000 7f799e8d4640 1 -- 172.21.15.45:0/755821295 wait complete. 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 Processor -- start 2024-04-25T00:24:12.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 -- start start 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 0x7f799819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79981066b0 0x7f799819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110a30 con 0x7f79981066b0 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79981108b0 con 0x7f7998100610 2024-04-25T00:24:12.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.052+0000 7f799e8d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110bb0 con 0x7f7998100fd0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49622/0 (socket says 172.21.15.45:49622) 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 -- 172.21.15.45:0/1233276324 learned_addr learned my addr 172.21.15.45:0/1233276324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f7997fff640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 0x7f799819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f799ce71640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79981066b0 0x7f799819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 -- 172.21.15.45:0/1233276324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 msgr2=0x7f799819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 0x7f799819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 -- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79981066b0 msgr2=0x7f799819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79981066b0 0x7f799819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f79977fe640 1 -- 172.21.15.45:0/1233276324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799819f170 con 0x7f7998100fd0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.053+0000 7f7997fff640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 0x7f799819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f79977fe640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9f0 tx=0x7f798c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f79957fa640 1 -- 172.21.15.45:0/1233276324 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00cd80 con 0x7f7998100fd0 2024-04-25T00:24:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f79957fa640 1 -- 172.21.15.45:0/1233276324 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c005500 con 0x7f7998100fd0 2024-04-25T00:24:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f79957fa640 1 -- 172.21.15.45:0/1233276324 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c010690 con 0x7f7998100fd0 2024-04-25T00:24:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799819f460 con 0x7f7998100fd0 2024-04-25T00:24:12.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.054+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981a6cf0 con 0x7f7998100fd0 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.056+0000 7f79957fa640 1 -- 172.21.15.45:0/1233276324 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f798c004f60 con 0x7f7998100fd0 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79a15e8740 1 Processor -- start 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79a15e8740 1 -- start start 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79957fa640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 0x7f79800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x557319380030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319380570 0x557319382960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.057+0000 7f79957fa640 1 -- 172.21.15.45:0/1233276324 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f798c0cfc30 con 0x7f7998100fd0 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f799ce71640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 0x7f79800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60606/0 (socket says 172.21.15.45:60606) 2024-04-25T00:24:12.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 -- 172.21.15.45:0/3776948246 learned_addr learned my addr 172.21.15.45:0/3776948246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5573192da670 con 0x557319375ea0 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557319282970 con 0x557319375580 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573192d6630 con 0x557319380570 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f79977fe640 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x557319380030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f799ce71640 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319380570 0x557319382960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 -- 172.21.15.45:0/3776948246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319380570 msgr2=0x557319382960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319380570 0x557319382960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 -- 172.21.15.45:0/3776948246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 msgr2=0x557319380030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x557319380030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 -- 172.21.15.45:0/3776948246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573192a6400 con 0x557319375580 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f7997fff640 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79880114c0 tx=0x7f7988011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e35630a6d36f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f799ce71640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 0x7f79800a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7990004ab0 tx=0x7f7990031750 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f797e7fc640 1 -- 172.21.15.45:0/3776948246 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988004020 con 0x557319375580 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.058+0000 7f797e7fc640 1 -- 172.21.15.45:0/3776948246 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988005500 con 0x557319375580 2024-04-25T00:24:12.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f797e7fc640 1 -- 172.21.15.45:0/3776948246 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988004d80 con 0x557319375580 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 msgr2=0x557319375960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f79880114c0 tx=0x7f7988011980 comp rx=0 tx=0).stop 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 shutdown_connections 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319380570 0x557319382960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x557319380030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3776948246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x557319375960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 >> 172.21.15.45:0/3776948246 conn(0x5573193718a0 msgr2=0x557319371c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 shutdown_connections 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.059+0000 7f79a15e8740 1 -- 172.21.15.45:0/3776948246 wait complete. 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 -- start start 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 Processor -- start 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 0x55731937d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 0x55731937d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557319383360 con 0x557319380570 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557319296050 con 0x557319375580 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.060+0000 7f79a15e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573193834e0 con 0x557319375ea0 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79a15e8740 10 client.0 _ll_register_callbacks cb 0x557318fedbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 0x55731937d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f799ce71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 0x55731937d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60612/0 (socket says 172.21.15.45:60612) 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 0x55731937d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49632/0 (socket says 172.21.15.45:49632) 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 -- 172.21.15.45:0/1024245051 learned_addr learned my addr 172.21.15.45:0/1024245051 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 -- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 msgr2=0x55731937d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 0x55731937d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 -- 172.21.15.45:0/1024245051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 msgr2=0x55731937d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 0x55731937d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 -- 172.21.15.45:0/1024245051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557319267dc0 con 0x557319375580 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f799ce71640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 0x55731937d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f7997fff640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7988009530 tx=0x7f7988005210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392c1ea11a6dfb7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79777fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988028020 con 0x557319375580 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988005500 con 0x557319375580 2024-04-25T00:24:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.061+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f798800f5c0 con 0x557319375580 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7988033050 con 0x557319375580 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 10 client.25826 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557318fee3f0 con 0x557319375580 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988005500 con 0x557319375580 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7988023470 con 0x557319375580 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 10 client.25826 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 10 client.25826 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 20 client.25826 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319399110 0x55731939b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 4 client.25826 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573192e7bb0 con 0x557319399110 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.062+0000 7f79977fe640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319399110 0x55731939b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.063+0000 7f79777fe640 10 client.25826 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.063+0000 7f79977fe640 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319399110 0x55731939b580 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6982957a523e57bc server_cookie=db440ef735586b72 in_seq=0 out_seq=0 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79777fe640 1 -- 172.21.15.45:0/1024245051 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573192e7bb0 con 0x557319399110 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79777fe640 10 client.25826 handle_command_reply: tid=0 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 client.25826 shutdown 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 msgr2=0x55731937fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7988009530 tx=0x7f7988005210 comp rx=0 tx=0).stop 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 shutdown_connections 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319399110 0x55731939b580 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319380570 0x55731937d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375ea0 0x55731937d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 --2- 172.21.15.45:0/1024245051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937fe40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 >> 172.21.15.45:0/1024245051 conn(0x5573193718a0 msgr2=0x5573193776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79777fe640 0 client.25826 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 shutdown_connections 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 1 -- 172.21.15.45:0/1024245051 wait complete. 2024-04-25T00:24:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.064+0000 7f79a15e8740 20 client.25826 trim_cache size 0 max 16384 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 Processor -- start 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 -- start start 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x5573193a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79981ac910 con 0x557319375ea0 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79981ae450 con 0x557319375580 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.085+0000 7f79a15e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79981acfe0 con 0x5573193718a0 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f79977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f79977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60622/0 (socket says 172.21.15.45:60622) 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49638/0 (socket says 172.21.15.45:49638) 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f79977fe640 1 -- 172.21.15.45:0/3222744343 learned_addr learned my addr 172.21.15.45:0/3222744343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f799ce71640 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x5573193a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 -- 172.21.15.45:0/3222744343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 msgr2=0x55731937f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 -- 172.21.15.45:0/3222744343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 msgr2=0x5573193a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x5573193a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 -- 172.21.15.45:0/3222744343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731937c120 con 0x5573193718a0 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f79977fe640 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f7997fff640 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7988023910 tx=0x7f79880096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78de9909d8aa1f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.086+0000 7f797dffb640 1 -- 172.21.15.45:0/3222744343 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988028040 con 0x5573193718a0 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f797dffb640 1 -- 172.21.15.45:0/3222744343 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988023d00 con 0x5573193718a0 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f797dffb640 1 -- 172.21.15.45:0/3222744343 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798801d040 con 0x5573193718a0 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 -- 172.21.15.45:0/3222744343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 msgr2=0x55731937ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7988023910 tx=0x7f79880096c0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 -- 172.21.15.45:0/3222744343 shutdown_connections 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557319375ea0 0x5573193a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375580 0x55731937f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3222744343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573193718a0 0x55731937ba40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 -- 172.21.15.45:0/3222744343 >> 172.21.15.45:0/3222744343 conn(0x55731937d7d0 msgr2=0x557319378740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 -- 172.21.15.45:0/3222744343 shutdown_connections 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.087+0000 7f79a15e8740 1 -- 172.21.15.45:0/3222744343 wait complete. 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 -- start start 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 Processor -- start 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375ea0 0x55731938c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79981add80 con 0x5573193718a0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79981ad6b0 con 0x557319375ea0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79981aa950 con 0x557319375580 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79a15e8740 10 client.0 _ll_register_callbacks cb 0x7f798c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f7997fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f7997fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45440/0 (socket says 172.21.15.45:45440) 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f79977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49654/0 (socket says 172.21.15.45:49654) 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.088+0000 7f7997fff640 1 -- 172.21.15.45:0/3251929670 learned_addr learned my addr 172.21.15.45:0/3251929670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f799ce71640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375ea0 0x55731938c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 -- 172.21.15.45:0/3251929670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375ea0 msgr2=0x55731938c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375ea0 0x55731938c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 -- 172.21.15.45:0/3251929670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 msgr2=0x557319390f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 -- 172.21.15.45:0/3251929670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573190b87d0 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f7997fff640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79977fe640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f798c009380 tx=0x7f798c004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f44b5fa72fedd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79777fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0b8030 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f798c098400 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988023d00 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f798c09a150 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.089+0000 7f79a15e8740 10 client.25819 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573193ff0f0 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798c0046c0 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f798c09b1e0 con 0x557319375580 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 10 client.25819 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 10 client.25819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 20 client.25819 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319380570 0x5573194018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 4 client.25819 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7988023d00 con 0x557319380570 2024-04-25T00:24:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.090+0000 7f7997fff640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319380570 0x5573194018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f7997fff640 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319380570 0x5573194018b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8afe6141594efb8d server_cookie=22ef88aaac2d67eb in_seq=0 out_seq=0 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79777fe640 10 client.25819 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79777fe640 1 -- 172.21.15.45:0/3251929670 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7988016020 con 0x557319380570 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79777fe640 10 client.25819 handle_command_reply: tid=0 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 client.25819 shutdown 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 msgr2=0x557319391490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f798c009380 tx=0x7f798c004b60 comp rx=0 tx=0).stop 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 shutdown_connections 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x557319380570 0x5573194018b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.091+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557319375ea0 0x55731938c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79777fe640 0 client.25819 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557319375580 0x557319391490 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 1 --2- 172.21.15.45:0/3251929670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573193718a0 0x557319390f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 >> 172.21.15.45:0/3251929670 conn(0x55731937d7d0 msgr2=0x557319370b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 shutdown_connections 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 1 -- 172.21.15.45:0/3251929670 wait complete. 2024-04-25T00:24:12.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f79a15e8740 20 client.25819 trim_cache size 0 max 16384 2024-04-25T00:24:12.099 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 msgr2=0x7f79800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 0x7f79800a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7990004ab0 tx=0x7f7990031750 comp rx=0 tx=0).stop 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 msgr2=0x7f799819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.092+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e9f0 tx=0x7f798c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 shutdown_connections 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f79800a2d00 0x7f79800a51c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79981066b0 0x7f799819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7998100fd0 0x7f799819a6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 --2- 172.21.15.45:0/1233276324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7998100610 0x7f799819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 >> 172.21.15.45:0/1233276324 conn(0x7f7998077740 msgr2=0x7f7998068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 shutdown_connections 2024-04-25T00:24:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.093+0000 7f799e8d4640 1 -- 172.21.15.45:0/1233276324 wait complete. 2024-04-25T00:24:12.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:24:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.233+0000 7f38948e1640 1 -- 172.21.15.45:0/3127112177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3890109920 msgr2=0x7f3890109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.233+0000 7f38948e1640 1 --2- 172.21.15.45:0/3127112177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3890109920 0x7f3890109d00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3880009890 tx=0x7f388002a360 comp rx=0 tx=0).stop 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.233+0000 7f38948e1640 1 -- 172.21.15.45:0/3127112177 shutdown_connections 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.233+0000 7f38948e1640 1 --2- 172.21.15.45:0/3127112177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38901042d0 0x7f38901107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.233+0000 7f38948e1640 1 --2- 172.21.15.45:0/3127112177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f3890103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 --2- 172.21.15.45:0/3127112177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3890109920 0x7f3890109d00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 -- 172.21.15.45:0/3127112177 >> 172.21.15.45:0/3127112177 conn(0x7f38900ff3c0 msgr2=0x7f38901017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 -- 172.21.15.45:0/3127112177 shutdown_connections 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 -- 172.21.15.45:0/3127112177 wait complete. 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 Processor -- start 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.234+0000 7f38948e1640 1 -- start start 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38901042d0 0x7f389019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 0x7f389019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113d30 con 0x7f3890109920 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113bb0 con 0x7f3890103910 2024-04-25T00:24:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f38948e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3890113eb0 con 0x7f38901042d0 2024-04-25T00:24:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60646/0 (socket says 172.21.15.45:60646) 2024-04-25T00:24:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388e59c640 1 -- 172.21.15.45:0/3764237159 learned_addr learned my addr 172.21.15.45:0/3764237159 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 0x7f389019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f3887fff640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38901042d0 0x7f389019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 -- 172.21.15.45:0/3764237159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38901042d0 msgr2=0x7f389019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38901042d0 0x7f389019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 -- 172.21.15.45:0/3764237159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 msgr2=0x7f389019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388ed9d640 1 -- 172.21.15.45:0/3764237159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389019f280 con 0x7f3890109920 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.235+0000 7f388e59c640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 0x7f389019eb50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f388800efc0 tx=0x7f388800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f38877fe640 1 -- 172.21.15.45:0/3764237159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888003970 con 0x7f3890109920 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f38877fe640 1 -- 172.21.15.45:0/3764237159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3888004020 con 0x7f3890109920 2024-04-25T00:24:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389019f570 con 0x7f3890109920 2024-04-25T00:24:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f38877fe640 1 -- 172.21.15.45:0/3764237159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388800b9f0 con 0x7f3890109920 2024-04-25T00:24:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.236+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a6e00 con 0x7f3890109920 2024-04-25T00:24:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.239+0000 7f38975f4740 1 Processor -- start 2024-04-25T00:24:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38877fe640 1 -- 172.21.15.45:0/3764237159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f388801d020 con 0x7f3890109920 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 -- start start 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12580 0x5616f8e12960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5616f8d77670 con 0x5616f8e12580 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5616f8d1f970 con 0x5616f8e12ea0 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f38975f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5616f8d73630 con 0x5616f8e1d570 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f388e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f388e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60650/0 (socket says 172.21.15.45:60650) 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f388e59c640 1 -- 172.21.15.45:0/1779895213 learned_addr learned my addr 172.21.15.45:0/1779895213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f3887fff640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12580 0x5616f8e12960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.240+0000 7f3887fff640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 -- 172.21.15.45:0/1779895213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 msgr2=0x5616f8e1f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 -- 172.21.15.45:0/1779895213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12580 msgr2=0x5616f8e12960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12580 0x5616f8e12960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f38877fe640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 0x7f387c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 -- 172.21.15.45:0/1779895213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f8d43400 con 0x5616f8e12ea0 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f3887fff640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 0x7f387c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f38877fe640 1 -- 172.21.15.45:0/3764237159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f38880cf5b0 con 0x7f3890109920 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388ed9d640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 0x7f387c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38880ccfa0 tx=0x7f3888004860 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.241+0000 7f388e59c640 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3880005e00 tx=0x7f3880009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f142529f2d770891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.242+0000 7f3884ff9640 1 -- 172.21.15.45:0/1779895213 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880037040 con 0x5616f8e12ea0 2024-04-25T00:24:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.242+0000 7f3884ff9640 1 -- 172.21.15.45:0/1779895213 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880004230 con 0x5616f8e12ea0 2024-04-25T00:24:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.242+0000 7f3884ff9640 1 -- 172.21.15.45:0/1779895213 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880003db0 con 0x5616f8e12ea0 2024-04-25T00:24:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.242+0000 7f38975f4740 1 -- 172.21.15.45:0/1779895213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 msgr2=0x5616f8e1d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.242+0000 7f38975f4740 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3880005e00 tx=0x7f3880009520 comp rx=0 tx=0).stop 2024-04-25T00:24:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 -- 172.21.15.45:0/1779895213 shutdown_connections 2024-04-25T00:24:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12ea0 0x5616f8e1d030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 --2- 172.21.15.45:0/1779895213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12580 0x5616f8e12960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 -- 172.21.15.45:0/1779895213 >> 172.21.15.45:0/1779895213 conn(0x5616f8e0e8a0 msgr2=0x5616f8e0ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 -- 172.21.15.45:0/1779895213 shutdown_connections 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.243+0000 7f38975f4740 1 -- 172.21.15.45:0/1779895213 wait complete. 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 -- start start 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 Processor -- start 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 0x5616f8e1a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5616f8e20300 con 0x5616f8e12ea0 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5616f8d33050 con 0x5616f8e12580 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5616f8e20480 con 0x5616f8e1d570 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f38975f4740 10 client.0 _ll_register_callbacks cb 0x5616f8a8abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f388e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 0x5616f8e1a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60656/0 (socket says 172.21.15.45:60656) 2024-04-25T00:24:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 -- 172.21.15.45:0/2545803493 learned_addr learned my addr 172.21.15.45:0/2545803493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 -- 172.21.15.45:0/2545803493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 msgr2=0x5616f8e1a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 -- 172.21.15.45:0/2545803493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 msgr2=0x5616f8e1a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 0x5616f8e1a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f3887fff640 1 -- 172.21.15.45:0/2545803493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f8d04dc0 con 0x5616f8e12580 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.244+0000 7f388e59c640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 0x5616f8e1a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3887fff640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3878010fc0 tx=0x7f3878015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53fa74cb7062e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387801c040 con 0x5616f8e12580 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f3878005220 con 0x5616f8e12580 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3880004230 con 0x5616f8e12580 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f387802a050 con 0x5616f8e12580 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f38975f4740 10 client.25844 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616f8a8b3f0 con 0x5616f8e12580 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.245+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880004230 con 0x5616f8e12580 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38780223d0 con 0x5616f8e12580 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 10 client.25844 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 10 client.25844 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 20 client.25844 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e2f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 4 client.25844 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f8d84bb0 con 0x5616f8e2d590 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.246+0000 7f388e59c640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e2f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.247+0000 7f388e59c640 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e2f9e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8df130b573dd2c5 server_cookie=b41358aff3fc78c7 in_seq=0 out_seq=0 2024-04-25T00:24:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.247+0000 7f3875ffb640 10 client.25844 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.247+0000 7f3875ffb640 1 -- 172.21.15.45:0/2545803493 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616f8d84bb0 con 0x5616f8e2d590 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.247+0000 7f3875ffb640 10 client.25844 handle_command_reply: tid=0 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.247+0000 7f38975f4740 1 client.25844 shutdown 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 msgr2=0x5616f8e1cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3878010fc0 tx=0x7f3878015040 comp rx=0 tx=0).stop 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 shutdown_connections 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e2f9e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e1d570 0x5616f8e1a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e12ea0 0x5616f8e1a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f3875ffb640 0 client.25844 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 --2- 172.21.15.45:0/2545803493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e1cf10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 >> 172.21.15.45:0/2545803493 conn(0x5616f8e0e8a0 msgr2=0x5616f8e0db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 shutdown_connections 2024-04-25T00:24:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 1 -- 172.21.15.45:0/2545803493 wait complete. 2024-04-25T00:24:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.248+0000 7f38975f4740 20 client.25844 trim_cache size 0 max 16384 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 Processor -- start 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 -- start start 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b0d60 con 0x5616f8e1d570 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b28a0 con 0x5616f8e12580 2024-04-25T00:24:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.265+0000 7f38975f4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b1430 con 0x5616f8e12ea0 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60668/0 (socket says 172.21.15.45:60668) 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 -- 172.21.15.45:0/2233112604 learned_addr learned my addr 172.21.15.45:0/2233112604 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f388e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 -- 172.21.15.45:0/2233112604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 msgr2=0x5616f8e1ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e1ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 -- 172.21.15.45:0/2233112604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 msgr2=0x5616f8e352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 -- 172.21.15.45:0/2233112604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f8b557d0 con 0x5616f8e12580 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3887fff640 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38780099f0 tx=0x7f3878005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af12c271b4ef0a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.266+0000 7f3877fff640 1 -- 172.21.15.45:0/2233112604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878022990 con 0x5616f8e12580 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.267+0000 7f3877fff640 1 -- 172.21.15.45:0/2233112604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3878022b30 con 0x5616f8e12580 2024-04-25T00:24:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.267+0000 7f3877fff640 1 -- 172.21.15.45:0/2233112604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878023d80 con 0x5616f8e12580 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.267+0000 7f38975f4740 1 -- 172.21.15.45:0/2233112604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 msgr2=0x5616f8e29220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.267+0000 7f38975f4740 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38780099f0 tx=0x7f3878005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 -- 172.21.15.45:0/2233112604 shutdown_connections 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 --2- 172.21.15.45:0/2233112604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e29220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 -- 172.21.15.45:0/2233112604 >> 172.21.15.45:0/2233112604 conn(0x5616f8e0e8a0 msgr2=0x5616f8e145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 -- 172.21.15.45:0/2233112604 shutdown_connections 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 -- 172.21.15.45:0/2233112604 wait complete. 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 -- start start 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.268+0000 7f38975f4740 1 Processor -- start 2024-04-25T00:24:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e8c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60680/0 (socket says 172.21.15.45:60680) 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 -- 172.21.15.45:0/2286501298 learned_addr learned my addr 172.21.15.45:0/2286501298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f388e59c640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e8c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b21d0 con 0x5616f8e1d570 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b1b00 con 0x5616f8e12580 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38901aed40 con 0x5616f8e12ea0 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 10 client.0 _ll_register_callbacks cb 0x7f3878009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 -- 172.21.15.45:0/2286501298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 msgr2=0x5616f8e8c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e8c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 -- 172.21.15.45:0/2286501298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 msgr2=0x5616f8e208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f3887fff640 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f8e20f90 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.269+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3878022b30 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3887fff640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3878009b90 tx=0x7f38780239e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5675702d18a0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3875ffb640 10 client.25850 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3875ffb640 1 -- 172.21.15.45:0/2286501298 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878033070 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3875ffb640 1 -- 172.21.15.45:0/2286501298 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f3878023cc0 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3875ffb640 1 -- 172.21.15.45:0/2286501298 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3878022990 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 10 client.25850 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5616f8e933e0 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f3875ffb640 1 -- 172.21.15.45:0/2286501298 <== mon.2 v2:172.21.15.71:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3878023460 con 0x5616f8e12580 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 10 client.25850 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 10 client.25850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 20 client.25850 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e95ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 4 client.25850 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.270+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616f8d7ef00 con 0x5616f8e2d590 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f388e59c640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e95ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f388e59c640 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e95ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eddd6d2c10e2d8d8 server_cookie=d4bc09e93770c4b3 in_seq=0 out_seq=0 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f3875ffb640 10 client.25850 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f3875ffb640 1 -- 172.21.15.45:0/2286501298 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616f8d7ef00 con 0x5616f8e2d590 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f3875ffb640 10 client.25850 handle_command_reply: tid=0 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 client.25850 shutdown 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 msgr2=0x5616f8e8be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3878009b90 tx=0x7f38780239e0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 shutdown_connections 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5616f8e2d590 0x5616f8e95ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f8e1d570 0x5616f8e208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616f8e12ea0 0x5616f8e8c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f3875ffb640 0 client.25850 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 --2- 172.21.15.45:0/2286501298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f8e12580 0x5616f8e8be20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.271+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 >> 172.21.15.45:0/2286501298 conn(0x5616f8e0e8a0 msgr2=0x5616f8e14e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 shutdown_connections 2024-04-25T00:24:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38975f4740 1 -- 172.21.15.45:0/2286501298 wait complete. 2024-04-25T00:24:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38975f4740 20 client.25850 trim_cache size 0 max 16384 2024-04-25T00:24:12.277 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 msgr2=0x7f387c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 0x7f387c0a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38880ccfa0 tx=0x7f3888004860 comp rx=0 tx=0).stop 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 msgr2=0x7f389019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.272+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 0x7f389019eb50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f388800efc0 tx=0x7f388800c800 comp rx=0 tx=0).stop 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 shutdown_connections 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f387c0a2d00 0x7f387c0a51c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3890109920 0x7f389019eb50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38901042d0 0x7f389019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 --2- 172.21.15.45:0/3764237159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3890103910 0x7f389019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 >> 172.21.15.45:0/3764237159 conn(0x7f38900ff3c0 msgr2=0x7f38900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 shutdown_connections 2024-04-25T00:24:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.273+0000 7f38948e1640 1 -- 172.21.15.45:0/3764237159 wait complete. 2024-04-25T00:24:12.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:24:12.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 -- 172.21.15.45:0/3214221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 msgr2=0x7f992c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 --2- 172.21.15.45:0/3214221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c069f30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f991c009900 tx=0x7f991c02a410 comp rx=0 tx=0).stop 2024-04-25T00:24:12.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 -- 172.21.15.45:0/3214221 shutdown_connections 2024-04-25T00:24:12.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 --2- 172.21.15.45:0/3214221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 --2- 172.21.15.45:0/3214221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 --2- 172.21.15.45:0/3214221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c069f30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 -- 172.21.15.45:0/3214221 >> 172.21.15.45:0/3214221 conn(0x7f992c0776e0 msgr2=0x7f992c077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 -- 172.21.15.45:0/3214221 shutdown_connections 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.414+0000 7f9931509640 1 -- 172.21.15.45:0/3214221 wait complete. 2024-04-25T00:24:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 Processor -- start 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 -- start start 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113d10 con 0x7f992c069b50 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113b90 con 0x7f992c10e150 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f9931509640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f992c113e90 con 0x7f992c06a470 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45504/0 (socket says 172.21.15.45:45504) 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992affd640 1 -- 172.21.15.45:0/3685692753 learned_addr learned my addr 172.21.15.45:0/3685692753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60684/0 (socket says 172.21.15.45:60684) 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.415+0000 7f992a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49720/0 (socket says 172.21.15.45:49720) 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 -- 172.21.15.45:0/3685692753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 msgr2=0x7f992c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 -- 172.21.15.45:0/3685692753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 msgr2=0x7f992c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 -- 172.21.15.45:0/3685692753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c19f240 con 0x7f992c069b50 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f992affd640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f991c006010 tx=0x7f991c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f9913fff640 1 -- 172.21.15.45:0/3685692753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c03f030 con 0x7f992c069b50 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.416+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c19f4d0 con 0x7f992c069b50 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.417+0000 7f9913fff640 1 -- 172.21.15.45:0/3685692753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f991c004440 con 0x7f992c069b50 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.417+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1a6dc0 con 0x7f992c069b50 2024-04-25T00:24:12.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.417+0000 7f9913fff640 1 -- 172.21.15.45:0/3685692753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c049060 con 0x7f992c069b50 2024-04-25T00:24:12.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.419+0000 7f9913fff640 1 -- 172.21.15.45:0/3685692753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f991c042030 con 0x7f992c069b50 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 Processor -- start 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 -- start start 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a132960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f9913fff640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 0x7f99140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a13d570 0x55651a13f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55651a097670 con 0x55651a132ea0 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55651a03f970 con 0x55651a13d570 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f993420d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55651a093630 con 0x55651a132580 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992a7fc640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 0x7f99140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45506/0 (socket says 172.21.15.45:45506) 2024-04-25T00:24:12.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992affd640 1 -- 172.21.15.45:0/834359693 learned_addr learned my addr 172.21.15.45:0/834359693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f9913fff640 1 -- 172.21.15.45:0/3685692753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f991c0efaa0 con 0x7f992c069b50 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.420+0000 7f992b7fe640 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a13d570 0x55651a13f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 -- 172.21.15.45:0/834359693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a13d570 msgr2=0x55651a13f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a13d570 0x55651a13f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 -- 172.21.15.45:0/834359693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 msgr2=0x55651a13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 -- 172.21.15.45:0/834359693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651a063400 con 0x55651a132580 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992affd640 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992a7fc640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 0x7f99140a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f992c10eea0 tx=0x7f9920005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f992b7fe640 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a132960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770d5577bc98df0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f99117fa640 1 -- 172.21.15.45:0/834359693 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240131d0 con 0x55651a132580 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f99117fa640 1 -- 172.21.15.45:0/834359693 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992400ec90 con 0x55651a132580 2024-04-25T00:24:12.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.421+0000 7f99117fa640 1 -- 172.21.15.45:0/834359693 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924011af0 con 0x55651a132580 2024-04-25T00:24:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- 172.21.15.45:0/834359693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 msgr2=0x55651a132960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a132960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9924005fd0 tx=0x7f9924009500 comp rx=0 tx=0).stop 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- 172.21.15.45:0/834359693 shutdown_connections 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a13d570 0x55651a13f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a132ea0 0x55651a13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 --2- 172.21.15.45:0/834359693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a132960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- 172.21.15.45:0/834359693 >> 172.21.15.45:0/834359693 conn(0x55651a12e8a0 msgr2=0x55651a12ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- 172.21.15.45:0/834359693 shutdown_connections 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- 172.21.15.45:0/834359693 wait complete. 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.422+0000 7f993420d740 1 -- start start 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 Processor -- start 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 0x55651a13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55651a140360 con 0x55651a13d570 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55651a053050 con 0x55651a132580 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55651a1404e0 con 0x55651a132ea0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f993420d740 10 client.0 _ll_register_callbacks cb 0x556519daabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f992affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f992affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a13ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60694/0 (socket says 172.21.15.45:60694) 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f992affd640 1 -- 172.21.15.45:0/246763885 learned_addr learned my addr 172.21.15.45:0/246763885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f992a7fc640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.423+0000 7f992b7fe640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 0x55651a13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 -- 172.21.15.45:0/246763885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 msgr2=0x55651a13ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a13ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 -- 172.21.15.45:0/246763885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 msgr2=0x55651a13a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 0x55651a13a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 -- 172.21.15.45:0/246763885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a024dc0 con 0x55651a132ea0 2024-04-25T00:24:12.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992b7fe640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 0x55651a13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f992a7fc640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13a170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f99200118d0 tx=0x7f9920004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd120b6ce31da8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f990eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992000ecc0 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9920005020 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992400ec90 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f992002e050 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f993420d740 10 client.25849 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.424+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556519dab3f0 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99200047a0 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f992001e420 con 0x55651a132ea0 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 10 client.25849 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 10 client.25849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 20 client.25849 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a14da90 0x55651a14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 4 client.25849 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f992400ec90 con 0x55651a14da90 2024-04-25T00:24:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.425+0000 7f992affd640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a14da90 0x55651a14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.426+0000 7f990eff5640 10 client.25849 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:12.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.426+0000 7f992affd640 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a14da90 0x55651a14feb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=424049338c0bcdf1 server_cookie=f087be52fda5044 in_seq=0 out_seq=0 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f990eff5640 1 -- 172.21.15.45:0/246763885 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f992400ec90 con 0x55651a14da90 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f990eff5640 10 client.25849 handle_command_reply: tid=0 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 client.25849 shutdown 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 msgr2=0x55651a13a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13a170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f99200118d0 tx=0x7f9920004730 comp rx=0 tx=0).stop 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 shutdown_connections 2024-04-25T00:24:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a14da90 0x55651a14feb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a13d570 0x55651a13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13a170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 --2- 172.21.15.45:0/246763885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a13ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 >> 172.21.15.45:0/246763885 conn(0x55651a12e8a0 msgr2=0x55651a1346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f990eff5640 0 client.25849 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 shutdown_connections 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 1 -- 172.21.15.45:0/246763885 wait complete. 2024-04-25T00:24:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.427+0000 7f993420d740 20 client.25849 trim_cache size 0 max 16384 2024-04-25T00:24:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.445+0000 7f993420d740 1 Processor -- start 2024-04-25T00:24:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.445+0000 7f993420d740 1 -- start start 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a1355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b0d80 con 0x55651a12e8a0 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b28c0 con 0x55651a132580 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f993420d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b1450 con 0x55651a132ea0 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45522/0 (socket says 172.21.15.45:45522) 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992affd640 1 -- 172.21.15.45:0/2310515894 learned_addr learned my addr 172.21.15.45:0/2310515894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a1355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992b7fe640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 -- 172.21.15.45:0/2310515894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 msgr2=0x55651a13c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 -- 172.21.15.45:0/2310515894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 msgr2=0x55651a149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 -- 172.21.15.45:0/2310515894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556519e757d0 con 0x55651a132580 2024-04-25T00:24:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992affd640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.446+0000 7f992a7fc640 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a1355c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f992001e840 tx=0x7f9920005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a732ef536757a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f9910ff9640 1 -- 172.21.15.45:0/2310515894 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992001e420 con 0x55651a132580 2024-04-25T00:24:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f9910ff9640 1 -- 172.21.15.45:0/2310515894 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992002b050 con 0x55651a132580 2024-04-25T00:24:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f9910ff9640 1 -- 172.21.15.45:0/2310515894 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920005450 con 0x55651a132580 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 -- 172.21.15.45:0/2310515894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 msgr2=0x55651a1355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a1355c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f992001e840 tx=0x7f9920005c70 comp rx=0 tx=0).stop 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 -- 172.21.15.45:0/2310515894 shutdown_connections 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132ea0 0x55651a13c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132580 0x55651a1355c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 --2- 172.21.15.45:0/2310515894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.447+0000 7f993420d740 1 -- 172.21.15.45:0/2310515894 >> 172.21.15.45:0/2310515894 conn(0x55651a13a810 msgr2=0x55651a134eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- 172.21.15.45:0/2310515894 shutdown_connections 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- 172.21.15.45:0/2310515894 wait complete. 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- start start 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 Processor -- start 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a145a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a136a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 0x55651a136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b21f0 con 0x55651a12e8a0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1b1b20 con 0x55651a132ea0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f992c1aed60 con 0x55651a132580 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f993420d740 10 client.0 _ll_register_callbacks cb 0x7f9920009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.448+0000 7f992a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 0x55651a136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a136a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49758/0 (socket says 172.21.15.45:49758) 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45538/0 (socket says 172.21.15.45:45538) 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992a7fc640 1 -- 172.21.15.45:0/549324760 learned_addr learned my addr 172.21.15.45:0/549324760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 -- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 msgr2=0x55651a136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 -- 172.21.15.45:0/549324760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 msgr2=0x55651a145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 -- 172.21.15.45:0/549324760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a1376e0 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f992b7fe640 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 0x55651a136fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9924011670 tx=0x7f9924004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3655814275dd1fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f990eff5640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992400e980 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f992400eb20 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992002b050 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.449+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9924032050 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 10 client.25868 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55651a1b3f60 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992400ed60 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9924005280 con 0x55651a132ea0 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 10 client.25868 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 10 client.25868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:24:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 20 client.25868 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a13d570 0x55651a1b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 4 client.25868 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f992002b050 con 0x55651a13d570 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.450+0000 7f992affd640 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a13d570 0x55651a1b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f990eff5640 10 client.25868 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f992affd640 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a13d570 0x55651a1b6440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e32489cb5b1670ce server_cookie=fc06526f1606b762 in_seq=0 out_seq=0 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f990eff5640 1 -- 172.21.15.45:0/549324760 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f992002b050 con 0x55651a13d570 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f990eff5640 10 client.25868 handle_command_reply: tid=0 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 client.25868 shutdown 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 msgr2=0x55651a136fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 0x55651a136fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9924011670 tx=0x7f9924004ef0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 shutdown_connections 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55651a13d570 0x55651a1b6440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55651a132ea0 0x55651a136fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55651a132580 0x55651a136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 --2- 172.21.15.45:0/549324760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55651a12e8a0 0x55651a145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f990eff5640 0 client.25868 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 >> 172.21.15.45:0/549324760 conn(0x55651a13a810 msgr2=0x55651a134560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 shutdown_connections 2024-04-25T00:24:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 1 -- 172.21.15.45:0/549324760 wait complete. 2024-04-25T00:24:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.451+0000 7f993420d740 20 client.25868 trim_cache size 0 max 16384 2024-04-25T00:24:12.457 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 msgr2=0x7f99140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 0x7f99140a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f992c10eea0 tx=0x7f9920005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 msgr2=0x7f992c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f991c006010 tx=0x7f991c004040 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 shutdown_connections 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f99140a2d00 0x7f99140a51c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f992c10e150 0x7f992c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f992c06a470 0x7f992c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 --2- 172.21.15.45:0/3685692753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f992c069b50 0x7f992c19a250 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 >> 172.21.15.45:0/3685692753 conn(0x7f992c0776e0 msgr2=0x7f992c10f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 shutdown_connections 2024-04-25T00:24:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.452+0000 7f9931509640 1 -- 172.21.15.45:0/3685692753 wait complete. 2024-04-25T00:24:12.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:24:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.607+0000 7f7186dc1640 1 -- 172.21.15.45:0/3639021662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 msgr2=0x7f7180115460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.607+0000 7f7186dc1640 1 --2- 172.21.15.45:0/3639021662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f7180115460 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f717400a9a0 tx=0x7f717402b690 comp rx=0 tx=0).stop 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.608+0000 7f7186dc1640 1 -- 172.21.15.45:0/3639021662 shutdown_connections 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.608+0000 7f7186dc1640 1 --2- 172.21.15.45:0/3639021662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f7180115460 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.608+0000 7f7186dc1640 1 --2- 172.21.15.45:0/3639021662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180108fd0 0x7f7180109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.608+0000 7f7186dc1640 1 --2- 172.21.15.45:0/3639021662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 0x7f718010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.608+0000 7f7186dc1640 1 -- 172.21.15.45:0/3639021662 >> 172.21.15.45:0/3639021662 conn(0x7f718006c850 msgr2=0x7f718006cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.609+0000 7f7186dc1640 1 -- 172.21.15.45:0/3639021662 shutdown_connections 2024-04-25T00:24:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.609+0000 7f7186dc1640 1 -- 172.21.15.45:0/3639021662 wait complete. 2024-04-25T00:24:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.610+0000 7f7186dc1640 1 Processor -- start 2024-04-25T00:24:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.610+0000 7f7186dc1640 1 -- start start 2024-04-25T00:24:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180108fd0 0x7f71801ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 0x7f71801a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7180118130 con 0x7f7180109990 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7180117fb0 con 0x7f7180108fd0 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.611+0000 7f7186dc1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71801182b0 con 0x7f718010e6e0 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.612+0000 7f717ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.612+0000 7f717ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45546/0 (socket says 172.21.15.45:45546) 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.612+0000 7f717ffff640 1 -- 172.21.15.45:0/216848707 learned_addr learned my addr 172.21.15.45:0/216848707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f718535e640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 0x7f71801a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f717ffff640 1 -- 172.21.15.45:0/216848707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 msgr2=0x7f71801a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f717ffff640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 0x7f71801a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f717ffff640 1 -- 172.21.15.45:0/216848707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180108fd0 msgr2=0x7f71801ac800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:24:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f717ffff640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180108fd0 0x7f71801ac800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.613+0000 7f717ffff640 1 -- 172.21.15.45:0/216848707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71801a7130 con 0x7f7180109990 2024-04-25T00:24:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.614+0000 7f717ffff640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f717000d950 tx=0x7f717000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.615+0000 7f717dffb640 1 -- 172.21.15.45:0/216848707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170017070 con 0x7f7180109990 2024-04-25T00:24:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.615+0000 7f717dffb640 1 -- 172.21.15.45:0/216848707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7170009bb0 con 0x7f7180109990 2024-04-25T00:24:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.615+0000 7f717dffb640 1 -- 172.21.15.45:0/216848707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717000f600 con 0x7f7180109990 2024-04-25T00:24:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.615+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71801a7420 con 0x7f7180109990 2024-04-25T00:24:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.615+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801b36c0 con 0x7f7180109990 2024-04-25T00:24:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 Processor -- start 2024-04-25T00:24:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 -- start start 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d90407f570 0x55d904081960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d903fd9670 con 0x55d904074ea0 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d903f81970 con 0x55d904074580 2024-04-25T00:24:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.619+0000 7f7189baf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d903fd5630 con 0x55d90407f570 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60758/0 (socket says 172.21.15.45:60758) 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 -- 172.21.15.45:0/4039361442 learned_addr learned my addr 172.21.15.45:0/4039361442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f7184b5d640 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 -- 172.21.15.45:0/4039361442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d90407f570 msgr2=0x55d904081960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d90407f570 0x55d904081960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 -- 172.21.15.45:0/4039361442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 msgr2=0x55d90407f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f718535e640 1 -- 172.21.15.45:0/4039361442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d903fa5400 con 0x55d904074580 2024-04-25T00:24:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.620+0000 7f717dffb640 1 -- 172.21.15.45:0/216848707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7170010460 con 0x7f7180109990 2024-04-25T00:24:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.621+0000 7f718535e640 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7174005d00 tx=0x7f7174032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8815a5b1ea0e5296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.621+0000 7f7166ffd640 1 -- 172.21.15.45:0/4039361442 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174004450 con 0x55d904074580 2024-04-25T00:24:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.621+0000 7f7166ffd640 1 -- 172.21.15.45:0/4039361442 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174041070 con 0x55d904074580 2024-04-25T00:24:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.621+0000 7f7166ffd640 1 -- 172.21.15.45:0/4039361442 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174032a60 con 0x55d904074580 2024-04-25T00:24:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.621+0000 7f717dffb640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 0x7f71680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.622+0000 7f717dffb640 1 -- 172.21.15.45:0/216848707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f71700cf200 con 0x7f7180109990 2024-04-25T00:24:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.622+0000 7f7184b5d640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 0x7f71680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.622+0000 7f7189baf740 1 -- 172.21.15.45:0/4039361442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 msgr2=0x55d904074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.622+0000 7f7189baf740 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7174005d00 tx=0x7f7174032c30 comp rx=0 tx=0).stop 2024-04-25T00:24:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7184b5d640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 0x7f71680a53c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d904075b40 tx=0x7f71780095c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7189baf740 1 -- 172.21.15.45:0/4039361442 shutdown_connections 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7189baf740 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d90407f570 0x55d904081960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7189baf740 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7189baf740 1 --2- 172.21.15.45:0/4039361442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904074960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.623+0000 7f7189baf740 1 -- 172.21.15.45:0/4039361442 >> 172.21.15.45:0/4039361442 conn(0x55d9040708a0 msgr2=0x55d904070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.624+0000 7f7189baf740 1 -- 172.21.15.45:0/4039361442 shutdown_connections 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.624+0000 7f7189baf740 1 -- 172.21.15.45:0/4039361442 wait complete. 2024-04-25T00:24:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.625+0000 7f7189baf740 1 -- start start 2024-04-25T00:24:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.625+0000 7f7189baf740 1 Processor -- start 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.626+0000 7f7189baf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 0x55d90407efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.626+0000 7f7189baf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7184b5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 0x55d90407efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7184b5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 0x55d90407efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49788/0 (socket says 172.21.15.45:49788) 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7184b5d640 1 -- 172.21.15.45:0/2863440479 learned_addr learned my addr 172.21.15.45:0/2863440479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d904082300 con 0x55d904074ea0 2024-04-25T00:24:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d903f95050 con 0x55d90407f570 2024-04-25T00:24:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f718535e640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d904082480 con 0x55d904074580 2024-04-25T00:24:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.627+0000 7f7189baf740 10 client.0 _ll_register_callbacks cb 0x55d903cecbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.629+0000 7f717ffff640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.630+0000 7f717ffff640 1 -- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 msgr2=0x55d90407efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.630+0000 7f717ffff640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 0x55d90407efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.630+0000 7f717ffff640 1 -- 172.21.15.45:0/2863440479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 msgr2=0x55d90407c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.630+0000 7f717ffff640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.630+0000 7f717ffff640 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d903f66dc0 con 0x55d904074ea0 2024-04-25T00:24:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.631+0000 7f717ffff640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407c2b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71700cc590 tx=0x7f717009ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110cf917113718ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.631+0000 7f715ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.631+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170020040 con 0x55d904074ea0 2024-04-25T00:24:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.631+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7174041070 con 0x55d904074ea0 2024-04-25T00:24:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.632+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f71700cc810 con 0x55d904074ea0 2024-04-25T00:24:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.632+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71700cca10 con 0x55d904074ea0 2024-04-25T00:24:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.633+0000 7f7189baf740 10 client.16338 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.633+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d903ced3f0 con 0x55d904074ea0 2024-04-25T00:24:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71700a3fd0 con 0x55d904074ea0 2024-04-25T00:24:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71700ccce0 con 0x55d904074ea0 2024-04-25T00:24:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 10 client.16338 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 20 client.16338 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d90408fa50 0x55d904091e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 4 client.16338 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.634+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7174041070 con 0x55d90408fa50 2024-04-25T00:24:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.635+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d90408fa50 0x55d904091e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.637+0000 7f715ffff640 10 client.16338 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.637+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d90408fa50 0x55d904091e70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddf8383cae8aeccf server_cookie=647c35b39b640fb4 in_seq=0 out_seq=0 2024-04-25T00:24:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.639+0000 7f715ffff640 1 -- 172.21.15.45:0/2863440479 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7174041070 con 0x55d90408fa50 2024-04-25T00:24:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.639+0000 7f715ffff640 10 client.16338 handle_command_reply: tid=0 2024-04-25T00:24:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.640+0000 7f7189baf740 1 client.16338 shutdown 2024-04-25T00:24:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.644+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 msgr2=0x55d90407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.644+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407c2b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71700cc590 tx=0x7f717009ba60 comp rx=0 tx=0).stop 2024-04-25T00:24:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 shutdown_connections 2024-04-25T00:24:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d90408fa50 0x55d904091e70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074ea0 0x55d90407c2b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 --2- 172.21.15.45:0/2863440479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074580 0x55d90407efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 >> 172.21.15.45:0/2863440479 conn(0x55d9040708a0 msgr2=0x55d90406fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.648+0000 7f715ffff640 0 client.16338 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:12 smithi045 ceph-mon[29194]: pgmap v70: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 339 op/s 2024-04-25T00:24:12.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:12 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2055548735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.652+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 shutdown_connections 2024-04-25T00:24:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.652+0000 7f7189baf740 1 -- 172.21.15.45:0/2863440479 wait complete. 2024-04-25T00:24:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.653+0000 7f7189baf740 20 client.16338 trim_cache size 0 max 16384 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.681+0000 7f7189baf740 1 Processor -- start 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.681+0000 7f7189baf740 1 -- start start 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 0x55d9040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f716c00fe70 con 0x55d904074580 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f717ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f717ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49796/0 (socket says 172.21.15.45:49796) 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f717ffff640 1 -- 172.21.15.45:0/2550058115 learned_addr learned my addr 172.21.15.45:0/2550058115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f718535e640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0119b0 con 0x55d90407f570 2024-04-25T00:24:12.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.682+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f716c010540 con 0x55d904074ea0 2024-04-25T00:24:12.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 0x55d9040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 -- 172.21.15.45:0/2550058115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 msgr2=0x55d9040763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 -- 172.21.15.45:0/2550058115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 msgr2=0x55d90407eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 -- 172.21.15.45:0/2550058115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d904081cd0 con 0x55d904074580 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f717ffff640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f718535e640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.683+0000 7f7184b5d640 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 0x55d9040702c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7178005fd0 tx=0x7f7178018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfc18ab778245cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.684+0000 7f71667fc640 1 -- 172.21.15.45:0/2550058115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717801acc0 con 0x55d904074580 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.684+0000 7f71667fc640 1 -- 172.21.15.45:0/2550058115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f717802d070 con 0x55d904074580 2024-04-25T00:24:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.684+0000 7f71667fc640 1 -- 172.21.15.45:0/2550058115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178026c80 con 0x55d904074580 2024-04-25T00:24:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 msgr2=0x55d9040702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 0x55d9040702c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7178005fd0 tx=0x7f7178018000 comp rx=0 tx=0).stop 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 shutdown_connections 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d90407f570 0x55d90407eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d9040763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 --2- 172.21.15.45:0/2550058115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d904074580 0x55d9040702c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 >> 172.21.15.45:0/2550058115 conn(0x55d9040708a0 msgr2=0x55d904075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.685+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 shutdown_connections 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.686+0000 7f7189baf740 1 -- 172.21.15.45:0/2550058115 wait complete. 2024-04-25T00:24:12.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.686+0000 7f7189baf740 1 -- start start 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.686+0000 7f7189baf740 1 Processor -- start 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d904087ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d90407f570 0x55d9040789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0112e0 con 0x55d90407f570 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7184b5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7184b5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60798/0 (socket says 172.21.15.45:60798) 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7184b5d640 1 -- 172.21.15.45:0/143886046 learned_addr learned my addr 172.21.15.45:0/143886046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f716c010c10 con 0x55d904074580 2024-04-25T00:24:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.687+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f716c00de50 con 0x55d904074ea0 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f717ffff640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d904087ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7189baf740 10 client.0 _ll_register_callbacks cb 0x7f717000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 -- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 msgr2=0x55d904087ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d904087ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 -- 172.21.15.45:0/143886046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d90407f570 msgr2=0x55d9040789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d90407f570 0x55d9040789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 -- 172.21.15.45:0/143886046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d903db77d0 con 0x55d904074580 2024-04-25T00:24:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f717802d070 con 0x55d904074580 2024-04-25T00:24:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.688+0000 7f7184b5d640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7178016040 tx=0x7f717801acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58c6a8c151c92e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f715ffff640 10 client.25880 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f715ffff640 1 -- 172.21.15.45:0/143886046 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178040050 con 0x55d904074580 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f715ffff640 1 -- 172.21.15.45:0/143886046 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f71780273c0 con 0x55d904074580 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f715ffff640 1 -- 172.21.15.45:0/143886046 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7178036070 con 0x55d904074580 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f7189baf740 10 client.25880 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.689+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d9040f57c0 con 0x55d904074580 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f715ffff640 1 -- 172.21.15.45:0/143886046 <== mon.2 v2:172.21.15.71:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f717800b040 con 0x55d904074580 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 10 client.25880 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 10 client.25880 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 20 client.25880 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d9040f5c80 0x55d9040f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 4 client.25880 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d903fe6bb0 con 0x55d9040f5c80 2024-04-25T00:24:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.690+0000 7f717ffff640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d9040f5c80 0x55d9040f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.692+0000 7f717ffff640 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d9040f5c80 0x55d9040f8090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a895d94d69aa5b8 server_cookie=526728d29bb6df78 in_seq=0 out_seq=0 2024-04-25T00:24:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.692+0000 7f715ffff640 10 client.25880 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.693+0000 7f715ffff640 1 -- 172.21.15.45:0/143886046 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d903fe6bb0 con 0x55d9040f5c80 2024-04-25T00:24:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.693+0000 7f715ffff640 10 client.25880 handle_command_reply: tid=0 2024-04-25T00:24:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.693+0000 7f7189baf740 1 client.25880 shutdown 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.694+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 msgr2=0x55d904087760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.694+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7178016040 tx=0x7f717801acc0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 shutdown_connections 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d9040f5c80 0x55d9040f8090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f715ffff640 0 client.25880 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d90407f570 0x55d9040789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d904074ea0 0x55d904087ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 --2- 172.21.15.45:0/143886046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d904074580 0x55d904087760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 >> 172.21.15.45:0/143886046 conn(0x55d9040708a0 msgr2=0x55d904076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 shutdown_connections 2024-04-25T00:24:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 1 -- 172.21.15.45:0/143886046 wait complete. 2024-04-25T00:24:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.695+0000 7f7189baf740 20 client.25880 trim_cache size 0 max 16384 2024-04-25T00:24:12.697 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 msgr2=0x7f71680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 0x7f71680a53c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d904075b40 tx=0x7f71780095c0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 msgr2=0x7f71801acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f717000d950 tx=0x7f717000de10 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 shutdown_connections 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f71680a2f00 0x7f71680a53c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f718010e6e0 0x7f71801a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7180109990 0x7f71801acd40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 --2- 172.21.15.45:0/216848707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180108fd0 0x7f71801ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.697+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 >> 172.21.15.45:0/216848707 conn(0x7f718006c850 msgr2=0x7f718010f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.698+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 shutdown_connections 2024-04-25T00:24:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.699+0000 7f7186dc1640 1 -- 172.21.15.45:0/216848707 wait complete. 2024-04-25T00:24:12.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:24:12.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:12 smithi122 ceph-mon[32959]: pgmap v70: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 339 op/s 2024-04-25T00:24:12.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:12 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2055548735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:12 smithi071 ceph-mon[32667]: pgmap v70: 129 pgs: 129 active+clean; 260 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 339 op/s 2024-04-25T00:24:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:12 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2055548735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:12.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.898+0000 7fa0d12fc640 1 -- 172.21.15.45:0/275529164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 msgr2=0x7fa0cc107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.898+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/275529164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc107b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009930 tx=0x7fa0bc02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 -- 172.21.15.45:0/275529164 shutdown_connections 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/275529164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc06a410 0x7fa0cc10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/275529164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc069a50 0x7fa0cc069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/275529164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc107b70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 -- 172.21.15.45:0/275529164 >> 172.21.15.45:0/275529164 conn(0x7fa0cc0761c0 msgr2=0x7fa0cc0765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 -- 172.21.15.45:0/275529164 shutdown_connections 2024-04-25T00:24:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 -- 172.21.15.45:0/275529164 wait complete. 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.899+0000 7fa0d12fc640 1 Processor -- start 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 -- start start 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 0x7fa0cc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc06a410 0x7fa0cc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc111b30 con 0x7fa0cc069a50 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1119b0 con 0x7fa0cc107790 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0d12fc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc111cb0 con 0x7fa0cc06a410 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 0x7fa0cc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 0x7fa0cc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45616/0 (socket says 172.21.15.45:45616) 2024-04-25T00:24:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0caffd640 1 -- 172.21.15.45:0/3713500874 learned_addr learned my addr 172.21.15.45:0/3713500874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.900+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc06a410 0x7fa0cc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 -- 172.21.15.45:0/3713500874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc06a410 msgr2=0x7fa0cc19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc06a410 0x7fa0cc19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 -- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 msgr2=0x7fa0cc19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 0x7fa0cc19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 -- 172.21.15.45:0/3713500874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc19f1d0 con 0x7fa0cc107790 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc19eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009fd0 tx=0x7fa0c400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0b3fff640 1 -- 172.21.15.45:0/3713500874 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400ce20 con 0x7fa0cc107790 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0b3fff640 1 -- 172.21.15.45:0/3713500874 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c4005020 con 0x7fa0cc107790 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc19f4c0 con 0x7fa0cc107790 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0b3fff640 1 -- 172.21.15.45:0/3713500874 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4010720 con 0x7fa0cc107790 2024-04-25T00:24:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.901+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc1a6d50 con 0x7fa0cc107790 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0b3fff640 1 -- 172.21.15.45:0/3713500874 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa0c4005540 con 0x7fa0cc107790 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 Processor -- start 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 -- start start 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e57f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 0x55781e58a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e58c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55781e4e4670 con 0x55781e57f580 2024-04-25T00:24:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55781e48c970 con 0x55781e58a570 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0d4008740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55781e4e0630 con 0x55781e57fea0 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0b3fff640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 0x7fa0b40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e57f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 0x55781e58a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e57f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45630/0 (socket says 172.21.15.45:45630) 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/3629277211 learned_addr learned my addr 172.21.15.45:0/3629277211 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.905+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 0x55781e58a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49820/0 (socket says 172.21.15.45:49820) 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e58c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0b3fff640 1 -- 172.21.15.45:0/3713500874 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa0c40cfe50 con 0x7fa0cc107790 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 0x7fa0b40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/3629277211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 msgr2=0x55781e58a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 0x55781e58a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/3629277211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 msgr2=0x55781e57f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e57f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/3629277211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e4b0400 con 0x55781e58a570 2024-04-25T00:24:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e58c960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00127c0 tx=0x7fa0c0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debabd9bf105180c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0cb7fe640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 0x7fa0b40a52c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409bf40 tx=0x7fa0c40bc040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0b17fa640 1 -- 172.21.15.45:0/3629277211 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00120d0 con 0x55781e58a570 2024-04-25T00:24:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0b17fa640 1 -- 172.21.15.45:0/3629277211 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c000ec30 con 0x55781e58a570 2024-04-25T00:24:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.906+0000 7fa0b17fa640 1 -- 172.21.15.45:0/3629277211 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00108d0 con 0x55781e58a570 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.907+0000 7fa0d4008740 1 -- 172.21.15.45:0/3629277211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 msgr2=0x55781e58c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.907+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e58c960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00127c0 tx=0x7fa0c0004730 comp rx=0 tx=0).stop 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 -- 172.21.15.45:0/3629277211 shutdown_connections 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e58c960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57fea0 0x55781e58a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3629277211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e57f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 -- 172.21.15.45:0/3629277211 >> 172.21.15.45:0/3629277211 conn(0x55781e57b8a0 msgr2=0x55781e57bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 -- 172.21.15.45:0/3629277211 shutdown_connections 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 -- 172.21.15.45:0/3629277211 wait complete. 2024-04-25T00:24:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 -- start start 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.908+0000 7fa0d4008740 1 Processor -- start 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e589f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e58a570 0x55781e587840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e589f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e589f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49828/0 (socket says 172.21.15.45:49828) 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0caffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:60826/0 (socket says 172.21.15.45:60826) 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/2856851843 learned_addr learned my addr 172.21.15.45:0/2856851843 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.909+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55781e58d300 con 0x55781e58a570 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55781e4a0050 con 0x55781e57fea0 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55781e58d480 con 0x55781e57f580 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0d4008740 10 client.0 _ll_register_callbacks cb 0x55781e1f7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e58a570 0x55781e587840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0caffd640 1 -- 172.21.15.45:0/2856851843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 msgr2=0x55781e589f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0caffd640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e589f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0caffd640 1 -- 172.21.15.45:0/2856851843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e58a570 msgr2=0x55781e587840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0caffd640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e58a570 0x55781e587840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.910+0000 7fa0caffd640 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781e471dc0 con 0x55781e57fea0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0caffd640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55781e580b40 tx=0x7fa0bc004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa5d5b5224f2d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc035040 con 0x55781e57fea0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa0bc005450 con 0x55781e57fea0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c000ec30 con 0x55781e57fea0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc005650 con 0x55781e57fea0 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.911+0000 7fa0d4008740 10 client.25892 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55781e1f83f0 con 0x55781e57fea0 2024-04-25T00:24:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c000ec30 con 0x55781e57fea0 2024-04-25T00:24:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa0bc003b00 con 0x55781e57fea0 2024-04-25T00:24:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0d4008740 10 client.25892 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0d4008740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:12.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.912+0000 7fa0d4008740 20 client.25892 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.913+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e59a7e0 0x55781e59cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.913+0000 7fa0d4008740 4 client.25892 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.913+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55781e4f1bb0 con 0x55781e59a7e0 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.913+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e59a7e0 0x55781e59cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.914+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e59a7e0 0x55781e59cbd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cfd04913a9dfc6 server_cookie=5b657a6158f6a908 in_seq=0 out_seq=0 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.914+0000 7fa0aeff5640 10 client.25892 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0aeff5640 1 -- 172.21.15.45:0/2856851843 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55781e4f1bb0 con 0x55781e59a7e0 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0aeff5640 10 client.25892 handle_command_reply: tid=0 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 client.25892 shutdown 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 msgr2=0x55781e587230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55781e580b40 tx=0x7fa0bc004440 comp rx=0 tx=0).stop 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 shutdown_connections 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e59a7e0 0x55781e59cbd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e58a570 0x55781e587840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e587230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0aeff5640 0 client.25892 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 --2- 172.21.15.45:0/2856851843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e589f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.915+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 >> 172.21.15.45:0/2856851843 conn(0x55781e57b8a0 msgr2=0x55781e57ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.916+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 shutdown_connections 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.916+0000 7fa0d4008740 1 -- 172.21.15.45:0/2856851843 wait complete. 2024-04-25T00:24:12.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.916+0000 7fa0d4008740 20 client.25892 trim_cache size 0 max 16384 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 Processor -- start 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 -- start start 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57fea0 0x55781e589a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e5a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1b0dd0 con 0x55781e57fea0 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1b2910 con 0x55781e58a570 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0d4008740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1b14a0 con 0x55781e57f580 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:49832/0 (socket says 172.21.15.45:49832) 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/1681197339 learned_addr learned my addr 172.21.15.45:0/1681197339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0caffd640 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57fea0 0x55781e589a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.933+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e5a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/1681197339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 msgr2=0x55781e5a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e5a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/1681197339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57fea0 msgr2=0x55781e589a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57fea0 0x55781e589a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/1681197339 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e2c27d0 con 0x55781e57f580 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0010700 tx=0x7fa0c0021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921f8cf8e2448c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0b0ff9640 1 -- 172.21.15.45:0/1681197339 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0028040 con 0x55781e57f580 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0b0ff9640 1 -- 172.21.15.45:0/1681197339 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0029040 con 0x55781e57f580 2024-04-25T00:24:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.934+0000 7fa0b0ff9640 1 -- 172.21.15.45:0/1681197339 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0005020 con 0x55781e57f580 2024-04-25T00:24:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- 172.21.15.45:0/1681197339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 msgr2=0x55781e5961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0010700 tx=0x7fa0c0021040 comp rx=0 tx=0).stop 2024-04-25T00:24:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- 172.21.15.45:0/1681197339 shutdown_connections 2024-04-25T00:24:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e58a570 0x55781e5a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57fea0 0x55781e589a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 --2- 172.21.15.45:0/1681197339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e57f580 0x55781e5961f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- 172.21.15.45:0/1681197339 >> 172.21.15.45:0/1681197339 conn(0x55781e57b8a0 msgr2=0x55781e581560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- 172.21.15.45:0/1681197339 shutdown_connections 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- 172.21.15.45:0/1681197339 wait complete. 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.935+0000 7fa0d4008740 1 -- start start 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 Processor -- start 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e58e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e58a570 0x55781e58ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1b2240 con 0x55781e57f580 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1b1b70 con 0x55781e57fea0 2024-04-25T00:24:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1aedb0 con 0x55781e58a570 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0d4008740 10 client.0 _ll_register_callbacks cb 0x7fa0bc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0ca7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45666/0 (socket says 172.21.15.45:45666) 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0ca7fc640 1 -- 172.21.15.45:0/3119164764 learned_addr learned my addr 172.21.15.45:0/3119164764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e58e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e58a570 0x55781e58ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 -- 172.21.15.45:0/3119164764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e58a570 msgr2=0x55781e58ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e58a570 0x55781e58ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 -- 172.21.15.45:0/3119164764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 msgr2=0x55781e58e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0caffd640 1 -- 172.21.15.45:0/3119164764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781e5ffc50 con 0x55781e57fea0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.936+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.937+0000 7fa0caffd640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e58e7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009850 tx=0x7fa0bc031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5476663780894fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.937+0000 7fa0aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.937+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc035070 con 0x55781e57fea0 2024-04-25T00:24:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.937+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa0bc0057f0 con 0x55781e57fea0 2024-04-25T00:24:12.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.937+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c0029040 con 0x55781e57fea0 2024-04-25T00:24:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.938+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc065070 con 0x55781e57fea0 2024-04-25T00:24:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.938+0000 7fa0d4008740 10 client.25898 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.938+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55781e600f30 con 0x55781e57fea0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.939+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0029040 con 0x55781e57fea0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.939+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa0bc003a30 con 0x55781e57fea0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 10 client.25898 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 20 client.25898 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e601650 0x55781e603a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 4 client.25898 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55781e4ebf00 con 0x55781e601650 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e601650 0x55781e603a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.940+0000 7fa0aeff5640 10 client.25898 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0ca7fc640 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e601650 0x55781e603a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31492230abd901f server_cookie=b96f5a9d9591956d in_seq=0 out_seq=0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0aeff5640 1 -- 172.21.15.45:0/3119164764 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55781e4ebf00 con 0x55781e601650 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0aeff5640 10 client.25898 handle_command_reply: tid=0 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0d4008740 1 client.25898 shutdown 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 msgr2=0x55781e58e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.941+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e58e7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009850 tx=0x7fa0bc031400 comp rx=0 tx=0).stop 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 shutdown_connections 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55781e58a570 0x55781e58ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781e57fea0 0x55781e58e7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55781e601650 0x55781e603a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0aeff5640 0 client.25898 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 --2- 172.21.15.45:0/3119164764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55781e57f580 0x55781e58e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 >> 172.21.15.45:0/3119164764 conn(0x55781e57b8a0 msgr2=0x55781e581d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 shutdown_connections 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 1 -- 172.21.15.45:0/3119164764 wait complete. 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.942+0000 7fa0d4008740 20 client.25898 trim_cache size 0 max 16384 2024-04-25T00:24:12.945 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 msgr2=0x7fa0b40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 0x7fa0b40a52c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409bf40 tx=0x7fa0c40bc040 comp rx=0 tx=0).stop 2024-04-25T00:24:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 msgr2=0x7fa0cc19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc19eaa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009fd0 tx=0x7fa0c400ef10 comp rx=0 tx=0).stop 2024-04-25T00:24:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 shutdown_connections 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa0b40a2e00 0x7fa0b40a52c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc107790 0x7fa0cc19eaa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc06a410 0x7fa0cc19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 --2- 172.21.15.45:0/3713500874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0cc069a50 0x7fa0cc19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 >> 172.21.15.45:0/3713500874 conn(0x7fa0cc0761c0 msgr2=0x7fa0cc10c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 shutdown_connections 2024-04-25T00:24:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:12.943+0000 7fa0d12fc640 1 -- 172.21.15.45:0/3713500874 wait complete. 2024-04-25T00:24:13.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:24:13.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:24:13.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 -- 172.21.15.45:0/2043684386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc100610 msgr2=0x7f8fcc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/2043684386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc100610 0x7f8fcc100a90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009cd0 tx=0x7f8fc002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 -- 172.21.15.45:0/2043684386 shutdown_connections 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/2043684386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/2043684386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc100610 0x7f8fcc100a90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/2043684386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 -- 172.21.15.45:0/2043684386 >> 172.21.15.45:0/2043684386 conn(0x7f8fcc077740 msgr2=0x7f8fcc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 -- 172.21.15.45:0/2043684386 shutdown_connections 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 -- 172.21.15.45:0/2043684386 wait complete. 2024-04-25T00:24:14.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.069+0000 7f8fd1b30640 1 Processor -- start 2024-04-25T00:24:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 -- start start 2024-04-25T00:24:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc100610 0x7f8fcc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1109f0 con 0x7f8fcc1066b0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc110870 con 0x7f8fcc100fd0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fd1b30640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc110b70 con 0x7f8fcc100610 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37246/0 (socket says 172.21.15.45:37246) 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 -- 172.21.15.45:0/1463880962 learned_addr learned my addr 172.21.15.45:0/1463880962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc100610 0x7f8fcc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 -- 172.21.15.45:0/1463880962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc100610 msgr2=0x7f8fcc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc100610 0x7f8fcc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 -- 172.21.15.45:0/1463880962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 msgr2=0x7f8fcc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 -- 172.21.15.45:0/1463880962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc19cf90 con 0x7f8fcc1066b0 2024-04-25T00:24:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.070+0000 7f8fcbfff640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400e970 tx=0x7f8fc400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.071+0000 7f8fc8ff9640 1 -- 172.21.15.45:0/1463880962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400ccc0 con 0x7f8fcc1066b0 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.071+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc19d280 con 0x7f8fcc1066b0 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.071+0000 7f8fc8ff9640 1 -- 172.21.15.45:0/1463880962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc400ce60 con 0x7f8fcc1066b0 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.071+0000 7f8fc8ff9640 1 -- 172.21.15.45:0/1463880962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4010490 con 0x7f8fcc1066b0 2024-04-25T00:24:14.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.071+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc1a4b10 con 0x7f8fcc1066b0 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.073+0000 7f8fc8ff9640 1 -- 172.21.15.45:0/1463880962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8fc4010690 con 0x7f8fcc1066b0 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.073+0000 7f8fd483f740 1 Processor -- start 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 -- start start 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d26543cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d26544710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26551400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559d264a82c0 con 0x559d26544290 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559d26450910 con 0x559d26545980 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fd483f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d264a4280 con 0x559d2654f010 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fc8ff9640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 0x7f8fb40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d26544710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d26544710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37252/0 (socket says 172.21.15.45:37252) 2024-04-25T00:24:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d26543cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcb7fe640 1 -- 172.21.15.45:0/4266241898 learned_addr learned my addr 172.21.15.45:0/4266241898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcbfff640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 0x7f8fb40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26551400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fc8ff9640 1 -- 172.21.15.45:0/1463880962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8fc40cfcb0 con 0x7f8fcc1066b0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4266241898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 msgr2=0x559d26551400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26551400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4266241898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 msgr2=0x559d26544710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d26544710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.074+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4266241898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d26461fb0 con 0x559d26545980 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.075+0000 7f8fcbfff640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 0x7f8fb40a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d010 tx=0x7f8fc40bc040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.075+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d26543cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005e30 tx=0x7f8fc0031820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b791ecd93c648cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.075+0000 7f8fb1ffb640 1 -- 172.21.15.45:0/4266241898 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0031b30 con 0x559d26545980 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.075+0000 7f8fb1ffb640 1 -- 172.21.15.45:0/4266241898 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc0036050 con 0x559d26545980 2024-04-25T00:24:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.075+0000 7f8fb1ffb640 1 -- 172.21.15.45:0/4266241898 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004690 con 0x559d26545980 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- 172.21.15.45:0/4266241898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 msgr2=0x559d26543cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d26543cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005e30 tx=0x7f8fc0031820 comp rx=0 tx=0).stop 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- 172.21.15.45:0/4266241898 shutdown_connections 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26551400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d26544710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4266241898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d26543cc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- 172.21.15.45:0/4266241898 >> 172.21.15.45:0/4266241898 conn(0x559d2650c950 msgr2=0x559d265407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- 172.21.15.45:0/4266241898 shutdown_connections 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- 172.21.15.45:0/4266241898 wait complete. 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 -- start start 2024-04-25T00:24:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.076+0000 7f8fd483f740 1 Processor -- start 2024-04-25T00:24:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26545980 0x559d2654c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2654f010 0x559d2654c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559d26551e40 con 0x559d26544290 2024-04-25T00:24:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559d26463e90 con 0x559d2654f010 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d26551fc0 con 0x559d26545980 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fd483f740 10 client.0 _ll_register_callbacks cb 0x559d2649f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37260/0 (socket says 172.21.15.45:37260) 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4032585153 learned_addr learned my addr 172.21.15.45:0/4032585153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26545980 0x559d2654c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2654f010 0x559d2654c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4032585153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26545980 msgr2=0x559d2654c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26545980 0x559d2654c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2654f010 msgr2=0x559d2654c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2654f010 0x559d2654c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 -- 172.21.15.45:0/4032585153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d262847e0 con 0x559d26544290 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8fc002a920 tx=0x7f8fc0005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d186c7468905d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.077+0000 7f8faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004860 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8fc0003970 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc0036050 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fc0003b70 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d262e4f70 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc0036050 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fc0030070 con 0x559d26544290 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 20 client.16371 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d2655ef30 0x559d26561320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 4 client.16371 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d2647c5a0 con 0x559d2655ef30 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.078+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d2655ef30 0x559d26561320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.079+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d2655ef30 0x559d26561320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a3b64a079bedd5 server_cookie=a60a271eb0f6a1d2 in_seq=0 out_seq=0 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.079+0000 7f8faaffd640 10 client.16371 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.079+0000 7f8faaffd640 1 -- 172.21.15.45:0/4032585153 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d2647c5a0 con 0x559d2655ef30 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8faaffd640 10 client.16371 handle_command_reply: tid=0 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 client.16371 shutdown 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 msgr2=0x559d2654bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8fc002a920 tx=0x7f8fc0005c30 comp rx=0 tx=0).stop 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 shutdown_connections 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2654f010 0x559d2654c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26545980 0x559d2654c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d2655ef30 0x559d26561320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 --2- 172.21.15.45:0/4032585153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26544290 0x559d2654bf30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 >> 172.21.15.45:0/4032585153 conn(0x559d2650c950 msgr2=0x559d265465f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8faaffd640 0 client.16371 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 shutdown_connections 2024-04-25T00:24:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 1 -- 172.21.15.45:0/4032585153 wait complete. 2024-04-25T00:24:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.080+0000 7f8fd483f740 20 client.16371 trim_cache size 0 max 16384 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 Processor -- start 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 -- start start 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26544290 0x559d2654af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d265491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1aea70 con 0x559d2654f010 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1b05b0 con 0x559d26545980 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fd483f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1af140 con 0x559d26544290 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37276/0 (socket says 172.21.15.45:37276) 2024-04-25T00:24:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fcaffd640 1 -- 172.21.15.45:0/1694474279 learned_addr learned my addr 172.21.15.45:0/1694474279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26544290 0x559d2654af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.098+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d265491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 -- 172.21.15.45:0/1694474279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26544290 msgr2=0x559d2654af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26544290 0x559d2654af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 -- 172.21.15.45:0/1694474279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 msgr2=0x559d265491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d265491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 -- 172.21.15.45:0/1694474279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2617c170 con 0x559d2654f010 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009a40 tx=0x7f8fc0031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c06ec1e24d54150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fb17fa640 1 -- 172.21.15.45:0/1694474279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00047a0 con 0x559d2654f010 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fb17fa640 1 -- 172.21.15.45:0/1694474279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc0036040 con 0x559d2654f010 2024-04-25T00:24:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.099+0000 7f8fb17fa640 1 -- 172.21.15.45:0/1694474279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0030a50 con 0x559d2654f010 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- 172.21.15.45:0/1694474279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 msgr2=0x559d265496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009a40 tx=0x7f8fc0031e40 comp rx=0 tx=0).stop 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- 172.21.15.45:0/1694474279 shutdown_connections 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d2654f010 0x559d265496f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26545980 0x559d265491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 --2- 172.21.15.45:0/1694474279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d26544290 0x559d2654af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- 172.21.15.45:0/1694474279 >> 172.21.15.45:0/1694474279 conn(0x559d2650c950 msgr2=0x559d2654a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- 172.21.15.45:0/1694474279 shutdown_connections 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- 172.21.15.45:0/1694474279 wait complete. 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 -- start start 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.100+0000 7f8fd483f740 1 Processor -- start 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 0x559d26561900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26562380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1afee0 con 0x559d26545980 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1af810 con 0x559d26544290 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1aca50 con 0x559d2654f010 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fd483f740 10 client.0 _ll_register_callbacks cb 0x7f8fbc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37284/0 (socket says 172.21.15.45:37284) 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 -- 172.21.15.45:0/2198552920 learned_addr learned my addr 172.21.15.45:0/2198552920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 0x559d26561900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26562380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 -- 172.21.15.45:0/2198552920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 msgr2=0x559d26562380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26562380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 -- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 msgr2=0x559d26561900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 0x559d26561900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 -- 172.21.15.45:0/2198552920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d265c4110 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 0x559d26561900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8fcb7fe640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x559d26552030 tx=0x7f8fbc0204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a15cc8692076ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.101+0000 7f8faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc005020 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8fbc0051c0 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc0036040 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc030050 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 10 client.16377 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d265c5420 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc0036040 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fbc003710 con 0x559d26545980 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 20 client.16377 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d265c5bd0 0x559d265c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 4 client.16377 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.102+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d264b5800 con 0x559d265c5bd0 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.103+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d265c5bd0 0x559d265c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.103+0000 7f8fcaffd640 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d265c5bd0 0x559d265c7ff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e56495a0c83ee9 server_cookie=7805ba50a00111c6 in_seq=0 out_seq=0 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.103+0000 7f8faaffd640 10 client.16377 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8faaffd640 1 -- 172.21.15.45:0/2198552920 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d264b5800 con 0x559d265c5bd0 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8faaffd640 10 client.16377 handle_command_reply: tid=0 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 client.16377 shutdown 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 msgr2=0x559d26561e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x559d26552030 tx=0x7f8fbc0204d0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 shutdown_connections 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d2654f010 0x559d26562380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559d265c5bd0 0x559d265c7ff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d26545980 0x559d26561e40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 --2- 172.21.15.45:0/2198552920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d26544290 0x559d26561900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8faaffd640 0 client.16377 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 >> 172.21.15.45:0/2198552920 conn(0x559d2650c950 msgr2=0x559d2654bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 shutdown_connections 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 1 -- 172.21.15.45:0/2198552920 wait complete. 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.104+0000 7f8fd483f740 20 client.16377 trim_cache size 0 max 16384 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "24833040-e344-4645-990a-72ea19a799f1", 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:24:14.110 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:14.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 msgr2=0x7f8fb40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 0x7f8fb40a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d010 tx=0x7f8fc40bc040 comp rx=0 tx=0).stop 2024-04-25T00:24:14.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 msgr2=0x7f8fcc19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400e970 tx=0x7f8fc400ed70 comp rx=0 tx=0).stop 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 shutdown_connections 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8fb40a2d00 0x7f8fb40a51c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc1066b0 0x7f8fcc19c860 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc100fd0 0x7f8fcc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 --2- 172.21.15.45:0/1463880962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fcc100610 0x7f8fcc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 >> 172.21.15.45:0/1463880962 conn(0x7f8fcc077740 msgr2=0x7f8fcc0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 shutdown_connections 2024-04-25T00:24:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.105+0000 7f8fd1b30640 1 -- 172.21.15.45:0/1463880962 wait complete. 2024-04-25T00:24:14.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:14.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 -- 172.21.15.45:0/3061057417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c1029c0 msgr2=0x7fab4c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 --2- 172.21.15.45:0/3061057417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c1029c0 0x7fab4c102da0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab48009900 tx=0x7fab4802a510 comp rx=0 tx=0).stop 2024-04-25T00:24:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 -- 172.21.15.45:0/3061057417 shutdown_connections 2024-04-25T00:24:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 --2- 172.21.15.45:0/3061057417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c10bd00 0x7fab4c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 --2- 172.21.15.45:0/3061057417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 --2- 172.21.15.45:0/3061057417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c1029c0 0x7fab4c102da0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 -- 172.21.15.45:0/3061057417 >> 172.21.15.45:0/3061057417 conn(0x7fab4c077740 msgr2=0x7fab4c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.226+0000 7fab537e9640 1 -- 172.21.15.45:0/3061057417 shutdown_connections 2024-04-25T00:24:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 -- 172.21.15.45:0/3061057417 wait complete. 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 Processor -- start 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 -- start start 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c1029c0 0x7fab4c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c111b50 con 0x7fab4c10bd00 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1119d0 con 0x7fab4c1029c0 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab537e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c111cd0 con 0x7fab4c1032e0 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.227+0000 7fab51d86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab51d86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37294/0 (socket says 172.21.15.45:37294) 2024-04-25T00:24:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab51d86640 1 -- 172.21.15.45:0/684125022 learned_addr learned my addr 172.21.15.45:0/684125022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab51585640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c1029c0 0x7fab4c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 -- 172.21.15.45:0/684125022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c1029c0 msgr2=0x7fab4c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c1029c0 0x7fab4c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 -- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 msgr2=0x7fab4c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 -- 172.21.15.45:0/684125022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c19d040 con 0x7fab4c1032e0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab51d86640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab50d84640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c198590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00e9f0 tx=0x7fab3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab427fc640 1 -- 172.21.15.45:0/684125022 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00ccc0 con 0x7fab4c1032e0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c19d330 con 0x7fab4c1032e0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab427fc640 1 -- 172.21.15.45:0/684125022 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3c005020 con 0x7fab4c1032e0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab427fc640 1 -- 172.21.15.45:0/684125022 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0049e0 con 0x7fab4c1032e0 2024-04-25T00:24:14.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.228+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1a4bc0 con 0x7fab4c1032e0 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.230+0000 7fab427fc640 1 -- 172.21.15.45:0/684125022 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fab3c005560 con 0x7fab4c1032e0 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 Processor -- start 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 -- start start 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 0x555cefb3fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555cefa976b0 con 0x555cefb32630 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555cefa3f880 con 0x555cefb3d620 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab565cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555cefa93670 con 0x555cefb32f50 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab427fc640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 0x7fab380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab51585640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab51585640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:52628/0 (socket says 172.21.15.45:52628) 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab51d86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab51585640 1 -- 172.21.15.45:0/636370967 learned_addr learned my addr 172.21.15.45:0/636370967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.231+0000 7fab51d86640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 0x7fab380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab50d84640 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 0x555cefb3fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab427fc640 1 -- 172.21.15.45:0/684125022 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fab3c0cf750 con 0x7fab4c1032e0 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 -- 172.21.15.45:0/636370967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 msgr2=0x555cefb3fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 0x555cefb3fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 -- 172.21.15.45:0/636370967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 msgr2=0x555cefb32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 -- 172.21.15.45:0/636370967 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cef8757d0 con 0x555cefb32f50 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab50d84640 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 0x555cefb3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51585640 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab48009900 tx=0x7fab48005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90fb49087f04d637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab51d86640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 0x7fab380a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab44005c40 tx=0x7fab44009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab377fe640 1 -- 172.21.15.45:0/636370967 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48035040 con 0x555cefb32f50 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab377fe640 1 -- 172.21.15.45:0/636370967 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48004400 con 0x555cefb32f50 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.232+0000 7fab377fe640 1 -- 172.21.15.45:0/636370967 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480035f0 con 0x555cefb32f50 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- 172.21.15.45:0/636370967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 msgr2=0x555cefb3d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab48009900 tx=0x7fab48005de0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- 172.21.15.45:0/636370967 shutdown_connections 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb3d620 0x555cefb3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb3d0e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 --2- 172.21.15.45:0/636370967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- 172.21.15.45:0/636370967 >> 172.21.15.45:0/636370967 conn(0x555cefb2e950 msgr2=0x555cefb2ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- 172.21.15.45:0/636370967 shutdown_connections 2024-04-25T00:24:14.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- 172.21.15.45:0/636370967 wait complete. 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 -- start start 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.233+0000 7fab565cb740 1 Processor -- start 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb3aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb3d620 0x555cefb3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555cefb403b0 con 0x555cefb32630 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555cefa52f50 con 0x555cefb32f50 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555cefb40530 con 0x555cefb3d620 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab565cb740 10 client.0 _ll_register_callbacks cb 0x555cefa8e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37312/0 (socket says 172.21.15.45:37312) 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 -- 172.21.15.45:0/1573527135 learned_addr learned my addr 172.21.15.45:0/1573527135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51d86640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb3d620 0x555cefb3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab50d84640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb3aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 -- 172.21.15.45:0/1573527135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb3d620 msgr2=0x555cefb3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb3d620 0x555cefb3aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 -- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 msgr2=0x555cefb3aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb3aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab51585640 1 -- 172.21.15.45:0/1573527135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cef7ab3f0 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.234+0000 7fab50d84640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb3aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab51585640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fab480045d0 tx=0x7fab4802a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10c27d765ea942f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48052060 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fab48003800 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab48004400 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab48035040 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 10 client.16389 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cef76b0f0 con 0x555cefb32630 2024-04-25T00:24:14.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48004400 con 0x555cefb32630 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fab4804c070 con 0x555cefb32630 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 20 client.16389 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb4d660 0x555cefb4fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 4 client.16389 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.235+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cefaa4bf0 con 0x555cefb4d660 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.236+0000 7fab50d84640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb4d660 0x555cefb4fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.236+0000 7fab50d84640 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb4d660 0x555cefb4fa50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b990cea8bddd13 server_cookie=726c10c6dfd46c23 in_seq=0 out_seq=0 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.236+0000 7fab34ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab34ff9640 1 -- 172.21.15.45:0/1573527135 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cefaa4bf0 con 0x555cefb4d660 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab34ff9640 10 client.16389 handle_command_reply: tid=0 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 client.16389 shutdown 2024-04-25T00:24:14.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 msgr2=0x555cefb3a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fab480045d0 tx=0x7fab4802a510 comp rx=0 tx=0).stop 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 shutdown_connections 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb3d620 0x555cefb3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb3aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb4d660 0x555cefb4fa50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab34ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 --2- 172.21.15.45:0/1573527135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb3a570 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 >> 172.21.15.45:0/1573527135 conn(0x555cefb2e950 msgr2=0x555cefb34750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 shutdown_connections 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 1 -- 172.21.15.45:0/1573527135 wait complete. 2024-04-25T00:24:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.237+0000 7fab565cb740 20 client.16389 trim_cache size 0 max 16384 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 Processor -- start 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 -- start start 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 0x555cefb49270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb2ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ac8e0 con 0x555cefb2e950 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ae420 con 0x555cefb32f50 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab565cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1acfb0 con 0x555cefb32630 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab51585640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab51585640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37320/0 (socket says 172.21.15.45:37320) 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab51585640 1 -- 172.21.15.45:0/2771694043 learned_addr learned my addr 172.21.15.45:0/2771694043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab50d84640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 0x555cefb49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.255+0000 7fab51d86640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb2ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 -- 172.21.15.45:0/2771694043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 msgr2=0x555cefb49270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 0x555cefb49270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 -- 172.21.15.45:0/2771694043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 msgr2=0x555cefb2ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb2ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 -- 172.21.15.45:0/2771694043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cefb4fdc0 con 0x555cefb2e950 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab50d84640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 0x555cefb49270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab51585640 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab480054a0 tx=0x7fab480048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb66e7ec5470e3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab36ffd640 1 -- 172.21.15.45:0/2771694043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48052070 con 0x555cefb2e950 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab36ffd640 1 -- 172.21.15.45:0/2771694043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48009520 con 0x555cefb2e950 2024-04-25T00:24:14.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.256+0000 7fab36ffd640 1 -- 172.21.15.45:0/2771694043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48003b00 con 0x555cefb2e950 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- 172.21.15.45:0/2771694043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 msgr2=0x555cefb3cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab480054a0 tx=0x7fab480048d0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- 172.21.15.45:0/2771694043 shutdown_connections 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb32f50 0x555cefb2ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32630 0x555cefb49270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 --2- 172.21.15.45:0/2771694043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb2e950 0x555cefb3cca0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- 172.21.15.45:0/2771694043 >> 172.21.15.45:0/2771694043 conn(0x555cefb35f40 msgr2=0x555cefb2d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- 172.21.15.45:0/2771694043 shutdown_connections 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- 172.21.15.45:0/2771694043 wait complete. 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 -- start start 2024-04-25T00:24:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.257+0000 7fab565cb740 1 Processor -- start 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 0x555cefb45a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb41300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1add50 con 0x555cefb32630 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ad680 con 0x555cefb2e950 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1aa8c0 con 0x555cefb32f50 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab565cb740 10 client.0 _ll_register_callbacks cb 0x7fab44015810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab50d84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab50d84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37322/0 (socket says 172.21.15.45:37322) 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab50d84640 1 -- 172.21.15.45:0/3932077057 learned_addr learned my addr 172.21.15.45:0/3932077057 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51d86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb41300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 0x555cefb45a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 -- 172.21.15.45:0/3932077057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 msgr2=0x555cefb41300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb41300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 -- 172.21.15.45:0/3932077057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 msgr2=0x555cefb40dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 -- 172.21.15.45:0/3932077057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cefb419f0 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51d86640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb41300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab50d84640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab51585640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 0x555cefb45a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab4802ab00 tx=0x7fab48004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a1424d438e5203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.258+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48052040 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fab48003800 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab48009520 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab48003a00 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 10 client.25931 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cefbb3770 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48009520 con 0x555cefb2e950 2024-04-25T00:24:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fab4805c030 con 0x555cefb2e950 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 10 client.25931 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 20 client.25931 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb3d620 0x555cefbb5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 4 client.25931 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cefa9ef40 con 0x555cefb3d620 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.259+0000 7fab50d84640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb3d620 0x555cefbb5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab50d84640 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb3d620 0x555cefbb5ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8367ba3c1339f8 server_cookie=bf615a62a3679b1e in_seq=0 out_seq=0 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab34ff9640 10 client.25931 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab34ff9640 1 -- 172.21.15.45:0/3932077057 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555cefa9ef40 con 0x555cefb3d620 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab34ff9640 10 client.25931 handle_command_reply: tid=0 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 client.25931 shutdown 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 msgr2=0x555cefb45a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 0x555cefb45a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab4802ab00 tx=0x7fab48004940 comp rx=0 tx=0).stop 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 shutdown_connections 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555cefb32f50 0x555cefb41300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x555cefb3d620 0x555cefbb5ca0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555cefb32630 0x555cefb40dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 --2- 172.21.15.45:0/3932077057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555cefb2e950 0x555cefb45a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab34ff9640 0 client.25931 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 >> 172.21.15.45:0/3932077057 conn(0x555cefb35f40 msgr2=0x555cefb3a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 shutdown_connections 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 1 -- 172.21.15.45:0/3932077057 wait complete. 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.260+0000 7fab565cb740 20 client.25931 trim_cache size 0 max 16384 2024-04-25T00:24:14.267 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "24833040-e344-4645-990a-72ea19a799f1": { 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "24833040-e344-4645-990a-72ea19a799f1", 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:24:14.268 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 msgr2=0x7fab380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 0x7fab380a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab44005c40 tx=0x7fab44009500 comp rx=0 tx=0).stop 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 msgr2=0x7fab4c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c198590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00e9f0 tx=0x7fab3c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 shutdown_connections 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fab380a2d00 0x7fab380a51c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c10bd00 0x7fab4c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab4c1032e0 0x7fab4c198590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 --2- 172.21.15.45:0/684125022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4c1029c0 0x7fab4c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 >> 172.21.15.45:0/684125022 conn(0x7fab4c077740 msgr2=0x7fab4c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 shutdown_connections 2024-04-25T00:24:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:14.261+0000 7fab537e9640 1 -- 172.21.15.45:0/684125022 wait complete. 2024-04-25T00:24:14.273 INFO:tasks.cephfs.filesystem:scrub status for tag:24833040-e344-4645-990a-72ea19a799f1 - {'path': '/', 'tag': '24833040-e344-4645-990a-72ea19a799f1', 'options': 'recursive,force'} 2024-04-25T00:24:14.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:14 smithi045 ceph-mon[29194]: pgmap v71: 129 pgs: 129 active+clean; 284 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 939 op/s 2024-04-25T00:24:14.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:14 smithi122 ceph-mon[32959]: pgmap v71: 129 pgs: 129 active+clean; 284 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 939 op/s 2024-04-25T00:24:14.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:14 smithi071 ceph-mon[32667]: pgmap v71: 129 pgs: 129 active+clean; 284 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 939 op/s 2024-04-25T00:24:15.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:15 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:24:15.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:15 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:15.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:15 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:24:15.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:15 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:15.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:15 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:24:15.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:15 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:15.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:15 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:24:15.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:15 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:15.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:15 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:24:15.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:15 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:15.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:15 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:24:15.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:15 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:16.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:16 smithi122 ceph-mon[32959]: pgmap v72: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:16 smithi071 ceph-mon[32667]: pgmap v72: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:16.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:16 smithi045 ceph-mon[29194]: pgmap v72: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:18.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:18 smithi122 ceph-mon[32959]: pgmap v73: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:18.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:18 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:24:18.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:18 smithi071 ceph-mon[32667]: pgmap v73: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:18.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:18 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:24:18.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:18 smithi045 ceph-mon[29194]: pgmap v73: 129 pgs: 129 active+clean; 297 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 MiB/s wr, 1.24k op/s 2024-04-25T00:24:18.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:18 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:24:20.649 INFO:tasks.workunit.client.0.smithi045.stdout:setting up random write file 2024-04-25T00:24:20.649 INFO:tasks.workunit.client.0.smithi045.stdout:done setting up random write file 2024-04-25T00:24:20.649 INFO:tasks.workunit.client.0.smithi045.stdout:starting fsync run 2024-04-25T00:24:20.649 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0107s 2024-04-25T00:24:20.649 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0065s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0063s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0056s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0062s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0068s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0071s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.650 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0062s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0059s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0059s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0061s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.651 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0062s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0063s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0061s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0061s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0058s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0005s fsync time: 0.0103s 2024-04-25T00:24:20.652 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0058s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0062s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0056s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.1241s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0060s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.653 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0003s fsync time: 0.0056s 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stdout:write time: 0.0002s fsync time: 0.0062s 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stdout:run done 61 fsyncs total, killing random writer 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-25T00:24:20.654 INFO:tasks.workunit.client.0.smithi045.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-25T00:24:20.655 INFO:tasks.workunit.client.0.smithi045.stderr:+ whereis lsof 2024-04-25T00:24:20.667 INFO:tasks.workunit.client.0.smithi045.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-04-25T00:24:20.667 INFO:tasks.workunit.client.0.smithi045.stderr:+ lsof 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/e3e1f1a0ae9d47a41eb7558367e574c697086102efe564893f82583ce468fe5d/merged 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f98a1230dd57187e1e8c93668b926cf4a153127b9448a48b9d36c4b33a97a138/merged 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/24aee8797610568b1423c3d83b413b84235fa81738f887dfc4e841fcab73aa32/merged 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/3fa21e6476b4a9966a4d5832bb0368345c9626d6ac64e5bf7fa2a973af02ea40/merged 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.686 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/015e9803ad621f1681b22c9ab807aafa22a23dbd5b534401b082dc4a8b7dd1ac/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d3092cdb0c48eb8e7e204179e295cf099e164142e74bcbde8ad82d87b1d4c77b/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/c5ecbe9ed7504af81f00348bd0f6e472aea3dd85d0ebe984fd7d23e4949bc121/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/90cbb45734a7bedacc29482cc8c5302739a309229b1d50c773a23e47ac8d62f4/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/62494f2c702d8003dc5b283a343b78cde9ac200a175f59231a39449c752872da/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/13b8f1fcf66d0ceef241bb23c89918bbff88377a35ac00c11f76ff74ef56d604/merged 2024-04-25T00:24:20.687 INFO:tasks.workunit.client.0.smithi045.stderr: Output information may be incomplete. 2024-04-25T00:24:20.751 INFO:tasks.workunit.client.0.smithi045.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-25T00:24:20.752 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-04-25T00:24:20.752 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-04-25T00:24:20.752 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-04-25T00:24:20.752 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-04-25T00:24:20.753 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-04-25T00:24:20.754 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-04-25T00:24:20.755 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-04-25T00:24:20.756 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-04-25T00:24:20.757 INFO:tasks.workunit.client.0.smithi045.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-04-25T00:24:20.758 INFO:tasks.workunit.client.0.smithi045.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-04-25T00:24:20.759 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-04-25T00:24:20.760 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-04-25T00:24:20.761 INFO:tasks.workunit.client.0.smithi045.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-04-25T00:24:20.762 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-04-25T00:24:20.763 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-04-25T00:24:20.764 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-04-25T00:24:20.765 INFO:tasks.workunit.client.0.smithi045.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-04-25T00:24:20.766 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-04-25T00:24:20.767 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-04-25T00:24:20.768 INFO:tasks.workunit.client.0.smithi045.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-04-25T00:24:20.769 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-04-25T00:24:20.770 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-04-25T00:24:20.771 INFO:tasks.workunit.client.0.smithi045.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-04-25T00:24:20.772 INFO:tasks.workunit.client.0.smithi045.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-04-25T00:24:20.773 INFO:tasks.workunit.client.0.smithi045.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:khugepage 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:khugepage 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:khugepage 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:khugepage 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:cryptd 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:cryptd 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:cryptd 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:cryptd 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kintegrit 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kintegrit 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-04-25T00:24:20.774 INFO:tasks.workunit.client.0.smithi045.stdout:kintegrit 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:kintegrit 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:kblockd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:kblockd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:kblockd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:kblockd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:blkcg_pun 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:blkcg_pun 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:blkcg_pun 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:blkcg_pun 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-04-25T00:24:20.775 INFO:tasks.workunit.client.0.smithi045.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-04-25T00:24:20.776 INFO:tasks.workunit.client.0.smithi045.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kthrotld 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kthrotld 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kthrotld 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:kthrotld 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:acpi_ther 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-04-25T00:24:20.777 INFO:tasks.workunit.client.0.smithi045.stdout:acpi_ther 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:acpi_ther 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:acpi_ther 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kmpath_rd 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kmpath_rd 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kmpath_rd 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kmpath_rd 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kaluad 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kaluad 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kaluad 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kaluad 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:mld 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:mld 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:mld 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:mld 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-04-25T00:24:20.778 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:ipv6_addr 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:ipv6_addr 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:ipv6_addr 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:ipv6_addr 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kstrp 123 root cwd unknown /proc/123/cwd (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kstrp 123 root rtd unknown /proc/123/root (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kstrp 123 root txt unknown /proc/123/exe (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:kstrp 123 root NOFD /proc/123/fd (opendir: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:zswap-shr 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-04-25T00:24:20.779 INFO:tasks.workunit.client.0.smithi045.stdout:zswap-shr 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:zswap-shr 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:zswap-shr 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 238 root cwd unknown /proc/238/cwd (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 238 root rtd unknown /proc/238/root (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 238 root txt unknown /proc/238/exe (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 238 root NOFD /proc/238/fd (opendir: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 277 root cwd unknown /proc/277/cwd (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 277 root rtd unknown /proc/277/root (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 277 root txt unknown /proc/277/exe (readlink: Permission denied) 2024-04-25T00:24:20.780 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 277 root NOFD /proc/277/fd (opendir: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 279 root cwd unknown /proc/279/cwd (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 279 root rtd unknown /proc/279/root (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 279 root txt unknown /proc/279/exe (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 279 root NOFD /proc/279/fd (opendir: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 281 root cwd unknown /proc/281/cwd (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 281 root rtd unknown /proc/281/root (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 281 root txt unknown /proc/281/exe (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 281 root NOFD /proc/281/fd (opendir: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 293 root cwd unknown /proc/293/cwd (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 293 root rtd unknown /proc/293/root (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 293 root txt unknown /proc/293/exe (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 293 root NOFD /proc/293/fd (opendir: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 297 root cwd unknown /proc/297/cwd (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 297 root rtd unknown /proc/297/root (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 297 root txt unknown /proc/297/exe (readlink: Permission denied) 2024-04-25T00:24:20.781 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 297 root NOFD /proc/297/fd (opendir: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 388 root cwd unknown /proc/388/cwd (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 388 root rtd unknown /proc/388/root (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 388 root txt unknown /proc/388/exe (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 388 root NOFD /proc/388/fd (opendir: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-wq 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-wq 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-wq 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-wq 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-rese 431 root cwd unknown /proc/431/cwd (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-rese 431 root rtd unknown /proc/431/root (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-rese 431 root txt unknown /proc/431/exe (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-rese 431 root NOFD /proc/431/fd (opendir: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-dele 433 root cwd unknown /proc/433/cwd (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-dele 433 root rtd unknown /proc/433/root (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-dele 433 root txt unknown /proc/433/exe (readlink: Permission denied) 2024-04-25T00:24:20.782 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-dele 433 root NOFD /proc/433/fd (opendir: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-auth 434 root cwd unknown /proc/434/cwd (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-auth 434 root rtd unknown /proc/434/root (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-auth 434 root txt unknown /proc/434/exe (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:nvme-auth 434 root NOFD /proc/434/fd (opendir: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ata_sff 439 root cwd unknown /proc/439/cwd (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ata_sff 439 root rtd unknown /proc/439/root (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ata_sff 439 root txt unknown /proc/439/exe (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ata_sff 439 root NOFD /proc/439/fd (opendir: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ixgbe 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ixgbe 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ixgbe 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:ixgbe 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_0 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-04-25T00:24:20.783 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_0 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_0 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_0 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_1 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_1 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_1 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_1 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-04-25T00:24:20.784 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_2 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_2 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_2 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_2 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_3 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_3 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_3 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_3 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_4 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-04-25T00:24:20.785 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_4 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_4 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_4 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_5 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_5 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_5 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_5 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-04-25T00:24:20.786 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_6 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_6 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_6 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_6 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_7 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_7 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_7 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_7 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-04-25T00:24:20.787 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_8 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_8 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_8 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_8 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_9 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_9 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_9 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_eh_9 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-04-25T00:24:20.788 INFO:tasks.workunit.client.0.smithi045.stdout:scsi_tmf_ 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:jbd2/sda1 497 root cwd unknown /proc/497/cwd (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:jbd2/sda1 497 root rtd unknown /proc/497/root (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:jbd2/sda1 497 root txt unknown /proc/497/exe (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:jbd2/sda1 497 root NOFD /proc/497/fd (opendir: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:ext4-rsv- 498 root cwd unknown /proc/498/cwd (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:ext4-rsv- 498 root rtd unknown /proc/498/root (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:ext4-rsv- 498 root txt unknown /proc/498/exe (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:ext4-rsv- 498 root NOFD /proc/498/fd (opendir: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-j 573 root cwd unknown /proc/573/cwd (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-j 573 root rtd unknown /proc/573/root (readlink: Permission denied) 2024-04-25T00:24:20.789 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-j 573 root txt unknown /proc/573/exe (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-j 573 root NOFD /proc/573/fd (opendir: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ext4lazyi 579 root cwd unknown /proc/579/cwd (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ext4lazyi 579 root rtd unknown /proc/579/root (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ext4lazyi 579 root txt unknown /proc/579/exe (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ext4lazyi 579 root NOFD /proc/579/fd (opendir: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-u 593 root cwd unknown /proc/593/cwd (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-u 593 root rtd unknown /proc/593/root (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-u 593 root txt unknown /proc/593/exe (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-u 593 root NOFD /proc/593/fd (opendir: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ipmi-msgh 645 root cwd unknown /proc/645/cwd (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ipmi-msgh 645 root rtd unknown /proc/645/root (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ipmi-msgh 645 root txt unknown /proc/645/exe (readlink: Permission denied) 2024-04-25T00:24:20.790 INFO:tasks.workunit.client.0.smithi045.stdout:ipmi-msgh 645 root NOFD /proc/645/fd (opendir: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:kipmi0 663 root cwd unknown /proc/663/cwd (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:kipmi0 663 root rtd unknown /proc/663/root (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:kipmi0 663 root txt unknown /proc/663/exe (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:kipmi0 663 root NOFD /proc/663/fd (opendir: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpcbind 668 rpc cwd unknown /proc/668/cwd (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpcbind 668 rpc rtd unknown /proc/668/root (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpcbind 668 rpc txt unknown /proc/668/exe (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpcbind 668 rpc NOFD /proc/668/fd (opendir: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpciod 673 root cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpciod 673 root rtd unknown /proc/673/root (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpciod 673 root txt unknown /proc/673/exe (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:rpciod 673 root NOFD /proc/673/fd (opendir: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:xprtiod 674 root cwd unknown /proc/674/cwd (readlink: Permission denied) 2024-04-25T00:24:20.791 INFO:tasks.workunit.client.0.smithi045.stdout:xprtiod 674 root rtd unknown /proc/674/root (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:xprtiod 674 root txt unknown /proc/674/exe (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:xprtiod 674 root NOFD /proc/674/fd (opendir: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 root cwd unknown /proc/675/cwd (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 root rtd unknown /proc/675/root (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 root txt unknown /proc/675/exe (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 root NOFD /proc/675/fd (opendir: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 676 auditd root cwd unknown /proc/675/task/676/cwd (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 676 auditd root rtd unknown /proc/675/task/676/root (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 676 auditd root txt unknown /proc/675/task/676/exe (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 676 auditd root NOFD /proc/675/task/676/fd (opendir: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 678 auditd root cwd unknown /proc/675/task/678/cwd (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 678 auditd root rtd unknown /proc/675/task/678/root (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 678 auditd root txt unknown /proc/675/task/678/exe (readlink: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:auditd 675 678 auditd root NOFD /proc/675/task/678/fd (opendir: Permission denied) 2024-04-25T00:24:20.792 INFO:tasks.workunit.client.0.smithi045.stdout:sedispatc 677 root cwd unknown /proc/677/cwd (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:sedispatc 677 root rtd unknown /proc/677/root (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:sedispatc 677 root txt unknown /proc/677/exe (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:sedispatc 677 root NOFD /proc/677/fd (opendir: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 717 dbus cwd unknown /proc/717/cwd (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 717 dbus rtd unknown /proc/717/root (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 717 dbus txt unknown /proc/717/exe (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 717 dbus NOFD /proc/717/fd (opendir: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 718 dbus cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 718 dbus rtd unknown /proc/718/root (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 718 dbus txt unknown /proc/718/exe (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:dbus-brok 718 dbus NOFD /proc/718/fd (opendir: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-04-25T00:24:20.793 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 768 gmain root cwd unknown /proc/719/task/768/cwd (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 768 gmain root rtd unknown /proc/719/task/768/root (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 768 gmain root txt unknown /proc/719/task/768/exe (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 768 gmain root NOFD /proc/719/task/768/fd (opendir: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 769 gdbus root cwd unknown /proc/719/task/769/cwd (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 769 gdbus root rtd unknown /proc/719/task/769/root (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 769 gdbus root txt unknown /proc/719/task/769/exe (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:NetworkMa 719 769 gdbus root NOFD /proc/719/task/769/fd (opendir: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 733 gmain root cwd unknown /proc/722/task/733/cwd (readlink: Permission denied) 2024-04-25T00:24:20.794 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 733 gmain root rtd unknown /proc/722/task/733/root (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 733 gmain root txt unknown /proc/722/task/733/exe (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:irqbalanc 722 733 gmain root NOFD /proc/722/task/733/fd (opendir: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:lsmd 723 libstoragemgmt cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:lsmd 723 libstoragemgmt rtd unknown /proc/723/root (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:lsmd 723 libstoragemgmt txt unknown /proc/723/exe (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:lsmd 723 libstoragemgmt NOFD /proc/723/fd (opendir: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:mcelog 724 root cwd unknown /proc/724/cwd (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:mcelog 724 root rtd unknown /proc/724/root (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:mcelog 724 root txt unknown /proc/724/exe (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:mcelog 724 root NOFD /proc/724/fd (opendir: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:smartd 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:smartd 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:smartd 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:smartd 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-04-25T00:24:20.795 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-l 730 root cwd unknown /proc/730/cwd (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-l 730 root rtd unknown /proc/730/root (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-l 730 root txt unknown /proc/730/exe (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:systemd-l 730 root NOFD /proc/730/fd (opendir: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 root cwd unknown /proc/825/cwd (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 root rtd unknown /proc/825/root (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 root txt unknown /proc/825/exe (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 root NOFD /proc/825/fd (opendir: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 826 gssproxy root cwd unknown /proc/825/task/826/cwd (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 826 gssproxy root rtd unknown /proc/825/task/826/root (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 826 gssproxy root txt unknown /proc/825/task/826/exe (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 826 gssproxy root NOFD /proc/825/task/826/fd (opendir: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 827 gssproxy root cwd unknown /proc/825/task/827/cwd (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 827 gssproxy root rtd unknown /proc/825/task/827/root (readlink: Permission denied) 2024-04-25T00:24:20.796 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 827 gssproxy root txt unknown /proc/825/task/827/exe (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 827 gssproxy root NOFD /proc/825/task/827/fd (opendir: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 828 gssproxy root cwd unknown /proc/825/task/828/cwd (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 828 gssproxy root rtd unknown /proc/825/task/828/root (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 828 gssproxy root txt unknown /proc/825/task/828/exe (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 828 gssproxy root NOFD /proc/825/task/828/fd (opendir: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 829 gssproxy root cwd unknown /proc/825/task/829/cwd (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 829 gssproxy root rtd unknown /proc/825/task/829/root (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 829 gssproxy root txt unknown /proc/825/task/829/exe (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 829 gssproxy root NOFD /proc/825/task/829/fd (opendir: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 830 gssproxy root cwd unknown /proc/825/task/830/cwd (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 830 gssproxy root rtd unknown /proc/825/task/830/root (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 830 gssproxy root txt unknown /proc/825/task/830/exe (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:gssproxy 825 830 gssproxy root NOFD /proc/825/task/830/fd (opendir: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:atd 832 root cwd unknown /proc/832/cwd (readlink: Permission denied) 2024-04-25T00:24:20.797 INFO:tasks.workunit.client.0.smithi045.stdout:atd 832 root rtd unknown /proc/832/root (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:atd 832 root txt unknown /proc/832/exe (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:atd 832 root NOFD /proc/832/fd (opendir: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:crond 833 root cwd unknown /proc/833/cwd (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:crond 833 root rtd unknown /proc/833/root (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:crond 833 root txt unknown /proc/833/exe (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:crond 833 root NOFD /proc/833/fd (opendir: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 838 root cwd unknown /proc/838/cwd (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 838 root rtd unknown /proc/838/root (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 838 root txt unknown /proc/838/exe (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 838 root NOFD /proc/838/fd (opendir: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:nrpe 862 nrpe cwd unknown /proc/862/cwd (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:nrpe 862 nrpe rtd unknown /proc/862/root (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:nrpe 862 nrpe txt unknown /proc/862/exe (readlink: Permission denied) 2024-04-25T00:24:20.798 INFO:tasks.workunit.client.0.smithi045.stdout:nrpe 862 nrpe NOFD /proc/862/fd (opendir: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5204 root cwd unknown /proc/5204/cwd (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5204 root rtd unknown /proc/5204/root (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5204 root txt unknown /proc/5204/exe (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5204 root NOFD /proc/5204/fd (opendir: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5206 root cwd unknown /proc/5206/cwd (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5206 root rtd unknown /proc/5206/root (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5206 root txt unknown /proc/5206/exe (readlink: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:agetty 5206 root NOFD /proc/5206/fd (opendir: Permission denied) 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu cwd DIR 8,1 4096 2 / 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu rtd DIR 8,1 4096 2 / 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 21219 62167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 598426 850 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-25T00:24:20.799 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-25T00:24:20.800 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-04-25T00:24:20.801 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 1u unix 0x0000000000000000 0t0 24777 type=STREAM (CONNECTED) 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 2u unix 0x0000000000000000 0t0 24777 type=STREAM (CONNECTED) 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 3u unix 0x0000000000000000 0t0 24805 type=DGRAM (CONNECTED) 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 4u a_inode 0,14 0 2071 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 5u a_inode 0,14 0 2071 [signalfd] 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 6r a_inode 0,14 0 2071 inotify 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 7r DIR 0,25 0 3609 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 8u a_inode 0,14 0 2071 [timerfd] 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 9u a_inode 0,14 0 2071 [eventpoll:10,13] 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 10r REG 0,20 0 24794 /proc/5214/mountinfo 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 11r a_inode 0,14 0 2071 inotify 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 12u unix 0x0000000000000000 0t0 24827 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 13r a_inode 0,14 0 2071 inotify 2024-04-25T00:24:20.802 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 15u netlink 0t0 24809 KOBJECT_UEVENT 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 16u unix 0x0000000000000000 0t0 24814 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 17u unix 0x0000000000000000 0t0 24815 type=DGRAM (CONNECTED) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 18u unix 0x0000000000000000 0t0 24816 type=DGRAM (CONNECTED) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 19u unix 0x0000000000000000 0t0 24817 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 20u unix 0x0000000000000000 0t0 24819 type=STREAM (CONNECTED) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 5214 ubuntu 21u a_inode 0,14 0 2071 [timerfd] 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 5216 ubuntu cwd unknown /proc/5216/cwd (readlink: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 5216 ubuntu rtd unknown /proc/5216/root (readlink: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 5216 ubuntu txt unknown /proc/5216/exe (readlink: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 5216 ubuntu NOFD /proc/5216/fd (opendir: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:tls-strp 5567 root cwd unknown /proc/5567/cwd (readlink: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:tls-strp 5567 root rtd unknown /proc/5567/root (readlink: Permission denied) 2024-04-25T00:24:20.803 INFO:tasks.workunit.client.0.smithi045.stdout:tls-strp 5567 root txt unknown /proc/5567/exe (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:tls-strp 5567 root NOFD /proc/5567/fd (opendir: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 polkitd cwd unknown /proc/5650/cwd (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 polkitd rtd unknown /proc/5650/root (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 polkitd txt unknown /proc/5650/exe (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 polkitd NOFD /proc/5650/fd (opendir: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5652 gmain polkitd cwd unknown /proc/5650/task/5652/cwd (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5652 gmain polkitd rtd unknown /proc/5650/task/5652/root (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5652 gmain polkitd txt unknown /proc/5650/task/5652/exe (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5652 gmain polkitd NOFD /proc/5650/task/5652/fd (opendir: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5653 gdbus polkitd cwd unknown /proc/5650/task/5653/cwd (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5653 gdbus polkitd rtd unknown /proc/5650/task/5653/root (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5653 gdbus polkitd txt unknown /proc/5650/task/5653/exe (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5653 gdbus polkitd NOFD /proc/5650/task/5653/fd (opendir: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5654 JS\x20Hel polkitd cwd unknown /proc/5650/task/5654/cwd (readlink: Permission denied) 2024-04-25T00:24:20.804 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5654 JS\x20Hel polkitd rtd unknown /proc/5650/task/5654/root (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5654 JS\x20Hel polkitd txt unknown /proc/5650/task/5654/exe (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5654 JS\x20Hel polkitd NOFD /proc/5650/task/5654/fd (opendir: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5655 JS\x20Hel polkitd cwd unknown /proc/5650/task/5655/cwd (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5655 JS\x20Hel polkitd rtd unknown /proc/5650/task/5655/root (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5655 JS\x20Hel polkitd txt unknown /proc/5650/task/5655/exe (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5655 JS\x20Hel polkitd NOFD /proc/5650/task/5655/fd (opendir: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5656 JS\x20Hel polkitd cwd unknown /proc/5650/task/5656/cwd (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5656 JS\x20Hel polkitd rtd unknown /proc/5650/task/5656/root (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5656 JS\x20Hel polkitd txt unknown /proc/5650/task/5656/exe (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5656 JS\x20Hel polkitd NOFD /proc/5650/task/5656/fd (opendir: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5657 JS\x20Hel polkitd cwd unknown /proc/5650/task/5657/cwd (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5657 JS\x20Hel polkitd rtd unknown /proc/5650/task/5657/root (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5657 JS\x20Hel polkitd txt unknown /proc/5650/task/5657/exe (readlink: Permission denied) 2024-04-25T00:24:20.805 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5657 JS\x20Hel polkitd NOFD /proc/5650/task/5657/fd (opendir: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5658 JS\x20Hel polkitd cwd unknown /proc/5650/task/5658/cwd (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5658 JS\x20Hel polkitd rtd unknown /proc/5650/task/5658/root (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5658 JS\x20Hel polkitd txt unknown /proc/5650/task/5658/exe (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5658 JS\x20Hel polkitd NOFD /proc/5650/task/5658/fd (opendir: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5659 JS\x20Hel polkitd cwd unknown /proc/5650/task/5659/cwd (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5659 JS\x20Hel polkitd rtd unknown /proc/5650/task/5659/root (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5659 JS\x20Hel polkitd txt unknown /proc/5650/task/5659/exe (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5659 JS\x20Hel polkitd NOFD /proc/5650/task/5659/fd (opendir: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5660 JS\x20Hel polkitd cwd unknown /proc/5650/task/5660/cwd (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5660 JS\x20Hel polkitd rtd unknown /proc/5650/task/5660/root (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5660 JS\x20Hel polkitd txt unknown /proc/5650/task/5660/exe (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5660 JS\x20Hel polkitd NOFD /proc/5650/task/5660/fd (opendir: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5661 JS\x20Hel polkitd cwd unknown /proc/5650/task/5661/cwd (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5661 JS\x20Hel polkitd rtd unknown /proc/5650/task/5661/root (readlink: Permission denied) 2024-04-25T00:24:20.806 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5661 JS\x20Hel polkitd txt unknown /proc/5650/task/5661/exe (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5661 JS\x20Hel polkitd NOFD /proc/5650/task/5661/fd (opendir: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5662 polkitd polkitd cwd unknown /proc/5650/task/5662/cwd (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5662 polkitd polkitd rtd unknown /proc/5650/task/5662/root (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5662 polkitd polkitd txt unknown /proc/5650/task/5662/exe (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:polkitd 5650 5662 polkitd polkitd NOFD /proc/5650/task/5662/fd (opendir: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 root cwd unknown /proc/5880/cwd (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 root rtd unknown /proc/5880/root (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 root txt unknown /proc/5880/exe (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 root NOFD /proc/5880/fd (opendir: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5882 in:imjour root cwd unknown /proc/5880/task/5882/cwd (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5882 in:imjour root rtd unknown /proc/5880/task/5882/root (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5882 in:imjour root txt unknown /proc/5880/task/5882/exe (readlink: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5882 in:imjour root NOFD /proc/5880/task/5882/fd (opendir: Permission denied) 2024-04-25T00:24:20.807 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5883 rs:main root cwd unknown /proc/5880/task/5883/cwd (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5883 rs:main root rtd unknown /proc/5880/task/5883/root (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5883 rs:main root txt unknown /proc/5880/task/5883/exe (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:rsyslogd 5880 5883 rs:main root NOFD /proc/5880/task/5883/fd (opendir: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 19174 root cwd unknown /proc/19174/cwd (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 19174 root rtd unknown /proc/19174/root (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 19174 root txt unknown /proc/19174/exe (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 19174 root NOFD /proc/19174/fd (opendir: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19269 root cwd unknown /proc/19269/cwd (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19269 root rtd unknown /proc/19269/root (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19269 root txt unknown /proc/19269/exe (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19269 root NOFD /proc/19269/fd (opendir: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19298 root cwd unknown /proc/19298/cwd (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19298 root rtd unknown /proc/19298/root (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19298 root txt unknown /proc/19298/exe (readlink: Permission denied) 2024-04-25T00:24:20.808 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19298 root NOFD /proc/19298/fd (opendir: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19327 root cwd unknown /proc/19327/cwd (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19327 root rtd unknown /proc/19327/root (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19327 root txt unknown /proc/19327/exe (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19327 root NOFD /proc/19327/fd (opendir: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19356 root cwd unknown /proc/19356/cwd (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19356 root rtd unknown /proc/19356/root (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19356 root txt unknown /proc/19356/exe (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19356 root NOFD /proc/19356/fd (opendir: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19385 root cwd unknown /proc/19385/cwd (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19385 root rtd unknown /proc/19385/root (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19385 root txt unknown /proc/19385/exe (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:kdmflush/ 19385 root NOFD /proc/19385/fd (opendir: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:xfsalloc 19549 root cwd unknown /proc/19549/cwd (readlink: Permission denied) 2024-04-25T00:24:20.809 INFO:tasks.workunit.client.0.smithi045.stdout:xfsalloc 19549 root rtd unknown /proc/19549/root (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfsalloc 19549 root txt unknown /proc/19549/exe (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfsalloc 19549 root NOFD /proc/19549/fd (opendir: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs_mru_c 19550 root cwd unknown /proc/19550/cwd (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs_mru_c 19550 root rtd unknown /proc/19550/root (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs_mru_c 19550 root txt unknown /proc/19550/exe (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs_mru_c 19550 root NOFD /proc/19550/fd (opendir: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-buf/d 19551 root cwd unknown /proc/19551/cwd (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-buf/d 19551 root rtd unknown /proc/19551/root (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-buf/d 19551 root txt unknown /proc/19551/exe (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-buf/d 19551 root NOFD /proc/19551/fd (opendir: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-conv/ 19552 root cwd unknown /proc/19552/cwd (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-conv/ 19552 root rtd unknown /proc/19552/root (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-conv/ 19552 root txt unknown /proc/19552/exe (readlink: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-conv/ 19552 root NOFD /proc/19552/fd (opendir: Permission denied) 2024-04-25T00:24:20.810 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-recla 19553 root cwd unknown /proc/19553/cwd (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-recla 19553 root rtd unknown /proc/19553/root (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-recla 19553 root txt unknown /proc/19553/exe (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-recla 19553 root NOFD /proc/19553/fd (opendir: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-block 19554 root cwd unknown /proc/19554/cwd (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-block 19554 root rtd unknown /proc/19554/root (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-block 19554 root txt unknown /proc/19554/exe (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-block 19554 root NOFD /proc/19554/fd (opendir: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-inode 19555 root cwd unknown /proc/19555/cwd (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-inode 19555 root rtd unknown /proc/19555/root (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-inode 19555 root txt unknown /proc/19555/exe (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-inode 19555 root NOFD /proc/19555/fd (opendir: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-log/d 19556 root cwd unknown /proc/19556/cwd (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-log/d 19556 root rtd unknown /proc/19556/root (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-log/d 19556 root txt unknown /proc/19556/exe (readlink: Permission denied) 2024-04-25T00:24:20.811 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-log/d 19556 root NOFD /proc/19556/fd (opendir: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-cil/d 19557 root cwd unknown /proc/19557/cwd (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-cil/d 19557 root rtd unknown /proc/19557/root (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-cil/d 19557 root txt unknown /proc/19557/exe (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfs-cil/d 19557 root NOFD /proc/19557/fd (opendir: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfsaild/d 19558 root cwd unknown /proc/19558/cwd (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfsaild/d 19558 root rtd unknown /proc/19558/root (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfsaild/d 19558 root txt unknown /proc/19558/exe (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:xfsaild/d 19558 root NOFD /proc/19558/fd (opendir: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20486 root cwd unknown /proc/20486/cwd (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20486 root rtd unknown /proc/20486/root (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20486 root txt unknown /proc/20486/exe (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20486 root NOFD /proc/20486/fd (opendir: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20489 ubuntu cwd unknown /proc/20489/cwd (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20489 ubuntu rtd unknown /proc/20489/root (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20489 ubuntu txt unknown /proc/20489/exe (readlink: Permission denied) 2024-04-25T00:24:20.812 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 20489 ubuntu NOFD /proc/20489/fd (opendir: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:chronyd 20549 chrony cwd unknown /proc/20549/cwd (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:chronyd 20549 chrony rtd unknown /proc/20549/root (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:chronyd 20549 chrony txt unknown /proc/20549/exe (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:chronyd 20549 chrony NOFD /proc/20549/fd (opendir: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-cras 20838 ceph cwd unknown /proc/20838/cwd (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-cras 20838 ceph rtd unknown /proc/20838/root (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-cras 20838 ceph txt unknown /proc/20838/exe (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-cras 20838 ceph NOFD /proc/20838/fd (opendir: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21863 root cwd unknown /proc/21863/cwd (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21863 root rtd unknown /proc/21863/root (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21863 root txt unknown /proc/21863/exe (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21863 root NOFD /proc/21863/fd (opendir: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21864 root cwd unknown /proc/21864/cwd (readlink: Permission denied) 2024-04-25T00:24:20.813 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21864 root rtd unknown /proc/21864/root (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21864 root txt unknown /proc/21864/exe (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 21864 root NOFD /proc/21864/fd (opendir: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 21957 root cwd unknown /proc/21957/cwd (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 21957 root rtd unknown /proc/21957/root (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 21957 root txt unknown /proc/21957/exe (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 21957 root NOFD /proc/21957/fd (opendir: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 28060 root cwd unknown /proc/28060/cwd (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 28060 root rtd unknown /proc/28060/root (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 28060 root txt unknown /proc/28060/exe (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 28060 root NOFD /proc/28060/fd (opendir: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28176 root cwd unknown /proc/28176/cwd (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28176 root rtd unknown /proc/28176/root (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28176 root txt unknown /proc/28176/exe (readlink: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28176 root NOFD /proc/28176/fd (opendir: Permission denied) 2024-04-25T00:24:20.814 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28193 root cwd unknown /proc/28193/cwd (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28193 root rtd unknown /proc/28193/root (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28193 root txt unknown /proc/28193/exe (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28193 root NOFD /proc/28193/fd (opendir: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28194 root cwd unknown /proc/28194/cwd (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28194 root rtd unknown /proc/28194/root (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28194 root txt unknown /proc/28194/exe (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 28194 root NOFD /proc/28194/fd (opendir: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28211 root cwd unknown /proc/28211/cwd (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28211 root rtd unknown /proc/28211/root (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28211 root txt unknown /proc/28211/exe (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 28211 root NOFD /proc/28211/fd (opendir: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 28336 root cwd unknown /proc/28336/cwd (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 28336 root rtd unknown /proc/28336/root (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 28336 root txt unknown /proc/28336/exe (readlink: Permission denied) 2024-04-25T00:24:20.815 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 28336 root NOFD /proc/28336/fd (opendir: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29190 root cwd unknown /proc/29190/cwd (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29190 root rtd unknown /proc/29190/root (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29190 root txt unknown /proc/29190/exe (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29190 root NOFD /proc/29190/fd (opendir: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29192 root cwd unknown /proc/29192/cwd (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29192 root rtd unknown /proc/29192/root (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29192 root txt unknown /proc/29192/exe (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29192 root NOFD /proc/29192/fd (opendir: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 ceph cwd unknown /proc/29194/cwd (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 ceph rtd unknown /proc/29194/root (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 ceph txt unknown /proc/29194/exe (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 ceph NOFD /proc/29194/fd (opendir: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29195 log ceph cwd unknown /proc/29194/task/29195/cwd (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29195 log ceph rtd unknown /proc/29194/task/29195/root (readlink: Permission denied) 2024-04-25T00:24:20.816 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29195 log ceph txt unknown /proc/29194/task/29195/exe (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29195 log ceph NOFD /proc/29194/task/29195/fd (opendir: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29196 service ceph cwd unknown /proc/29194/task/29196/cwd (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29196 service ceph rtd unknown /proc/29194/task/29196/root (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29196 service ceph txt unknown /proc/29194/task/29196/exe (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29196 service ceph NOFD /proc/29194/task/29196/fd (opendir: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29197 admin_soc ceph cwd unknown /proc/29194/task/29197/cwd (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29197 admin_soc ceph rtd unknown /proc/29194/task/29197/root (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29197 admin_soc ceph txt unknown /proc/29194/task/29197/exe (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29197 admin_soc ceph NOFD /proc/29194/task/29197/fd (opendir: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29198 signal_ha ceph cwd unknown /proc/29194/task/29198/cwd (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29198 signal_ha ceph rtd unknown /proc/29194/task/29198/root (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29198 signal_ha ceph txt unknown /proc/29194/task/29198/exe (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29198 signal_ha ceph NOFD /proc/29194/task/29198/fd (opendir: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29199 rocksdb:l ceph cwd unknown /proc/29194/task/29199/cwd (readlink: Permission denied) 2024-04-25T00:24:20.817 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29199 rocksdb:l ceph rtd unknown /proc/29194/task/29199/root (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29199 rocksdb:l ceph txt unknown /proc/29194/task/29199/exe (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29199 rocksdb:l ceph NOFD /proc/29194/task/29199/fd (opendir: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29200 rocksdb:h ceph cwd unknown /proc/29194/task/29200/cwd (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29200 rocksdb:h ceph rtd unknown /proc/29194/task/29200/root (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29200 rocksdb:h ceph txt unknown /proc/29194/task/29200/exe (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29200 rocksdb:h ceph NOFD /proc/29194/task/29200/fd (opendir: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29216 ceph-mon ceph cwd unknown /proc/29194/task/29216/cwd (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29216 ceph-mon ceph rtd unknown /proc/29194/task/29216/root (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29216 ceph-mon ceph txt unknown /proc/29194/task/29216/exe (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29216 ceph-mon ceph NOFD /proc/29194/task/29216/fd (opendir: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29217 fn_monsto ceph cwd unknown /proc/29194/task/29217/cwd (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29217 fn_monsto ceph rtd unknown /proc/29194/task/29217/root (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29217 fn_monsto ceph txt unknown /proc/29194/task/29217/exe (readlink: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29217 fn_monsto ceph NOFD /proc/29194/task/29217/fd (opendir: Permission denied) 2024-04-25T00:24:20.818 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29218 msgr-work ceph cwd unknown /proc/29194/task/29218/cwd (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29218 msgr-work ceph rtd unknown /proc/29194/task/29218/root (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29218 msgr-work ceph txt unknown /proc/29194/task/29218/exe (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29218 msgr-work ceph NOFD /proc/29194/task/29218/fd (opendir: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29219 msgr-work ceph cwd unknown /proc/29194/task/29219/cwd (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29219 msgr-work ceph rtd unknown /proc/29194/task/29219/root (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29219 msgr-work ceph txt unknown /proc/29194/task/29219/exe (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29219 msgr-work ceph NOFD /proc/29194/task/29219/fd (opendir: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29220 msgr-work ceph cwd unknown /proc/29194/task/29220/cwd (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29220 msgr-work ceph rtd unknown /proc/29194/task/29220/root (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29220 msgr-work ceph txt unknown /proc/29194/task/29220/exe (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29220 msgr-work ceph NOFD /proc/29194/task/29220/fd (opendir: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29221 OpHistory ceph cwd unknown /proc/29194/task/29221/cwd (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29221 OpHistory ceph rtd unknown /proc/29194/task/29221/root (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29221 OpHistory ceph txt unknown /proc/29194/task/29221/exe (readlink: Permission denied) 2024-04-25T00:24:20.819 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29221 OpHistory ceph NOFD /proc/29194/task/29221/fd (opendir: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29222 fin ceph cwd unknown /proc/29194/task/29222/cwd (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29222 fin ceph rtd unknown /proc/29194/task/29222/root (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29222 fin ceph txt unknown /proc/29194/task/29222/exe (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29222 fin ceph NOFD /proc/29194/task/29222/fd (opendir: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29223 safe_time ceph cwd unknown /proc/29194/task/29223/cwd (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29223 safe_time ceph rtd unknown /proc/29194/task/29223/root (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29223 safe_time ceph txt unknown /proc/29194/task/29223/exe (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29223 safe_time ceph NOFD /proc/29194/task/29223/fd (opendir: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29224 cpu_tp ceph cwd unknown /proc/29194/task/29224/cwd (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29224 cpu_tp ceph rtd unknown /proc/29194/task/29224/root (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29224 cpu_tp ceph txt unknown /proc/29194/task/29224/exe (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29224 cpu_tp ceph NOFD /proc/29194/task/29224/fd (opendir: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29225 cpu_tp ceph cwd unknown /proc/29194/task/29225/cwd (readlink: Permission denied) 2024-04-25T00:24:20.820 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29225 cpu_tp ceph rtd unknown /proc/29194/task/29225/root (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29225 cpu_tp ceph txt unknown /proc/29194/task/29225/exe (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29225 cpu_tp ceph NOFD /proc/29194/task/29225/fd (opendir: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29226 cpu_tp ceph cwd unknown /proc/29194/task/29226/cwd (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29226 cpu_tp ceph rtd unknown /proc/29194/task/29226/root (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29226 cpu_tp ceph txt unknown /proc/29194/task/29226/exe (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29226 cpu_tp ceph NOFD /proc/29194/task/29226/fd (opendir: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29227 cpu_tp ceph cwd unknown /proc/29194/task/29227/cwd (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29227 cpu_tp ceph rtd unknown /proc/29194/task/29227/root (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29227 cpu_tp ceph txt unknown /proc/29194/task/29227/exe (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29227 cpu_tp ceph NOFD /proc/29194/task/29227/fd (opendir: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29228 ms_dispat ceph cwd unknown /proc/29194/task/29228/cwd (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29228 ms_dispat ceph rtd unknown /proc/29194/task/29228/root (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29228 ms_dispat ceph txt unknown /proc/29194/task/29228/exe (readlink: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29228 ms_dispat ceph NOFD /proc/29194/task/29228/fd (opendir: Permission denied) 2024-04-25T00:24:20.821 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29229 ms_local ceph cwd unknown /proc/29194/task/29229/cwd (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29229 ms_local ceph rtd unknown /proc/29194/task/29229/root (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29229 ms_local ceph txt unknown /proc/29194/task/29229/exe (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29229 ms_local ceph NOFD /proc/29194/task/29229/fd (opendir: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29230 safe_time ceph cwd unknown /proc/29194/task/29230/cwd (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29230 safe_time ceph rtd unknown /proc/29194/task/29230/root (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29230 safe_time ceph txt unknown /proc/29194/task/29230/exe (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29230 safe_time ceph NOFD /proc/29194/task/29230/fd (opendir: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29231 ms_dispat ceph cwd unknown /proc/29194/task/29231/cwd (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29231 ms_dispat ceph rtd unknown /proc/29194/task/29231/root (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29231 ms_dispat ceph txt unknown /proc/29194/task/29231/exe (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29231 ms_dispat ceph NOFD /proc/29194/task/29231/fd (opendir: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29232 ms_local ceph cwd unknown /proc/29194/task/29232/cwd (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29232 ms_local ceph rtd unknown /proc/29194/task/29232/root (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29232 ms_local ceph txt unknown /proc/29194/task/29232/exe (readlink: Permission denied) 2024-04-25T00:24:20.822 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 29232 ms_local ceph NOFD /proc/29194/task/29232/fd (opendir: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 66168 rstore_co ceph cwd unknown /proc/29194/task/66168/cwd (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 66168 rstore_co ceph rtd unknown /proc/29194/task/66168/root (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 66168 rstore_co ceph txt unknown /proc/29194/task/66168/exe (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mon 29194 66168 rstore_co ceph NOFD /proc/29194/task/66168/fd (opendir: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29425 root cwd unknown /proc/29425/cwd (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29425 root rtd unknown /proc/29425/root (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29425 root txt unknown /proc/29425/exe (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 29425 root NOFD /proc/29425/fd (opendir: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29427 root cwd unknown /proc/29427/cwd (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29427 root rtd unknown /proc/29427/root (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29427 root txt unknown /proc/29427/exe (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 29427 root NOFD /proc/29427/fd (opendir: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 ceph cwd unknown /proc/29429/cwd (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 ceph rtd unknown /proc/29429/root (readlink: Permission denied) 2024-04-25T00:24:20.823 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 ceph txt unknown /proc/29429/exe (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 ceph NOFD /proc/29429/fd (opendir: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80686 log ceph cwd unknown /proc/29429/task/80686/cwd (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80686 log ceph rtd unknown /proc/29429/task/80686/root (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80686 log ceph txt unknown /proc/29429/task/80686/exe (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80686 log ceph NOFD /proc/29429/task/80686/fd (opendir: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80688 msgr-work ceph cwd unknown /proc/29429/task/80688/cwd (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80688 msgr-work ceph rtd unknown /proc/29429/task/80688/root (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80688 msgr-work ceph txt unknown /proc/29429/task/80688/exe (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80688 msgr-work ceph NOFD /proc/29429/task/80688/fd (opendir: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80689 msgr-work ceph cwd unknown /proc/29429/task/80689/cwd (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80689 msgr-work ceph rtd unknown /proc/29429/task/80689/root (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80689 msgr-work ceph txt unknown /proc/29429/task/80689/exe (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80689 msgr-work ceph NOFD /proc/29429/task/80689/fd (opendir: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80690 msgr-work ceph cwd unknown /proc/29429/task/80690/cwd (readlink: Permission denied) 2024-04-25T00:24:20.824 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80690 msgr-work ceph rtd unknown /proc/29429/task/80690/root (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80690 msgr-work ceph txt unknown /proc/29429/task/80690/exe (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80690 msgr-work ceph NOFD /proc/29429/task/80690/fd (opendir: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80694 service ceph cwd unknown /proc/29429/task/80694/cwd (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80694 service ceph rtd unknown /proc/29429/task/80694/root (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80694 service ceph txt unknown /proc/29429/task/80694/exe (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80694 service ceph NOFD /proc/29429/task/80694/fd (opendir: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80695 admin_soc ceph cwd unknown /proc/29429/task/80695/cwd (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80695 admin_soc ceph rtd unknown /proc/29429/task/80695/root (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80695 admin_soc ceph txt unknown /proc/29429/task/80695/exe (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80695 admin_soc ceph NOFD /proc/29429/task/80695/fd (opendir: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80696 ceph_time ceph cwd unknown /proc/29429/task/80696/cwd (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80696 ceph_time ceph rtd unknown /proc/29429/task/80696/root (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80696 ceph_time ceph txt unknown /proc/29429/task/80696/exe (readlink: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80696 ceph_time ceph NOFD /proc/29429/task/80696/fd (opendir: Permission denied) 2024-04-25T00:24:20.825 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80697 fn_anonym ceph cwd unknown /proc/29429/task/80697/cwd (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80697 fn_anonym ceph rtd unknown /proc/29429/task/80697/root (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80697 fn_anonym ceph txt unknown /proc/29429/task/80697/exe (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80697 fn_anonym ceph NOFD /proc/29429/task/80697/fd (opendir: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80698 signal_ha ceph cwd unknown /proc/29429/task/80698/cwd (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80698 signal_ha ceph rtd unknown /proc/29429/task/80698/root (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80698 signal_ha ceph txt unknown /proc/29429/task/80698/exe (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80698 signal_ha ceph NOFD /proc/29429/task/80698/fd (opendir: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80699 mgrsb-fin ceph cwd unknown /proc/29429/task/80699/cwd (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80699 mgrsb-fin ceph rtd unknown /proc/29429/task/80699/root (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80699 mgrsb-fin ceph txt unknown /proc/29429/task/80699/exe (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80699 mgrsb-fin ceph NOFD /proc/29429/task/80699/fd (opendir: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80700 ms_dispat ceph cwd unknown /proc/29429/task/80700/cwd (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80700 ms_dispat ceph rtd unknown /proc/29429/task/80700/root (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80700 ms_dispat ceph txt unknown /proc/29429/task/80700/exe (readlink: Permission denied) 2024-04-25T00:24:20.826 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80700 ms_dispat ceph NOFD /proc/29429/task/80700/fd (opendir: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80701 ms_local ceph cwd unknown /proc/29429/task/80701/cwd (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80701 ms_local ceph rtd unknown /proc/29429/task/80701/root (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80701 ms_local ceph txt unknown /proc/29429/task/80701/exe (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80701 ms_local ceph NOFD /proc/29429/task/80701/fd (opendir: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80702 io_contex ceph cwd unknown /proc/29429/task/80702/cwd (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80702 io_contex ceph rtd unknown /proc/29429/task/80702/root (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80702 io_contex ceph txt unknown /proc/29429/task/80702/exe (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80702 io_contex ceph NOFD /proc/29429/task/80702/fd (opendir: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80703 io_contex ceph cwd unknown /proc/29429/task/80703/cwd (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80703 io_contex ceph rtd unknown /proc/29429/task/80703/root (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80703 io_contex ceph txt unknown /proc/29429/task/80703/exe (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80703 io_contex ceph NOFD /proc/29429/task/80703/fd (opendir: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80704 safe_time ceph cwd unknown /proc/29429/task/80704/cwd (readlink: Permission denied) 2024-04-25T00:24:20.827 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80704 safe_time ceph rtd unknown /proc/29429/task/80704/root (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80704 safe_time ceph txt unknown /proc/29429/task/80704/exe (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80704 safe_time ceph NOFD /proc/29429/task/80704/fd (opendir: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80705 safe_time ceph cwd unknown /proc/29429/task/80705/cwd (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80705 safe_time ceph rtd unknown /proc/29429/task/80705/root (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80705 safe_time ceph txt unknown /proc/29429/task/80705/exe (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80705 safe_time ceph NOFD /proc/29429/task/80705/fd (opendir: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80706 safe_time ceph cwd unknown /proc/29429/task/80706/cwd (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80706 safe_time ceph rtd unknown /proc/29429/task/80706/root (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80706 safe_time ceph txt unknown /proc/29429/task/80706/exe (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80706 safe_time ceph NOFD /proc/29429/task/80706/fd (opendir: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80707 fn_anonym ceph cwd unknown /proc/29429/task/80707/cwd (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80707 fn_anonym ceph rtd unknown /proc/29429/task/80707/root (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80707 fn_anonym ceph txt unknown /proc/29429/task/80707/exe (readlink: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80707 fn_anonym ceph NOFD /proc/29429/task/80707/fd (opendir: Permission denied) 2024-04-25T00:24:20.828 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80708 flusher ceph cwd unknown /proc/29429/task/80708/cwd (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80708 flusher ceph rtd unknown /proc/29429/task/80708/root (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80708 flusher ceph txt unknown /proc/29429/task/80708/exe (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80708 flusher ceph NOFD /proc/29429/task/80708/fd (opendir: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80709 safe_time ceph cwd unknown /proc/29429/task/80709/cwd (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80709 safe_time ceph rtd unknown /proc/29429/task/80709/root (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80709 safe_time ceph txt unknown /proc/29429/task/80709/exe (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 80709 safe_time ceph NOFD /proc/29429/task/80709/fd (opendir: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81731 mgr-fin ceph cwd unknown /proc/29429/task/81731/cwd (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81731 mgr-fin ceph rtd unknown /proc/29429/task/81731/root (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81731 mgr-fin ceph txt unknown /proc/29429/task/81731/exe (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81731 mgr-fin ceph NOFD /proc/29429/task/81731/fd (opendir: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81732 ms_dispat ceph cwd unknown /proc/29429/task/81732/cwd (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81732 ms_dispat ceph rtd unknown /proc/29429/task/81732/root (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81732 ms_dispat ceph txt unknown /proc/29429/task/81732/exe (readlink: Permission denied) 2024-04-25T00:24:20.829 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81732 ms_dispat ceph NOFD /proc/29429/task/81732/fd (opendir: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81733 ms_local ceph cwd unknown /proc/29429/task/81733/cwd (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81733 ms_local ceph rtd unknown /proc/29429/task/81733/root (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81733 ms_local ceph txt unknown /proc/29429/task/81733/exe (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81733 ms_local ceph NOFD /proc/29429/task/81733/fd (opendir: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81734 safe_time ceph cwd unknown /proc/29429/task/81734/cwd (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81734 safe_time ceph rtd unknown /proc/29429/task/81734/root (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81734 safe_time ceph txt unknown /proc/29429/task/81734/exe (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81734 safe_time ceph NOFD /proc/29429/task/81734/fd (opendir: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81735 cmdfin ceph cwd unknown /proc/29429/task/81735/cwd (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81735 cmdfin ceph rtd unknown /proc/29429/task/81735/root (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81735 cmdfin ceph txt unknown /proc/29429/task/81735/exe (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81735 cmdfin ceph NOFD /proc/29429/task/81735/fd (opendir: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81739 io_contex ceph cwd unknown /proc/29429/task/81739/cwd (readlink: Permission denied) 2024-04-25T00:24:20.830 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81739 io_contex ceph rtd unknown /proc/29429/task/81739/root (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81739 io_contex ceph txt unknown /proc/29429/task/81739/exe (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81739 io_contex ceph NOFD /proc/29429/task/81739/fd (opendir: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81740 io_contex ceph cwd unknown /proc/29429/task/81740/cwd (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81740 io_contex ceph rtd unknown /proc/29429/task/81740/root (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81740 io_contex ceph txt unknown /proc/29429/task/81740/exe (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81740 io_contex ceph NOFD /proc/29429/task/81740/fd (opendir: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81741 ceph_time ceph cwd unknown /proc/29429/task/81741/cwd (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81741 ceph_time ceph rtd unknown /proc/29429/task/81741/root (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81741 ceph_time ceph txt unknown /proc/29429/task/81741/exe (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81741 ceph_time ceph NOFD /proc/29429/task/81741/fd (opendir: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81742 ms_dispat ceph cwd unknown /proc/29429/task/81742/cwd (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81742 ms_dispat ceph rtd unknown /proc/29429/task/81742/root (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81742 ms_dispat ceph txt unknown /proc/29429/task/81742/exe (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81742 ms_dispat ceph NOFD /proc/29429/task/81742/fd (opendir: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81743 ms_local ceph cwd unknown /proc/29429/task/81743/cwd (readlink: Permission denied) 2024-04-25T00:24:20.831 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81743 ms_local ceph rtd unknown /proc/29429/task/81743/root (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81743 ms_local ceph txt unknown /proc/29429/task/81743/exe (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81743 ms_local ceph NOFD /proc/29429/task/81743/fd (opendir: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81744 safe_time ceph cwd unknown /proc/29429/task/81744/cwd (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81744 safe_time ceph rtd unknown /proc/29429/task/81744/root (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81744 safe_time ceph txt unknown /proc/29429/task/81744/exe (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81744 safe_time ceph NOFD /proc/29429/task/81744/fd (opendir: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81745 safe_time ceph cwd unknown /proc/29429/task/81745/cwd (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81745 safe_time ceph rtd unknown /proc/29429/task/81745/root (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81745 safe_time ceph txt unknown /proc/29429/task/81745/exe (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81745 safe_time ceph NOFD /proc/29429/task/81745/fd (opendir: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81746 balancer ceph cwd unknown /proc/29429/task/81746/cwd (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81746 balancer ceph rtd unknown /proc/29429/task/81746/root (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81746 balancer ceph txt unknown /proc/29429/task/81746/exe (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81746 balancer ceph NOFD /proc/29429/task/81746/fd (opendir: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81747 m-fin-bal ceph cwd unknown /proc/29429/task/81747/cwd (readlink: Permission denied) 2024-04-25T00:24:20.832 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81747 m-fin-bal ceph rtd unknown /proc/29429/task/81747/root (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81747 m-fin-bal ceph txt unknown /proc/29429/task/81747/exe (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81747 m-fin-bal ceph NOFD /proc/29429/task/81747/fd (opendir: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81767 mgr-fin ceph cwd unknown /proc/29429/task/81767/cwd (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81767 mgr-fin ceph rtd unknown /proc/29429/task/81767/root (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81767 mgr-fin ceph txt unknown /proc/29429/task/81767/exe (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81767 mgr-fin ceph NOFD /proc/29429/task/81767/fd (opendir: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81768 mgr-fin ceph cwd unknown /proc/29429/task/81768/cwd (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81768 mgr-fin ceph rtd unknown /proc/29429/task/81768/root (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81768 mgr-fin ceph txt unknown /proc/29429/task/81768/exe (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81768 mgr-fin ceph NOFD /proc/29429/task/81768/fd (opendir: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81769 mgr-fin ceph cwd unknown /proc/29429/task/81769/cwd (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81769 mgr-fin ceph rtd unknown /proc/29429/task/81769/root (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81769 mgr-fin ceph txt unknown /proc/29429/task/81769/exe (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81769 mgr-fin ceph NOFD /proc/29429/task/81769/fd (opendir: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81770 mgr-fin ceph cwd unknown /proc/29429/task/81770/cwd (readlink: Permission denied) 2024-04-25T00:24:20.833 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81770 mgr-fin ceph rtd unknown /proc/29429/task/81770/root (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81770 mgr-fin ceph txt unknown /proc/29429/task/81770/exe (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81770 mgr-fin ceph NOFD /proc/29429/task/81770/fd (opendir: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81771 mgr-fin ceph cwd unknown /proc/29429/task/81771/cwd (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81771 mgr-fin ceph rtd unknown /proc/29429/task/81771/root (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81771 mgr-fin ceph txt unknown /proc/29429/task/81771/exe (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81771 mgr-fin ceph NOFD /proc/29429/task/81771/fd (opendir: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81772 mgr-fin ceph cwd unknown /proc/29429/task/81772/cwd (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81772 mgr-fin ceph rtd unknown /proc/29429/task/81772/root (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81772 mgr-fin ceph txt unknown /proc/29429/task/81772/exe (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81772 mgr-fin ceph NOFD /proc/29429/task/81772/fd (opendir: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81773 mgr-fin ceph cwd unknown /proc/29429/task/81773/cwd (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81773 mgr-fin ceph rtd unknown /proc/29429/task/81773/root (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81773 mgr-fin ceph txt unknown /proc/29429/task/81773/exe (readlink: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81773 mgr-fin ceph NOFD /proc/29429/task/81773/fd (opendir: Permission denied) 2024-04-25T00:24:20.834 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81774 mgr-fin ceph cwd unknown /proc/29429/task/81774/cwd (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81774 mgr-fin ceph rtd unknown /proc/29429/task/81774/root (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81774 mgr-fin ceph txt unknown /proc/29429/task/81774/exe (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81774 mgr-fin ceph NOFD /proc/29429/task/81774/fd (opendir: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81775 mgr-fin ceph cwd unknown /proc/29429/task/81775/cwd (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81775 mgr-fin ceph rtd unknown /proc/29429/task/81775/root (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81775 mgr-fin ceph txt unknown /proc/29429/task/81775/exe (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81775 mgr-fin ceph NOFD /proc/29429/task/81775/fd (opendir: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81776 mgr-fin ceph cwd unknown /proc/29429/task/81776/cwd (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81776 mgr-fin ceph rtd unknown /proc/29429/task/81776/root (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81776 mgr-fin ceph txt unknown /proc/29429/task/81776/exe (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81776 mgr-fin ceph NOFD /proc/29429/task/81776/fd (opendir: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81777 mgr-fin ceph cwd unknown /proc/29429/task/81777/cwd (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81777 mgr-fin ceph rtd unknown /proc/29429/task/81777/root (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81777 mgr-fin ceph txt unknown /proc/29429/task/81777/exe (readlink: Permission denied) 2024-04-25T00:24:20.835 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81777 mgr-fin ceph NOFD /proc/29429/task/81777/fd (opendir: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81778 mgr-fin ceph cwd unknown /proc/29429/task/81778/cwd (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81778 mgr-fin ceph rtd unknown /proc/29429/task/81778/root (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81778 mgr-fin ceph txt unknown /proc/29429/task/81778/exe (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81778 mgr-fin ceph NOFD /proc/29429/task/81778/fd (opendir: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81779 mgr-fin ceph cwd unknown /proc/29429/task/81779/cwd (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81779 mgr-fin ceph rtd unknown /proc/29429/task/81779/root (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81779 mgr-fin ceph txt unknown /proc/29429/task/81779/exe (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81779 mgr-fin ceph NOFD /proc/29429/task/81779/fd (opendir: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81780 mgr-fin ceph cwd unknown /proc/29429/task/81780/cwd (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81780 mgr-fin ceph rtd unknown /proc/29429/task/81780/root (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81780 mgr-fin ceph txt unknown /proc/29429/task/81780/exe (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81780 mgr-fin ceph NOFD /proc/29429/task/81780/fd (opendir: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81781 mgr-fin ceph cwd unknown /proc/29429/task/81781/cwd (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81781 mgr-fin ceph rtd unknown /proc/29429/task/81781/root (readlink: Permission denied) 2024-04-25T00:24:20.836 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81781 mgr-fin ceph txt unknown /proc/29429/task/81781/exe (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81781 mgr-fin ceph NOFD /proc/29429/task/81781/fd (opendir: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81782 mgr-fin ceph cwd unknown /proc/29429/task/81782/cwd (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81782 mgr-fin ceph rtd unknown /proc/29429/task/81782/root (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81782 mgr-fin ceph txt unknown /proc/29429/task/81782/exe (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81782 mgr-fin ceph NOFD /proc/29429/task/81782/fd (opendir: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81783 cephadm ceph cwd unknown /proc/29429/task/81783/cwd (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81783 cephadm ceph rtd unknown /proc/29429/task/81783/root (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81783 cephadm ceph txt unknown /proc/29429/task/81783/exe (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81783 cephadm ceph NOFD /proc/29429/task/81783/fd (opendir: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81784 m-fin-cep ceph cwd unknown /proc/29429/task/81784/cwd (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81784 m-fin-cep ceph rtd unknown /proc/29429/task/81784/root (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81784 m-fin-cep ceph txt unknown /proc/29429/task/81784/exe (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81784 m-fin-cep ceph NOFD /proc/29429/task/81784/fd (opendir: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81785 mgr-fin ceph cwd unknown /proc/29429/task/81785/cwd (readlink: Permission denied) 2024-04-25T00:24:20.837 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81785 mgr-fin ceph rtd unknown /proc/29429/task/81785/root (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81785 mgr-fin ceph txt unknown /proc/29429/task/81785/exe (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81785 mgr-fin ceph NOFD /proc/29429/task/81785/fd (opendir: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81786 cephadm ceph cwd unknown /proc/29429/task/81786/cwd (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81786 cephadm ceph rtd unknown /proc/29429/task/81786/root (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81786 cephadm ceph txt unknown /proc/29429/task/81786/exe (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81786 cephadm ceph NOFD /proc/29429/task/81786/fd (opendir: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81787 mgr-fin ceph cwd unknown /proc/29429/task/81787/cwd (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81787 mgr-fin ceph rtd unknown /proc/29429/task/81787/root (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81787 mgr-fin ceph txt unknown /proc/29429/task/81787/exe (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81787 mgr-fin ceph NOFD /proc/29429/task/81787/fd (opendir: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81788 mgr-fin ceph cwd unknown /proc/29429/task/81788/cwd (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81788 mgr-fin ceph rtd unknown /proc/29429/task/81788/root (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81788 mgr-fin ceph txt unknown /proc/29429/task/81788/exe (readlink: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81788 mgr-fin ceph NOFD /proc/29429/task/81788/fd (opendir: Permission denied) 2024-04-25T00:24:20.838 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81789 crash ceph cwd unknown /proc/29429/task/81789/cwd (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81789 crash ceph rtd unknown /proc/29429/task/81789/root (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81789 crash ceph txt unknown /proc/29429/task/81789/exe (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81789 crash ceph NOFD /proc/29429/task/81789/fd (opendir: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81790 m-fin-cra ceph cwd unknown /proc/29429/task/81790/cwd (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81790 m-fin-cra ceph rtd unknown /proc/29429/task/81790/root (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81790 m-fin-cra ceph txt unknown /proc/29429/task/81790/exe (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81790 m-fin-cra ceph NOFD /proc/29429/task/81790/fd (opendir: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81791 dashboard ceph cwd unknown /proc/29429/task/81791/cwd (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81791 dashboard ceph rtd unknown /proc/29429/task/81791/root (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81791 dashboard ceph txt unknown /proc/29429/task/81791/exe (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81791 dashboard ceph NOFD /proc/29429/task/81791/fd (opendir: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81792 m-fin-das ceph cwd unknown /proc/29429/task/81792/cwd (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81792 m-fin-das ceph rtd unknown /proc/29429/task/81792/root (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81792 m-fin-das ceph txt unknown /proc/29429/task/81792/exe (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81792 m-fin-das ceph NOFD /proc/29429/task/81792/fd (opendir: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81793 mgr-fin ceph cwd unknown /proc/29429/task/81793/cwd (readlink: Permission denied) 2024-04-25T00:24:20.839 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81793 mgr-fin ceph rtd unknown /proc/29429/task/81793/root (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81793 mgr-fin ceph txt unknown /proc/29429/task/81793/exe (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81793 mgr-fin ceph NOFD /proc/29429/task/81793/fd (opendir: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81794 mgr-fin ceph cwd unknown /proc/29429/task/81794/cwd (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81794 mgr-fin ceph rtd unknown /proc/29429/task/81794/root (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81794 mgr-fin ceph txt unknown /proc/29429/task/81794/exe (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81794 mgr-fin ceph NOFD /proc/29429/task/81794/fd (opendir: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81795 mgr-fin ceph cwd unknown /proc/29429/task/81795/cwd (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81795 mgr-fin ceph rtd unknown /proc/29429/task/81795/root (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81795 mgr-fin ceph txt unknown /proc/29429/task/81795/exe (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81795 mgr-fin ceph NOFD /proc/29429/task/81795/fd (opendir: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81796 mgr-fin ceph cwd unknown /proc/29429/task/81796/cwd (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81796 mgr-fin ceph rtd unknown /proc/29429/task/81796/root (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81796 mgr-fin ceph txt unknown /proc/29429/task/81796/exe (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81796 mgr-fin ceph NOFD /proc/29429/task/81796/fd (opendir: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81797 devicehea ceph cwd unknown /proc/29429/task/81797/cwd (readlink: Permission denied) 2024-04-25T00:24:20.840 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81797 devicehea ceph rtd unknown /proc/29429/task/81797/root (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81797 devicehea ceph txt unknown /proc/29429/task/81797/exe (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81797 devicehea ceph NOFD /proc/29429/task/81797/fd (opendir: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81798 m-fin-dev ceph cwd unknown /proc/29429/task/81798/cwd (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81798 m-fin-dev ceph rtd unknown /proc/29429/task/81798/root (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81798 m-fin-dev ceph txt unknown /proc/29429/task/81798/exe (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81798 m-fin-dev ceph NOFD /proc/29429/task/81798/fd (opendir: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81799 mgr-fin ceph cwd unknown /proc/29429/task/81799/cwd (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81799 mgr-fin ceph rtd unknown /proc/29429/task/81799/root (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81799 mgr-fin ceph txt unknown /proc/29429/task/81799/exe (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81799 mgr-fin ceph NOFD /proc/29429/task/81799/fd (opendir: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81801 m-fin-ios ceph cwd unknown /proc/29429/task/81801/cwd (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81801 m-fin-ios ceph rtd unknown /proc/29429/task/81801/root (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81801 m-fin-ios ceph txt unknown /proc/29429/task/81801/exe (readlink: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81801 m-fin-ios ceph NOFD /proc/29429/task/81801/fd (opendir: Permission denied) 2024-04-25T00:24:20.841 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81802 mgr-fin ceph cwd unknown /proc/29429/task/81802/cwd (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81802 mgr-fin ceph rtd unknown /proc/29429/task/81802/root (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81802 mgr-fin ceph txt unknown /proc/29429/task/81802/exe (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81802 mgr-fin ceph NOFD /proc/29429/task/81802/fd (opendir: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81804 m-fin-nfs ceph cwd unknown /proc/29429/task/81804/cwd (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81804 m-fin-nfs ceph rtd unknown /proc/29429/task/81804/root (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81804 m-fin-nfs ceph txt unknown /proc/29429/task/81804/exe (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81804 m-fin-nfs ceph NOFD /proc/29429/task/81804/fd (opendir: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81805 mgr-fin ceph cwd unknown /proc/29429/task/81805/cwd (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81805 mgr-fin ceph rtd unknown /proc/29429/task/81805/root (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81805 mgr-fin ceph txt unknown /proc/29429/task/81805/exe (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81805 mgr-fin ceph NOFD /proc/29429/task/81805/fd (opendir: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81807 m-fin-orc ceph cwd unknown /proc/29429/task/81807/cwd (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81807 m-fin-orc ceph rtd unknown /proc/29429/task/81807/root (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81807 m-fin-orc ceph txt unknown /proc/29429/task/81807/exe (readlink: Permission denied) 2024-04-25T00:24:20.842 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81807 m-fin-orc ceph NOFD /proc/29429/task/81807/fd (opendir: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81808 pg_autosc ceph cwd unknown /proc/29429/task/81808/cwd (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81808 pg_autosc ceph rtd unknown /proc/29429/task/81808/root (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81808 pg_autosc ceph txt unknown /proc/29429/task/81808/exe (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81808 pg_autosc ceph NOFD /proc/29429/task/81808/fd (opendir: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81809 mgr-fin ceph cwd unknown /proc/29429/task/81809/cwd (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81809 mgr-fin ceph rtd unknown /proc/29429/task/81809/root (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81809 mgr-fin ceph txt unknown /proc/29429/task/81809/exe (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81809 mgr-fin ceph NOFD /proc/29429/task/81809/fd (opendir: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81810 m-fin-pg_ ceph cwd unknown /proc/29429/task/81810/cwd (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81810 m-fin-pg_ ceph rtd unknown /proc/29429/task/81810/root (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81810 m-fin-pg_ ceph txt unknown /proc/29429/task/81810/exe (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81810 m-fin-pg_ ceph NOFD /proc/29429/task/81810/fd (opendir: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81811 progress ceph cwd unknown /proc/29429/task/81811/cwd (readlink: Permission denied) 2024-04-25T00:24:20.843 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81811 progress ceph rtd unknown /proc/29429/task/81811/root (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81811 progress ceph txt unknown /proc/29429/task/81811/exe (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81811 progress ceph NOFD /proc/29429/task/81811/fd (opendir: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81812 m-fin-pro ceph cwd unknown /proc/29429/task/81812/cwd (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81812 m-fin-pro ceph rtd unknown /proc/29429/task/81812/root (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81812 m-fin-pro ceph txt unknown /proc/29429/task/81812/exe (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81812 m-fin-pro ceph NOFD /proc/29429/task/81812/fd (opendir: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81813 mgr-fin ceph cwd unknown /proc/29429/task/81813/cwd (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81813 mgr-fin ceph rtd unknown /proc/29429/task/81813/root (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81813 mgr-fin ceph txt unknown /proc/29429/task/81813/exe (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81813 mgr-fin ceph NOFD /proc/29429/task/81813/fd (opendir: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81817 io_contex ceph cwd unknown /proc/29429/task/81817/cwd (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81817 io_contex ceph rtd unknown /proc/29429/task/81817/root (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81817 io_contex ceph txt unknown /proc/29429/task/81817/exe (readlink: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81817 io_contex ceph NOFD /proc/29429/task/81817/fd (opendir: Permission denied) 2024-04-25T00:24:20.844 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81818 io_contex ceph cwd unknown /proc/29429/task/81818/cwd (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81818 io_contex ceph rtd unknown /proc/29429/task/81818/root (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81818 io_contex ceph txt unknown /proc/29429/task/81818/exe (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81818 io_contex ceph NOFD /proc/29429/task/81818/fd (opendir: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81819 ceph_time ceph cwd unknown /proc/29429/task/81819/cwd (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81819 ceph_time ceph rtd unknown /proc/29429/task/81819/root (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81819 ceph_time ceph txt unknown /proc/29429/task/81819/exe (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81819 ceph_time ceph NOFD /proc/29429/task/81819/fd (opendir: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81820 ms_dispat ceph cwd unknown /proc/29429/task/81820/cwd (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81820 ms_dispat ceph rtd unknown /proc/29429/task/81820/root (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81820 ms_dispat ceph txt unknown /proc/29429/task/81820/exe (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81820 ms_dispat ceph NOFD /proc/29429/task/81820/fd (opendir: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81821 ms_local ceph cwd unknown /proc/29429/task/81821/cwd (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81821 ms_local ceph rtd unknown /proc/29429/task/81821/root (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81821 ms_local ceph txt unknown /proc/29429/task/81821/exe (readlink: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81821 ms_local ceph NOFD /proc/29429/task/81821/fd (opendir: Permission denied) 2024-04-25T00:24:20.845 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81822 safe_time ceph cwd unknown /proc/29429/task/81822/cwd (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81822 safe_time ceph rtd unknown /proc/29429/task/81822/root (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81822 safe_time ceph txt unknown /proc/29429/task/81822/exe (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81822 safe_time ceph NOFD /proc/29429/task/81822/fd (opendir: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81823 safe_time ceph cwd unknown /proc/29429/task/81823/cwd (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81823 safe_time ceph rtd unknown /proc/29429/task/81823/root (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81823 safe_time ceph txt unknown /proc/29429/task/81823/exe (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81823 safe_time ceph NOFD /proc/29429/task/81823/fd (opendir: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81824 mgr-fin ceph cwd unknown /proc/29429/task/81824/cwd (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81824 mgr-fin ceph rtd unknown /proc/29429/task/81824/root (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81824 mgr-fin ceph txt unknown /proc/29429/task/81824/exe (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81824 mgr-fin ceph NOFD /proc/29429/task/81824/fd (opendir: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81826 m-fin-rbd ceph cwd unknown /proc/29429/task/81826/cwd (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81826 m-fin-rbd ceph rtd unknown /proc/29429/task/81826/root (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81826 m-fin-rbd ceph txt unknown /proc/29429/task/81826/exe (readlink: Permission denied) 2024-04-25T00:24:20.846 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81826 m-fin-rbd ceph NOFD /proc/29429/task/81826/fd (opendir: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81827 restful ceph cwd unknown /proc/29429/task/81827/cwd (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81827 restful ceph rtd unknown /proc/29429/task/81827/root (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81827 restful ceph txt unknown /proc/29429/task/81827/exe (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81827 restful ceph NOFD /proc/29429/task/81827/fd (opendir: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81828 m-fin-res ceph cwd unknown /proc/29429/task/81828/cwd (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81828 m-fin-res ceph rtd unknown /proc/29429/task/81828/root (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81828 m-fin-res ceph txt unknown /proc/29429/task/81828/exe (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81828 m-fin-res ceph NOFD /proc/29429/task/81828/fd (opendir: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81829 mgr-fin ceph cwd unknown /proc/29429/task/81829/cwd (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81829 mgr-fin ceph rtd unknown /proc/29429/task/81829/root (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81829 mgr-fin ceph txt unknown /proc/29429/task/81829/exe (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81829 mgr-fin ceph NOFD /proc/29429/task/81829/fd (opendir: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81830 mgr-fin ceph cwd unknown /proc/29429/task/81830/cwd (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81830 mgr-fin ceph rtd unknown /proc/29429/task/81830/root (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81830 mgr-fin ceph txt unknown /proc/29429/task/81830/exe (readlink: Permission denied) 2024-04-25T00:24:20.847 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81830 mgr-fin ceph NOFD /proc/29429/task/81830/fd (opendir: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81831 mgr-fin ceph cwd unknown /proc/29429/task/81831/cwd (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81831 mgr-fin ceph rtd unknown /proc/29429/task/81831/root (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81831 mgr-fin ceph txt unknown /proc/29429/task/81831/exe (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81831 mgr-fin ceph NOFD /proc/29429/task/81831/fd (opendir: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81832 mgr-fin ceph cwd unknown /proc/29429/task/81832/cwd (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81832 mgr-fin ceph rtd unknown /proc/29429/task/81832/root (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81832 mgr-fin ceph txt unknown /proc/29429/task/81832/exe (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81832 mgr-fin ceph NOFD /proc/29429/task/81832/fd (opendir: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81833 mgr-fin ceph cwd unknown /proc/29429/task/81833/cwd (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81833 mgr-fin ceph rtd unknown /proc/29429/task/81833/root (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81833 mgr-fin ceph txt unknown /proc/29429/task/81833/exe (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81833 mgr-fin ceph NOFD /proc/29429/task/81833/fd (opendir: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81834 mgr-fin ceph cwd unknown /proc/29429/task/81834/cwd (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81834 mgr-fin ceph rtd unknown /proc/29429/task/81834/root (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81834 mgr-fin ceph txt unknown /proc/29429/task/81834/exe (readlink: Permission denied) 2024-04-25T00:24:20.848 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81834 mgr-fin ceph NOFD /proc/29429/task/81834/fd (opendir: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81838 io_contex ceph cwd unknown /proc/29429/task/81838/cwd (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81838 io_contex ceph rtd unknown /proc/29429/task/81838/root (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81838 io_contex ceph txt unknown /proc/29429/task/81838/exe (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81838 io_contex ceph NOFD /proc/29429/task/81838/fd (opendir: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81839 io_contex ceph cwd unknown /proc/29429/task/81839/cwd (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81839 io_contex ceph rtd unknown /proc/29429/task/81839/root (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81839 io_contex ceph txt unknown /proc/29429/task/81839/exe (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81839 io_contex ceph NOFD /proc/29429/task/81839/fd (opendir: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81840 ceph_time ceph cwd unknown /proc/29429/task/81840/cwd (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81840 ceph_time ceph rtd unknown /proc/29429/task/81840/root (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81840 ceph_time ceph txt unknown /proc/29429/task/81840/exe (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81840 ceph_time ceph NOFD /proc/29429/task/81840/fd (opendir: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81841 ms_dispat ceph cwd unknown /proc/29429/task/81841/cwd (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81841 ms_dispat ceph rtd unknown /proc/29429/task/81841/root (readlink: Permission denied) 2024-04-25T00:24:20.849 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81841 ms_dispat ceph txt unknown /proc/29429/task/81841/exe (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81841 ms_dispat ceph NOFD /proc/29429/task/81841/fd (opendir: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81842 ms_local ceph cwd unknown /proc/29429/task/81842/cwd (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81842 ms_local ceph rtd unknown /proc/29429/task/81842/root (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81842 ms_local ceph txt unknown /proc/29429/task/81842/exe (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81842 ms_local ceph NOFD /proc/29429/task/81842/fd (opendir: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81843 safe_time ceph cwd unknown /proc/29429/task/81843/cwd (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81843 safe_time ceph rtd unknown /proc/29429/task/81843/root (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81843 safe_time ceph txt unknown /proc/29429/task/81843/exe (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81843 safe_time ceph NOFD /proc/29429/task/81843/fd (opendir: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81844 safe_time ceph cwd unknown /proc/29429/task/81844/cwd (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81844 safe_time ceph rtd unknown /proc/29429/task/81844/root (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81844 safe_time ceph txt unknown /proc/29429/task/81844/exe (readlink: Permission denied) 2024-04-25T00:24:20.850 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81844 safe_time ceph NOFD /proc/29429/task/81844/fd (opendir: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81848 m-fin-sna ceph cwd unknown /proc/29429/task/81848/cwd (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81848 m-fin-sna ceph rtd unknown /proc/29429/task/81848/root (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81848 m-fin-sna ceph txt unknown /proc/29429/task/81848/exe (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81848 m-fin-sna ceph NOFD /proc/29429/task/81848/fd (opendir: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81850 m-fin-sta ceph cwd unknown /proc/29429/task/81850/cwd (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81850 m-fin-sta ceph rtd unknown /proc/29429/task/81850/root (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81850 m-fin-sta ceph txt unknown /proc/29429/task/81850/exe (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81850 m-fin-sta ceph NOFD /proc/29429/task/81850/fd (opendir: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81851 telemetry ceph cwd unknown /proc/29429/task/81851/cwd (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81851 telemetry ceph rtd unknown /proc/29429/task/81851/root (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81851 telemetry ceph txt unknown /proc/29429/task/81851/exe (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81851 telemetry ceph NOFD /proc/29429/task/81851/fd (opendir: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81852 m-fin-tel ceph cwd unknown /proc/29429/task/81852/cwd (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81852 m-fin-tel ceph rtd unknown /proc/29429/task/81852/root (readlink: Permission denied) 2024-04-25T00:24:20.851 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81852 m-fin-tel ceph txt unknown /proc/29429/task/81852/exe (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81852 m-fin-tel ceph NOFD /proc/29429/task/81852/fd (opendir: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81853 mgr-fin ceph cwd unknown /proc/29429/task/81853/cwd (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81853 mgr-fin ceph rtd unknown /proc/29429/task/81853/root (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81853 mgr-fin ceph txt unknown /proc/29429/task/81853/exe (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81853 mgr-fin ceph NOFD /proc/29429/task/81853/fd (opendir: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81857 io_contex ceph cwd unknown /proc/29429/task/81857/cwd (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81857 io_contex ceph rtd unknown /proc/29429/task/81857/root (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81857 io_contex ceph txt unknown /proc/29429/task/81857/exe (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81857 io_contex ceph NOFD /proc/29429/task/81857/fd (opendir: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81858 io_contex ceph cwd unknown /proc/29429/task/81858/cwd (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81858 io_contex ceph rtd unknown /proc/29429/task/81858/root (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81858 io_contex ceph txt unknown /proc/29429/task/81858/exe (readlink: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81858 io_contex ceph NOFD /proc/29429/task/81858/fd (opendir: Permission denied) 2024-04-25T00:24:20.852 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81859 ceph_time ceph cwd unknown /proc/29429/task/81859/cwd (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81859 ceph_time ceph rtd unknown /proc/29429/task/81859/root (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81859 ceph_time ceph txt unknown /proc/29429/task/81859/exe (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81859 ceph_time ceph NOFD /proc/29429/task/81859/fd (opendir: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81860 ms_dispat ceph cwd unknown /proc/29429/task/81860/cwd (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81860 ms_dispat ceph rtd unknown /proc/29429/task/81860/root (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81860 ms_dispat ceph txt unknown /proc/29429/task/81860/exe (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81860 ms_dispat ceph NOFD /proc/29429/task/81860/fd (opendir: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81861 ms_local ceph cwd unknown /proc/29429/task/81861/cwd (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81861 ms_local ceph rtd unknown /proc/29429/task/81861/root (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81861 ms_local ceph txt unknown /proc/29429/task/81861/exe (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81861 ms_local ceph NOFD /proc/29429/task/81861/fd (opendir: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81862 safe_time ceph cwd unknown /proc/29429/task/81862/cwd (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81862 safe_time ceph rtd unknown /proc/29429/task/81862/root (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81862 safe_time ceph txt unknown /proc/29429/task/81862/exe (readlink: Permission denied) 2024-04-25T00:24:20.853 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81862 safe_time ceph NOFD /proc/29429/task/81862/fd (opendir: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81863 safe_time ceph cwd unknown /proc/29429/task/81863/cwd (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81863 safe_time ceph rtd unknown /proc/29429/task/81863/root (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81863 safe_time ceph txt unknown /proc/29429/task/81863/exe (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81863 safe_time ceph NOFD /proc/29429/task/81863/fd (opendir: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81864 mgr-fin ceph cwd unknown /proc/29429/task/81864/cwd (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81864 mgr-fin ceph rtd unknown /proc/29429/task/81864/root (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81864 mgr-fin ceph txt unknown /proc/29429/task/81864/exe (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81864 mgr-fin ceph NOFD /proc/29429/task/81864/fd (opendir: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81865 mgr-fin ceph cwd unknown /proc/29429/task/81865/cwd (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81865 mgr-fin ceph rtd unknown /proc/29429/task/81865/root (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81865 mgr-fin ceph txt unknown /proc/29429/task/81865/exe (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81865 mgr-fin ceph NOFD /proc/29429/task/81865/fd (opendir: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81866 mgr-fin ceph cwd unknown /proc/29429/task/81866/cwd (readlink: Permission denied) 2024-04-25T00:24:20.854 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81866 mgr-fin ceph rtd unknown /proc/29429/task/81866/root (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81866 mgr-fin ceph txt unknown /proc/29429/task/81866/exe (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81866 mgr-fin ceph NOFD /proc/29429/task/81866/fd (opendir: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81867 mgr-fin ceph cwd unknown /proc/29429/task/81867/cwd (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81867 mgr-fin ceph rtd unknown /proc/29429/task/81867/root (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81867 mgr-fin ceph txt unknown /proc/29429/task/81867/exe (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81867 mgr-fin ceph NOFD /proc/29429/task/81867/fd (opendir: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81868 mgr-fin ceph cwd unknown /proc/29429/task/81868/cwd (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81868 mgr-fin ceph rtd unknown /proc/29429/task/81868/root (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81868 mgr-fin ceph txt unknown /proc/29429/task/81868/exe (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81868 mgr-fin ceph NOFD /proc/29429/task/81868/fd (opendir: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81869 mgr-fin ceph cwd unknown /proc/29429/task/81869/cwd (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81869 mgr-fin ceph rtd unknown /proc/29429/task/81869/root (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81869 mgr-fin ceph txt unknown /proc/29429/task/81869/exe (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81869 mgr-fin ceph NOFD /proc/29429/task/81869/fd (opendir: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81870 mgr-fin ceph cwd unknown /proc/29429/task/81870/cwd (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81870 mgr-fin ceph rtd unknown /proc/29429/task/81870/root (readlink: Permission denied) 2024-04-25T00:24:20.855 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81870 mgr-fin ceph txt unknown /proc/29429/task/81870/exe (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81870 mgr-fin ceph NOFD /proc/29429/task/81870/fd (opendir: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81871 mgr-fin ceph cwd unknown /proc/29429/task/81871/cwd (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81871 mgr-fin ceph rtd unknown /proc/29429/task/81871/root (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81871 mgr-fin ceph txt unknown /proc/29429/task/81871/exe (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81871 mgr-fin ceph NOFD /proc/29429/task/81871/fd (opendir: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81872 mgr-fin ceph cwd unknown /proc/29429/task/81872/cwd (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81872 mgr-fin ceph rtd unknown /proc/29429/task/81872/root (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81872 mgr-fin ceph txt unknown /proc/29429/task/81872/exe (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81872 mgr-fin ceph NOFD /proc/29429/task/81872/fd (opendir: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81873 mgr-fin ceph cwd unknown /proc/29429/task/81873/cwd (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81873 mgr-fin ceph rtd unknown /proc/29429/task/81873/root (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81873 mgr-fin ceph txt unknown /proc/29429/task/81873/exe (readlink: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81873 mgr-fin ceph NOFD /proc/29429/task/81873/fd (opendir: Permission denied) 2024-04-25T00:24:20.856 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81874 mgr-fin ceph cwd unknown /proc/29429/task/81874/cwd (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81874 mgr-fin ceph rtd unknown /proc/29429/task/81874/root (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81874 mgr-fin ceph txt unknown /proc/29429/task/81874/exe (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81874 mgr-fin ceph NOFD /proc/29429/task/81874/fd (opendir: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81875 mgr-fin ceph cwd unknown /proc/29429/task/81875/cwd (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81875 mgr-fin ceph rtd unknown /proc/29429/task/81875/root (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81875 mgr-fin ceph txt unknown /proc/29429/task/81875/exe (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81875 mgr-fin ceph NOFD /proc/29429/task/81875/fd (opendir: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81877 m-fin-vol ceph cwd unknown /proc/29429/task/81877/cwd (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81877 m-fin-vol ceph rtd unknown /proc/29429/task/81877/root (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81877 m-fin-vol ceph txt unknown /proc/29429/task/81877/exe (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81877 m-fin-vol ceph NOFD /proc/29429/task/81877/fd (opendir: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81878 io_contex ceph cwd unknown /proc/29429/task/81878/cwd (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81878 io_contex ceph rtd unknown /proc/29429/task/81878/root (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81878 io_contex ceph txt unknown /proc/29429/task/81878/exe (readlink: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81878 io_contex ceph NOFD /proc/29429/task/81878/fd (opendir: Permission denied) 2024-04-25T00:24:20.857 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81879 io_contex ceph cwd unknown /proc/29429/task/81879/cwd (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81879 io_contex ceph rtd unknown /proc/29429/task/81879/root (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81879 io_contex ceph txt unknown /proc/29429/task/81879/exe (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81879 io_contex ceph NOFD /proc/29429/task/81879/fd (opendir: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81946 dashboard ceph cwd unknown /proc/29429/task/81946/cwd (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81946 dashboard ceph rtd unknown /proc/29429/task/81946/root (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81946 dashboard ceph txt unknown /proc/29429/task/81946/exe (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81946 dashboard ceph NOFD /proc/29429/task/81946/fd (opendir: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81947 dashboard ceph cwd unknown /proc/29429/task/81947/cwd (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81947 dashboard ceph rtd unknown /proc/29429/task/81947/root (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81947 dashboard ceph txt unknown /proc/29429/task/81947/exe (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81947 dashboard ceph NOFD /proc/29429/task/81947/fd (opendir: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81948 dashboard ceph cwd unknown /proc/29429/task/81948/cwd (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81948 dashboard ceph rtd unknown /proc/29429/task/81948/root (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81948 dashboard ceph txt unknown /proc/29429/task/81948/exe (readlink: Permission denied) 2024-04-25T00:24:20.858 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81948 dashboard ceph NOFD /proc/29429/task/81948/fd (opendir: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81949 dashboard ceph cwd unknown /proc/29429/task/81949/cwd (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81949 dashboard ceph rtd unknown /proc/29429/task/81949/root (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81949 dashboard ceph txt unknown /proc/29429/task/81949/exe (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81949 dashboard ceph NOFD /proc/29429/task/81949/fd (opendir: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81950 dashboard ceph cwd unknown /proc/29429/task/81950/cwd (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81950 dashboard ceph rtd unknown /proc/29429/task/81950/root (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81950 dashboard ceph txt unknown /proc/29429/task/81950/exe (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81950 dashboard ceph NOFD /proc/29429/task/81950/fd (opendir: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81951 dashboard ceph cwd unknown /proc/29429/task/81951/cwd (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81951 dashboard ceph rtd unknown /proc/29429/task/81951/root (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81951 dashboard ceph txt unknown /proc/29429/task/81951/exe (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81951 dashboard ceph NOFD /proc/29429/task/81951/fd (opendir: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81952 dashboard ceph cwd unknown /proc/29429/task/81952/cwd (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81952 dashboard ceph rtd unknown /proc/29429/task/81952/root (readlink: Permission denied) 2024-04-25T00:24:20.859 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81952 dashboard ceph txt unknown /proc/29429/task/81952/exe (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81952 dashboard ceph NOFD /proc/29429/task/81952/fd (opendir: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81953 dashboard ceph cwd unknown /proc/29429/task/81953/cwd (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81953 dashboard ceph rtd unknown /proc/29429/task/81953/root (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81953 dashboard ceph txt unknown /proc/29429/task/81953/exe (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81953 dashboard ceph NOFD /proc/29429/task/81953/fd (opendir: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81954 dashboard ceph cwd unknown /proc/29429/task/81954/cwd (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81954 dashboard ceph rtd unknown /proc/29429/task/81954/root (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81954 dashboard ceph txt unknown /proc/29429/task/81954/exe (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81954 dashboard ceph NOFD /proc/29429/task/81954/fd (opendir: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81955 dashboard ceph cwd unknown /proc/29429/task/81955/cwd (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81955 dashboard ceph rtd unknown /proc/29429/task/81955/root (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81955 dashboard ceph txt unknown /proc/29429/task/81955/exe (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81955 dashboard ceph NOFD /proc/29429/task/81955/fd (opendir: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81956 dashboard ceph cwd unknown /proc/29429/task/81956/cwd (readlink: Permission denied) 2024-04-25T00:24:20.860 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81956 dashboard ceph rtd unknown /proc/29429/task/81956/root (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81956 dashboard ceph txt unknown /proc/29429/task/81956/exe (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81956 dashboard ceph NOFD /proc/29429/task/81956/fd (opendir: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81975 dashboard ceph cwd unknown /proc/29429/task/81975/cwd (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81975 dashboard ceph rtd unknown /proc/29429/task/81975/root (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81975 dashboard ceph txt unknown /proc/29429/task/81975/exe (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 81975 dashboard ceph NOFD /proc/29429/task/81975/fd (opendir: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82067 mgr-fin ceph cwd unknown /proc/29429/task/82067/cwd (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82067 mgr-fin ceph rtd unknown /proc/29429/task/82067/root (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82067 mgr-fin ceph txt unknown /proc/29429/task/82067/exe (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82067 mgr-fin ceph NOFD /proc/29429/task/82067/fd (opendir: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82068 mgr-fin ceph cwd unknown /proc/29429/task/82068/cwd (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82068 mgr-fin ceph rtd unknown /proc/29429/task/82068/root (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82068 mgr-fin ceph txt unknown /proc/29429/task/82068/exe (readlink: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82068 mgr-fin ceph NOFD /proc/29429/task/82068/fd (opendir: Permission denied) 2024-04-25T00:24:20.861 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82069 mgr-fin ceph cwd unknown /proc/29429/task/82069/cwd (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82069 mgr-fin ceph rtd unknown /proc/29429/task/82069/root (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82069 mgr-fin ceph txt unknown /proc/29429/task/82069/exe (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82069 mgr-fin ceph NOFD /proc/29429/task/82069/fd (opendir: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82070 mgr-fin ceph cwd unknown /proc/29429/task/82070/cwd (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82070 mgr-fin ceph rtd unknown /proc/29429/task/82070/root (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82070 mgr-fin ceph txt unknown /proc/29429/task/82070/exe (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82070 mgr-fin ceph NOFD /proc/29429/task/82070/fd (opendir: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82071 mgr-fin ceph cwd unknown /proc/29429/task/82071/cwd (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82071 mgr-fin ceph rtd unknown /proc/29429/task/82071/root (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82071 mgr-fin ceph txt unknown /proc/29429/task/82071/exe (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82071 mgr-fin ceph NOFD /proc/29429/task/82071/fd (opendir: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82072 mgr-fin ceph cwd unknown /proc/29429/task/82072/cwd (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82072 mgr-fin ceph rtd unknown /proc/29429/task/82072/root (readlink: Permission denied) 2024-04-25T00:24:20.862 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82072 mgr-fin ceph txt unknown /proc/29429/task/82072/exe (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82072 mgr-fin ceph NOFD /proc/29429/task/82072/fd (opendir: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82073 mgr-fin ceph cwd unknown /proc/29429/task/82073/cwd (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82073 mgr-fin ceph rtd unknown /proc/29429/task/82073/root (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82073 mgr-fin ceph txt unknown /proc/29429/task/82073/exe (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82073 mgr-fin ceph NOFD /proc/29429/task/82073/fd (opendir: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82074 mgr-fin ceph cwd unknown /proc/29429/task/82074/cwd (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82074 mgr-fin ceph rtd unknown /proc/29429/task/82074/root (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82074 mgr-fin ceph txt unknown /proc/29429/task/82074/exe (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82074 mgr-fin ceph NOFD /proc/29429/task/82074/fd (opendir: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82075 mgr-fin ceph cwd unknown /proc/29429/task/82075/cwd (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82075 mgr-fin ceph rtd unknown /proc/29429/task/82075/root (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82075 mgr-fin ceph txt unknown /proc/29429/task/82075/exe (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82075 mgr-fin ceph NOFD /proc/29429/task/82075/fd (opendir: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82076 mgr-fin ceph cwd unknown /proc/29429/task/82076/cwd (readlink: Permission denied) 2024-04-25T00:24:20.863 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82076 mgr-fin ceph rtd unknown /proc/29429/task/82076/root (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82076 mgr-fin ceph txt unknown /proc/29429/task/82076/exe (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82076 mgr-fin ceph NOFD /proc/29429/task/82076/fd (opendir: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82077 mgr-fin ceph cwd unknown /proc/29429/task/82077/cwd (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82077 mgr-fin ceph rtd unknown /proc/29429/task/82077/root (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82077 mgr-fin ceph txt unknown /proc/29429/task/82077/exe (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82077 mgr-fin ceph NOFD /proc/29429/task/82077/fd (opendir: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82098 mgr-fin ceph cwd unknown /proc/29429/task/82098/cwd (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82098 mgr-fin ceph rtd unknown /proc/29429/task/82098/root (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82098 mgr-fin ceph txt unknown /proc/29429/task/82098/exe (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82098 mgr-fin ceph NOFD /proc/29429/task/82098/fd (opendir: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82099 mgr-fin ceph cwd unknown /proc/29429/task/82099/cwd (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82099 mgr-fin ceph rtd unknown /proc/29429/task/82099/root (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82099 mgr-fin ceph txt unknown /proc/29429/task/82099/exe (readlink: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82099 mgr-fin ceph NOFD /proc/29429/task/82099/fd (opendir: Permission denied) 2024-04-25T00:24:20.864 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82100 mgr-fin ceph cwd unknown /proc/29429/task/82100/cwd (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82100 mgr-fin ceph rtd unknown /proc/29429/task/82100/root (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82100 mgr-fin ceph txt unknown /proc/29429/task/82100/exe (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82100 mgr-fin ceph NOFD /proc/29429/task/82100/fd (opendir: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82101 mgr-fin ceph cwd unknown /proc/29429/task/82101/cwd (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82101 mgr-fin ceph rtd unknown /proc/29429/task/82101/root (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82101 mgr-fin ceph txt unknown /proc/29429/task/82101/exe (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82101 mgr-fin ceph NOFD /proc/29429/task/82101/fd (opendir: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82102 mgr-fin ceph cwd unknown /proc/29429/task/82102/cwd (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82102 mgr-fin ceph rtd unknown /proc/29429/task/82102/root (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82102 mgr-fin ceph txt unknown /proc/29429/task/82102/exe (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82102 mgr-fin ceph NOFD /proc/29429/task/82102/fd (opendir: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82103 mgr-fin ceph cwd unknown /proc/29429/task/82103/cwd (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82103 mgr-fin ceph rtd unknown /proc/29429/task/82103/root (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82103 mgr-fin ceph txt unknown /proc/29429/task/82103/exe (readlink: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82103 mgr-fin ceph NOFD /proc/29429/task/82103/fd (opendir: Permission denied) 2024-04-25T00:24:20.865 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82104 mgr-fin ceph cwd unknown /proc/29429/task/82104/cwd (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82104 mgr-fin ceph rtd unknown /proc/29429/task/82104/root (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82104 mgr-fin ceph txt unknown /proc/29429/task/82104/exe (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82104 mgr-fin ceph NOFD /proc/29429/task/82104/fd (opendir: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82105 mgr-fin ceph cwd unknown /proc/29429/task/82105/cwd (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82105 mgr-fin ceph rtd unknown /proc/29429/task/82105/root (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82105 mgr-fin ceph txt unknown /proc/29429/task/82105/exe (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82105 mgr-fin ceph NOFD /proc/29429/task/82105/fd (opendir: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82106 mgr-fin ceph cwd unknown /proc/29429/task/82106/cwd (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82106 mgr-fin ceph rtd unknown /proc/29429/task/82106/root (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82106 mgr-fin ceph txt unknown /proc/29429/task/82106/exe (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82106 mgr-fin ceph NOFD /proc/29429/task/82106/fd (opendir: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82107 mgr-fin ceph cwd unknown /proc/29429/task/82107/cwd (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82107 mgr-fin ceph rtd unknown /proc/29429/task/82107/root (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82107 mgr-fin ceph txt unknown /proc/29429/task/82107/exe (readlink: Permission denied) 2024-04-25T00:24:20.866 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82107 mgr-fin ceph NOFD /proc/29429/task/82107/fd (opendir: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82108 mgr-fin ceph cwd unknown /proc/29429/task/82108/cwd (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82108 mgr-fin ceph rtd unknown /proc/29429/task/82108/root (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82108 mgr-fin ceph txt unknown /proc/29429/task/82108/exe (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82108 mgr-fin ceph NOFD /proc/29429/task/82108/fd (opendir: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82402 devicehea ceph cwd unknown /proc/29429/task/82402/cwd (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82402 devicehea ceph rtd unknown /proc/29429/task/82402/root (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82402 devicehea ceph txt unknown /proc/29429/task/82402/exe (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82402 devicehea ceph NOFD /proc/29429/task/82402/fd (opendir: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82414 io_contex ceph cwd unknown /proc/29429/task/82414/cwd (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82414 io_contex ceph rtd unknown /proc/29429/task/82414/root (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82414 io_contex ceph txt unknown /proc/29429/task/82414/exe (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82414 io_contex ceph NOFD /proc/29429/task/82414/fd (opendir: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82415 io_contex ceph cwd unknown /proc/29429/task/82415/cwd (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82415 io_contex ceph rtd unknown /proc/29429/task/82415/root (readlink: Permission denied) 2024-04-25T00:24:20.867 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82415 io_contex ceph txt unknown /proc/29429/task/82415/exe (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82415 io_contex ceph NOFD /proc/29429/task/82415/fd (opendir: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82416 ceph_time ceph cwd unknown /proc/29429/task/82416/cwd (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82416 ceph_time ceph rtd unknown /proc/29429/task/82416/root (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82416 ceph_time ceph txt unknown /proc/29429/task/82416/exe (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82416 ceph_time ceph NOFD /proc/29429/task/82416/fd (opendir: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82417 ms_dispat ceph cwd unknown /proc/29429/task/82417/cwd (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82417 ms_dispat ceph rtd unknown /proc/29429/task/82417/root (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82417 ms_dispat ceph txt unknown /proc/29429/task/82417/exe (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82417 ms_dispat ceph NOFD /proc/29429/task/82417/fd (opendir: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82418 ms_local ceph cwd unknown /proc/29429/task/82418/cwd (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82418 ms_local ceph rtd unknown /proc/29429/task/82418/root (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82418 ms_local ceph txt unknown /proc/29429/task/82418/exe (readlink: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82418 ms_local ceph NOFD /proc/29429/task/82418/fd (opendir: Permission denied) 2024-04-25T00:24:20.868 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82419 safe_time ceph cwd unknown /proc/29429/task/82419/cwd (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82419 safe_time ceph rtd unknown /proc/29429/task/82419/root (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82419 safe_time ceph txt unknown /proc/29429/task/82419/exe (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82419 safe_time ceph NOFD /proc/29429/task/82419/fd (opendir: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82420 safe_time ceph cwd unknown /proc/29429/task/82420/cwd (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82420 safe_time ceph rtd unknown /proc/29429/task/82420/root (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82420 safe_time ceph txt unknown /proc/29429/task/82420/exe (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 82420 safe_time ceph NOFD /proc/29429/task/82420/fd (opendir: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84934 ceph_time ceph cwd unknown /proc/29429/task/84934/cwd (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84934 ceph_time ceph rtd unknown /proc/29429/task/84934/root (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84934 ceph_time ceph txt unknown /proc/29429/task/84934/exe (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84934 ceph_time ceph NOFD /proc/29429/task/84934/fd (opendir: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84935 fn_anonym ceph cwd unknown /proc/29429/task/84935/cwd (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84935 fn_anonym ceph rtd unknown /proc/29429/task/84935/root (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84935 fn_anonym ceph txt unknown /proc/29429/task/84935/exe (readlink: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84935 fn_anonym ceph NOFD /proc/29429/task/84935/fd (opendir: Permission denied) 2024-04-25T00:24:20.869 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84936 safe_time ceph cwd unknown /proc/29429/task/84936/cwd (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84936 safe_time ceph rtd unknown /proc/29429/task/84936/root (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84936 safe_time ceph txt unknown /proc/29429/task/84936/exe (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84936 safe_time ceph NOFD /proc/29429/task/84936/fd (opendir: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84937 fn_anonym ceph cwd unknown /proc/29429/task/84937/cwd (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84937 fn_anonym ceph rtd unknown /proc/29429/task/84937/root (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84937 fn_anonym ceph txt unknown /proc/29429/task/84937/exe (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84937 fn_anonym ceph NOFD /proc/29429/task/84937/fd (opendir: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84938 flusher ceph cwd unknown /proc/29429/task/84938/cwd (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84938 flusher ceph rtd unknown /proc/29429/task/84938/root (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84938 flusher ceph txt unknown /proc/29429/task/84938/exe (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84938 flusher ceph NOFD /proc/29429/task/84938/fd (opendir: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84939 ms_dispat ceph cwd unknown /proc/29429/task/84939/cwd (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84939 ms_dispat ceph rtd unknown /proc/29429/task/84939/root (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84939 ms_dispat ceph txt unknown /proc/29429/task/84939/exe (readlink: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84939 ms_dispat ceph NOFD /proc/29429/task/84939/fd (opendir: Permission denied) 2024-04-25T00:24:20.870 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84940 ms_local ceph cwd unknown /proc/29429/task/84940/cwd (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84940 ms_local ceph rtd unknown /proc/29429/task/84940/root (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84940 ms_local ceph txt unknown /proc/29429/task/84940/exe (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84940 ms_local ceph NOFD /proc/29429/task/84940/fd (opendir: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84941 safe_time ceph cwd unknown /proc/29429/task/84941/cwd (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84941 safe_time ceph rtd unknown /proc/29429/task/84941/root (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84941 safe_time ceph txt unknown /proc/29429/task/84941/exe (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84941 safe_time ceph NOFD /proc/29429/task/84941/fd (opendir: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84942 m-fin-sna ceph cwd unknown /proc/29429/task/84942/cwd (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84942 m-fin-sna ceph rtd unknown /proc/29429/task/84942/root (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84942 m-fin-sna ceph txt unknown /proc/29429/task/84942/exe (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 84942 m-fin-sna ceph NOFD /proc/29429/task/84942/fd (opendir: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 86416 m-fin-sna ceph cwd unknown /proc/29429/task/86416/cwd (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 86416 m-fin-sna ceph rtd unknown /proc/29429/task/86416/root (readlink: Permission denied) 2024-04-25T00:24:20.871 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 86416 m-fin-sna ceph txt unknown /proc/29429/task/86416/exe (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mgr 29429 86416 m-fin-sna ceph NOFD /proc/29429/task/86416/fd (opendir: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 29992 root cwd unknown /proc/29992/cwd (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 29992 root rtd unknown /proc/29992/root (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 29992 root txt unknown /proc/29992/exe (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 29992 root NOFD /proc/29992/fd (opendir: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 30517 root cwd unknown /proc/30517/cwd (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 30517 root rtd unknown /proc/30517/root (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 30517 root txt unknown /proc/30517/exe (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:systemd 30517 root NOFD /proc/30517/fd (opendir: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 30520 root cwd unknown /proc/30520/cwd (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 30520 root rtd unknown /proc/30520/root (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 30520 root txt unknown /proc/30520/exe (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:(sd-pam) 30520 root NOFD /proc/30520/fd (opendir: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 32947 root cwd unknown /proc/32947/cwd (readlink: Permission denied) 2024-04-25T00:24:20.872 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 32947 root rtd unknown /proc/32947/root (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 32947 root txt unknown /proc/32947/exe (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/3 32947 root NOFD /proc/32947/fd (opendir: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 37004 root cwd unknown /proc/37004/cwd (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 37004 root rtd unknown /proc/37004/root (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 37004 root txt unknown /proc/37004/exe (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 37004 root NOFD /proc/37004/fd (opendir: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 39297 root cwd unknown /proc/39297/cwd (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 39297 root rtd unknown /proc/39297/root (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 39297 root txt unknown /proc/39297/exe (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 39297 root NOFD /proc/39297/fd (opendir: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 41753 root cwd unknown /proc/41753/cwd (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 41753 root rtd unknown /proc/41753/root (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 41753 root txt unknown /proc/41753/exe (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 41753 root NOFD /proc/41753/fd (opendir: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 41755 root cwd unknown /proc/41755/cwd (readlink: Permission denied) 2024-04-25T00:24:20.873 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 41755 root rtd unknown /proc/41755/root (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 41755 root txt unknown /proc/41755/exe (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 41755 root NOFD /proc/41755/fd (opendir: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 ceph cwd unknown /proc/41758/cwd (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 ceph rtd unknown /proc/41758/root (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 ceph txt unknown /proc/41758/exe (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 ceph NOFD /proc/41758/fd (opendir: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41759 log ceph cwd unknown /proc/41758/task/41759/cwd (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41759 log ceph rtd unknown /proc/41758/task/41759/root (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41759 log ceph txt unknown /proc/41758/task/41759/exe (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41759 log ceph NOFD /proc/41758/task/41759/fd (opendir: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41761 msgr-work ceph cwd unknown /proc/41758/task/41761/cwd (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41761 msgr-work ceph rtd unknown /proc/41758/task/41761/root (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41761 msgr-work ceph txt unknown /proc/41758/task/41761/exe (readlink: Permission denied) 2024-04-25T00:24:20.874 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41761 msgr-work ceph NOFD /proc/41758/task/41761/fd (opendir: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41762 msgr-work ceph cwd unknown /proc/41758/task/41762/cwd (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41762 msgr-work ceph rtd unknown /proc/41758/task/41762/root (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41762 msgr-work ceph txt unknown /proc/41758/task/41762/exe (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41762 msgr-work ceph NOFD /proc/41758/task/41762/fd (opendir: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41763 msgr-work ceph cwd unknown /proc/41758/task/41763/cwd (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41763 msgr-work ceph rtd unknown /proc/41758/task/41763/root (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41763 msgr-work ceph txt unknown /proc/41758/task/41763/exe (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41763 msgr-work ceph NOFD /proc/41758/task/41763/fd (opendir: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41767 service ceph cwd unknown /proc/41758/task/41767/cwd (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41767 service ceph rtd unknown /proc/41758/task/41767/root (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41767 service ceph txt unknown /proc/41758/task/41767/exe (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41767 service ceph NOFD /proc/41758/task/41767/fd (opendir: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41768 admin_soc ceph cwd unknown /proc/41758/task/41768/cwd (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41768 admin_soc ceph rtd unknown /proc/41758/task/41768/root (readlink: Permission denied) 2024-04-25T00:24:20.875 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41768 admin_soc ceph txt unknown /proc/41758/task/41768/exe (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41768 admin_soc ceph NOFD /proc/41758/task/41768/fd (opendir: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41918 signal_ha ceph cwd unknown /proc/41758/task/41918/cwd (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41918 signal_ha ceph rtd unknown /proc/41758/task/41918/root (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41918 signal_ha ceph txt unknown /proc/41758/task/41918/exe (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41918 signal_ha ceph NOFD /proc/41758/task/41918/fd (opendir: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41919 io_contex ceph cwd unknown /proc/41758/task/41919/cwd (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41919 io_contex ceph rtd unknown /proc/41758/task/41919/root (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41919 io_contex ceph txt unknown /proc/41758/task/41919/exe (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41919 io_contex ceph NOFD /proc/41758/task/41919/fd (opendir: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41920 io_contex ceph cwd unknown /proc/41758/task/41920/cwd (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41920 io_contex ceph rtd unknown /proc/41758/task/41920/root (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41920 io_contex ceph txt unknown /proc/41758/task/41920/exe (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 41920 io_contex ceph NOFD /proc/41758/task/41920/fd (opendir: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42036 OpHistory ceph cwd unknown /proc/41758/task/42036/cwd (readlink: Permission denied) 2024-04-25T00:24:20.876 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42036 OpHistory ceph rtd unknown /proc/41758/task/42036/root (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42036 OpHistory ceph txt unknown /proc/41758/task/42036/exe (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42036 OpHistory ceph NOFD /proc/41758/task/42036/fd (opendir: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42037 ceph_time ceph cwd unknown /proc/41758/task/42037/cwd (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42037 ceph_time ceph rtd unknown /proc/41758/task/42037/root (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42037 ceph_time ceph txt unknown /proc/41758/task/42037/exe (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42037 ceph_time ceph NOFD /proc/41758/task/42037/fd (opendir: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42058 ceph-osd ceph cwd unknown /proc/41758/task/42058/cwd (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42058 ceph-osd ceph rtd unknown /proc/41758/task/42058/root (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42058 ceph-osd ceph txt unknown /proc/41758/task/42058/exe (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42058 ceph-osd ceph NOFD /proc/41758/task/42058/fd (opendir: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42062 ceph-osd ceph cwd unknown /proc/41758/task/42062/cwd (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42062 ceph-osd ceph rtd unknown /proc/41758/task/42062/root (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42062 ceph-osd ceph txt unknown /proc/41758/task/42062/exe (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42062 ceph-osd ceph NOFD /proc/41758/task/42062/fd (opendir: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42068 ceph-osd ceph cwd unknown /proc/41758/task/42068/cwd (readlink: Permission denied) 2024-04-25T00:24:20.877 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42068 ceph-osd ceph rtd unknown /proc/41758/task/42068/root (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42068 ceph-osd ceph txt unknown /proc/41758/task/42068/exe (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42068 ceph-osd ceph NOFD /proc/41758/task/42068/fd (opendir: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42092 ceph-osd ceph cwd unknown /proc/41758/task/42092/cwd (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42092 ceph-osd ceph rtd unknown /proc/41758/task/42092/root (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42092 ceph-osd ceph txt unknown /proc/41758/task/42092/exe (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42092 ceph-osd ceph NOFD /proc/41758/task/42092/fd (opendir: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42114 ceph-osd ceph cwd unknown /proc/41758/task/42114/cwd (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42114 ceph-osd ceph rtd unknown /proc/41758/task/42114/root (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42114 ceph-osd ceph txt unknown /proc/41758/task/42114/exe (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42114 ceph-osd ceph NOFD /proc/41758/task/42114/fd (opendir: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42148 ceph-osd ceph cwd unknown /proc/41758/task/42148/cwd (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42148 ceph-osd ceph rtd unknown /proc/41758/task/42148/root (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42148 ceph-osd ceph txt unknown /proc/41758/task/42148/exe (readlink: Permission denied) 2024-04-25T00:24:20.878 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42148 ceph-osd ceph NOFD /proc/41758/task/42148/fd (opendir: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42193 ceph-osd ceph cwd unknown /proc/41758/task/42193/cwd (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42193 ceph-osd ceph rtd unknown /proc/41758/task/42193/root (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42193 ceph-osd ceph txt unknown /proc/41758/task/42193/exe (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42193 ceph-osd ceph NOFD /proc/41758/task/42193/fd (opendir: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42234 ceph-osd ceph cwd unknown /proc/41758/task/42234/cwd (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42234 ceph-osd ceph rtd unknown /proc/41758/task/42234/root (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42234 ceph-osd ceph txt unknown /proc/41758/task/42234/exe (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42234 ceph-osd ceph NOFD /proc/41758/task/42234/fd (opendir: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42235 ceph-osd ceph cwd unknown /proc/41758/task/42235/cwd (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42235 ceph-osd ceph rtd unknown /proc/41758/task/42235/root (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42235 ceph-osd ceph txt unknown /proc/41758/task/42235/exe (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42235 ceph-osd ceph NOFD /proc/41758/task/42235/fd (opendir: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42236 safe_time ceph cwd unknown /proc/41758/task/42236/cwd (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42236 safe_time ceph rtd unknown /proc/41758/task/42236/root (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42236 safe_time ceph txt unknown /proc/41758/task/42236/exe (readlink: Permission denied) 2024-04-25T00:24:20.879 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42236 safe_time ceph NOFD /proc/41758/task/42236/fd (opendir: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42237 safe_time ceph cwd unknown /proc/41758/task/42237/cwd (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42237 safe_time ceph rtd unknown /proc/41758/task/42237/root (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42237 safe_time ceph txt unknown /proc/41758/task/42237/exe (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42237 safe_time ceph NOFD /proc/41758/task/42237/fd (opendir: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42238 safe_time ceph cwd unknown /proc/41758/task/42238/cwd (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42238 safe_time ceph rtd unknown /proc/41758/task/42238/root (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42238 safe_time ceph txt unknown /proc/41758/task/42238/exe (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42238 safe_time ceph NOFD /proc/41758/task/42238/fd (opendir: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42239 safe_time ceph cwd unknown /proc/41758/task/42239/cwd (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42239 safe_time ceph rtd unknown /proc/41758/task/42239/root (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42239 safe_time ceph txt unknown /proc/41758/task/42239/exe (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42239 safe_time ceph NOFD /proc/41758/task/42239/fd (opendir: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42241 fn_anonym ceph cwd unknown /proc/41758/task/42241/cwd (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42241 fn_anonym ceph rtd unknown /proc/41758/task/42241/root (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42241 fn_anonym ceph txt unknown /proc/41758/task/42241/exe (readlink: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42241 fn_anonym ceph NOFD /proc/41758/task/42241/fd (opendir: Permission denied) 2024-04-25T00:24:20.880 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42244 rocksdb:l ceph cwd unknown /proc/41758/task/42244/cwd (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42244 rocksdb:l ceph rtd unknown /proc/41758/task/42244/root (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42244 rocksdb:l ceph txt unknown /proc/41758/task/42244/exe (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42244 rocksdb:l ceph NOFD /proc/41758/task/42244/fd (opendir: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42245 rocksdb:l ceph cwd unknown /proc/41758/task/42245/cwd (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42245 rocksdb:l ceph rtd unknown /proc/41758/task/42245/root (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42245 rocksdb:l ceph txt unknown /proc/41758/task/42245/exe (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42245 rocksdb:l ceph NOFD /proc/41758/task/42245/fd (opendir: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42246 rocksdb:l ceph cwd unknown /proc/41758/task/42246/cwd (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42246 rocksdb:l ceph rtd unknown /proc/41758/task/42246/root (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42246 rocksdb:l ceph txt unknown /proc/41758/task/42246/exe (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42246 rocksdb:l ceph NOFD /proc/41758/task/42246/fd (opendir: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42247 rocksdb:h ceph cwd unknown /proc/41758/task/42247/cwd (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42247 rocksdb:h ceph rtd unknown /proc/41758/task/42247/root (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42247 rocksdb:h ceph txt unknown /proc/41758/task/42247/exe (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42247 rocksdb:h ceph NOFD /proc/41758/task/42247/fd (opendir: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42629 bstore_ai ceph cwd unknown /proc/41758/task/42629/cwd (readlink: Permission denied) 2024-04-25T00:24:20.881 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42629 bstore_ai ceph rtd unknown /proc/41758/task/42629/root (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42629 bstore_ai ceph txt unknown /proc/41758/task/42629/exe (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42629 bstore_ai ceph NOFD /proc/41758/task/42629/fd (opendir: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42815 bstore_ai ceph cwd unknown /proc/41758/task/42815/cwd (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42815 bstore_ai ceph rtd unknown /proc/41758/task/42815/root (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42815 bstore_ai ceph txt unknown /proc/41758/task/42815/exe (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42815 bstore_ai ceph NOFD /proc/41758/task/42815/fd (opendir: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42997 ceph-osd ceph cwd unknown /proc/41758/task/42997/cwd (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42997 ceph-osd ceph rtd unknown /proc/41758/task/42997/root (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42997 ceph-osd ceph txt unknown /proc/41758/task/42997/exe (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42997 ceph-osd ceph NOFD /proc/41758/task/42997/fd (opendir: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42998 cfin ceph cwd unknown /proc/41758/task/42998/cwd (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42998 cfin ceph rtd unknown /proc/41758/task/42998/root (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42998 cfin ceph txt unknown /proc/41758/task/42998/exe (readlink: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42998 cfin ceph NOFD /proc/41758/task/42998/fd (opendir: Permission denied) 2024-04-25T00:24:20.882 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42999 bstore_kv ceph cwd unknown /proc/41758/task/42999/cwd (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42999 bstore_kv ceph rtd unknown /proc/41758/task/42999/root (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42999 bstore_kv ceph txt unknown /proc/41758/task/42999/exe (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 42999 bstore_kv ceph NOFD /proc/41758/task/42999/fd (opendir: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43000 bstore_kv ceph cwd unknown /proc/41758/task/43000/cwd (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43000 bstore_kv ceph rtd unknown /proc/41758/task/43000/root (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43000 bstore_kv ceph txt unknown /proc/41758/task/43000/exe (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43000 bstore_kv ceph NOFD /proc/41758/task/43000/fd (opendir: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43001 bstore_me ceph cwd unknown /proc/41758/task/43001/cwd (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43001 bstore_me ceph rtd unknown /proc/41758/task/43001/root (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43001 bstore_me ceph txt unknown /proc/41758/task/43001/exe (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43001 bstore_me ceph NOFD /proc/41758/task/43001/fd (opendir: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43002 ms_dispat ceph cwd unknown /proc/41758/task/43002/cwd (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43002 ms_dispat ceph rtd unknown /proc/41758/task/43002/root (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43002 ms_dispat ceph txt unknown /proc/41758/task/43002/exe (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43002 ms_dispat ceph NOFD /proc/41758/task/43002/fd (opendir: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43003 ms_local ceph cwd unknown /proc/41758/task/43003/cwd (readlink: Permission denied) 2024-04-25T00:24:20.883 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43003 ms_local ceph rtd unknown /proc/41758/task/43003/root (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43003 ms_local ceph txt unknown /proc/41758/task/43003/exe (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43003 ms_local ceph NOFD /proc/41758/task/43003/fd (opendir: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43004 safe_time ceph cwd unknown /proc/41758/task/43004/cwd (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43004 safe_time ceph rtd unknown /proc/41758/task/43004/root (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43004 safe_time ceph txt unknown /proc/41758/task/43004/exe (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43004 safe_time ceph NOFD /proc/41758/task/43004/fd (opendir: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43005 safe_time ceph cwd unknown /proc/41758/task/43005/cwd (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43005 safe_time ceph rtd unknown /proc/41758/task/43005/root (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43005 safe_time ceph txt unknown /proc/41758/task/43005/exe (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43005 safe_time ceph NOFD /proc/41758/task/43005/fd (opendir: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43006 ms_dispat ceph cwd unknown /proc/41758/task/43006/cwd (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43006 ms_dispat ceph rtd unknown /proc/41758/task/43006/root (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43006 ms_dispat ceph txt unknown /proc/41758/task/43006/exe (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43006 ms_dispat ceph NOFD /proc/41758/task/43006/fd (opendir: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43007 ms_local ceph cwd unknown /proc/41758/task/43007/cwd (readlink: Permission denied) 2024-04-25T00:24:20.884 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43007 ms_local ceph rtd unknown /proc/41758/task/43007/root (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43007 ms_local ceph txt unknown /proc/41758/task/43007/exe (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43007 ms_local ceph NOFD /proc/41758/task/43007/fd (opendir: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43008 ms_dispat ceph cwd unknown /proc/41758/task/43008/cwd (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43008 ms_dispat ceph rtd unknown /proc/41758/task/43008/root (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43008 ms_dispat ceph txt unknown /proc/41758/task/43008/exe (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43008 ms_dispat ceph NOFD /proc/41758/task/43008/fd (opendir: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43009 ms_local ceph cwd unknown /proc/41758/task/43009/cwd (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43009 ms_local ceph rtd unknown /proc/41758/task/43009/root (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43009 ms_local ceph txt unknown /proc/41758/task/43009/exe (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43009 ms_local ceph NOFD /proc/41758/task/43009/fd (opendir: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43010 ms_dispat ceph cwd unknown /proc/41758/task/43010/cwd (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43010 ms_dispat ceph rtd unknown /proc/41758/task/43010/root (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43010 ms_dispat ceph txt unknown /proc/41758/task/43010/exe (readlink: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43010 ms_dispat ceph NOFD /proc/41758/task/43010/fd (opendir: Permission denied) 2024-04-25T00:24:20.885 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43011 ms_local ceph cwd unknown /proc/41758/task/43011/cwd (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43011 ms_local ceph rtd unknown /proc/41758/task/43011/root (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43011 ms_local ceph txt unknown /proc/41758/task/43011/exe (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43011 ms_local ceph NOFD /proc/41758/task/43011/fd (opendir: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43012 ms_dispat ceph cwd unknown /proc/41758/task/43012/cwd (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43012 ms_dispat ceph rtd unknown /proc/41758/task/43012/root (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43012 ms_dispat ceph txt unknown /proc/41758/task/43012/exe (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43012 ms_dispat ceph NOFD /proc/41758/task/43012/fd (opendir: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43013 ms_local ceph cwd unknown /proc/41758/task/43013/cwd (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43013 ms_local ceph rtd unknown /proc/41758/task/43013/root (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43013 ms_local ceph txt unknown /proc/41758/task/43013/exe (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43013 ms_local ceph NOFD /proc/41758/task/43013/fd (opendir: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43014 ms_dispat ceph cwd unknown /proc/41758/task/43014/cwd (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43014 ms_dispat ceph rtd unknown /proc/41758/task/43014/root (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43014 ms_dispat ceph txt unknown /proc/41758/task/43014/exe (readlink: Permission denied) 2024-04-25T00:24:20.886 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43014 ms_dispat ceph NOFD /proc/41758/task/43014/fd (opendir: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43015 ms_local ceph cwd unknown /proc/41758/task/43015/cwd (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43015 ms_local ceph rtd unknown /proc/41758/task/43015/root (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43015 ms_local ceph txt unknown /proc/41758/task/43015/exe (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43015 ms_local ceph NOFD /proc/41758/task/43015/fd (opendir: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43016 ms_dispat ceph cwd unknown /proc/41758/task/43016/cwd (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43016 ms_dispat ceph rtd unknown /proc/41758/task/43016/root (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43016 ms_dispat ceph txt unknown /proc/41758/task/43016/exe (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43016 ms_dispat ceph NOFD /proc/41758/task/43016/fd (opendir: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43017 ms_local ceph cwd unknown /proc/41758/task/43017/cwd (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43017 ms_local ceph rtd unknown /proc/41758/task/43017/root (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43017 ms_local ceph txt unknown /proc/41758/task/43017/exe (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43017 ms_local ceph NOFD /proc/41758/task/43017/fd (opendir: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43018 fn_anonym ceph cwd unknown /proc/41758/task/43018/cwd (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43018 fn_anonym ceph rtd unknown /proc/41758/task/43018/root (readlink: Permission denied) 2024-04-25T00:24:20.887 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43018 fn_anonym ceph txt unknown /proc/41758/task/43018/exe (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43018 fn_anonym ceph NOFD /proc/41758/task/43018/fd (opendir: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43019 finisher ceph cwd unknown /proc/41758/task/43019/cwd (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43019 finisher ceph rtd unknown /proc/41758/task/43019/root (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43019 finisher ceph txt unknown /proc/41758/task/43019/exe (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43019 finisher ceph NOFD /proc/41758/task/43019/fd (opendir: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43020 safe_time ceph cwd unknown /proc/41758/task/43020/cwd (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43020 safe_time ceph rtd unknown /proc/41758/task/43020/root (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43020 safe_time ceph txt unknown /proc/41758/task/43020/exe (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43020 safe_time ceph NOFD /proc/41758/task/43020/fd (opendir: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43021 safe_time ceph cwd unknown /proc/41758/task/43021/cwd (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43021 safe_time ceph rtd unknown /proc/41758/task/43021/root (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43021 safe_time ceph txt unknown /proc/41758/task/43021/exe (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43021 safe_time ceph NOFD /proc/41758/task/43021/fd (opendir: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43022 ceph-osd ceph cwd unknown /proc/41758/task/43022/cwd (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43022 ceph-osd ceph rtd unknown /proc/41758/task/43022/root (readlink: Permission denied) 2024-04-25T00:24:20.888 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43022 ceph-osd ceph txt unknown /proc/41758/task/43022/exe (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43022 ceph-osd ceph NOFD /proc/41758/task/43022/fd (opendir: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43023 osd_srv_a ceph cwd unknown /proc/41758/task/43023/cwd (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43023 osd_srv_a ceph rtd unknown /proc/41758/task/43023/root (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43023 osd_srv_a ceph txt unknown /proc/41758/task/43023/exe (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43023 osd_srv_a ceph NOFD /proc/41758/task/43023/fd (opendir: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43024 tp_osd_tp ceph cwd unknown /proc/41758/task/43024/cwd (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43024 tp_osd_tp ceph rtd unknown /proc/41758/task/43024/root (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43024 tp_osd_tp ceph txt unknown /proc/41758/task/43024/exe (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43024 tp_osd_tp ceph NOFD /proc/41758/task/43024/fd (opendir: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43025 tp_osd_tp ceph cwd unknown /proc/41758/task/43025/cwd (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43025 tp_osd_tp ceph rtd unknown /proc/41758/task/43025/root (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43025 tp_osd_tp ceph txt unknown /proc/41758/task/43025/exe (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43025 tp_osd_tp ceph NOFD /proc/41758/task/43025/fd (opendir: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43026 tp_osd_tp ceph cwd unknown /proc/41758/task/43026/cwd (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43026 tp_osd_tp ceph rtd unknown /proc/41758/task/43026/root (readlink: Permission denied) 2024-04-25T00:24:20.889 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43026 tp_osd_tp ceph txt unknown /proc/41758/task/43026/exe (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43026 tp_osd_tp ceph NOFD /proc/41758/task/43026/fd (opendir: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43027 tp_osd_tp ceph cwd unknown /proc/41758/task/43027/cwd (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43027 tp_osd_tp ceph rtd unknown /proc/41758/task/43027/root (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43027 tp_osd_tp ceph txt unknown /proc/41758/task/43027/exe (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43027 tp_osd_tp ceph NOFD /proc/41758/task/43027/fd (opendir: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43028 tp_osd_tp ceph cwd unknown /proc/41758/task/43028/cwd (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43028 tp_osd_tp ceph rtd unknown /proc/41758/task/43028/root (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43028 tp_osd_tp ceph txt unknown /proc/41758/task/43028/exe (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43028 tp_osd_tp ceph NOFD /proc/41758/task/43028/fd (opendir: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43029 tp_osd_tp ceph cwd unknown /proc/41758/task/43029/cwd (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43029 tp_osd_tp ceph rtd unknown /proc/41758/task/43029/root (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43029 tp_osd_tp ceph txt unknown /proc/41758/task/43029/exe (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43029 tp_osd_tp ceph NOFD /proc/41758/task/43029/fd (opendir: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43030 tp_osd_tp ceph cwd unknown /proc/41758/task/43030/cwd (readlink: Permission denied) 2024-04-25T00:24:20.890 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43030 tp_osd_tp ceph rtd unknown /proc/41758/task/43030/root (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43030 tp_osd_tp ceph txt unknown /proc/41758/task/43030/exe (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43030 tp_osd_tp ceph NOFD /proc/41758/task/43030/fd (opendir: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43031 tp_osd_tp ceph cwd unknown /proc/41758/task/43031/cwd (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43031 tp_osd_tp ceph rtd unknown /proc/41758/task/43031/root (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43031 tp_osd_tp ceph txt unknown /proc/41758/task/43031/exe (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43031 tp_osd_tp ceph NOFD /proc/41758/task/43031/fd (opendir: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43032 tp_osd_tp ceph cwd unknown /proc/41758/task/43032/cwd (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43032 tp_osd_tp ceph rtd unknown /proc/41758/task/43032/root (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43032 tp_osd_tp ceph txt unknown /proc/41758/task/43032/exe (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43032 tp_osd_tp ceph NOFD /proc/41758/task/43032/fd (opendir: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43033 tp_osd_tp ceph cwd unknown /proc/41758/task/43033/cwd (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43033 tp_osd_tp ceph rtd unknown /proc/41758/task/43033/root (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43033 tp_osd_tp ceph txt unknown /proc/41758/task/43033/exe (readlink: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43033 tp_osd_tp ceph NOFD /proc/41758/task/43033/fd (opendir: Permission denied) 2024-04-25T00:24:20.891 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43034 tp_osd_tp ceph cwd unknown /proc/41758/task/43034/cwd (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43034 tp_osd_tp ceph rtd unknown /proc/41758/task/43034/root (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43034 tp_osd_tp ceph txt unknown /proc/41758/task/43034/exe (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43034 tp_osd_tp ceph NOFD /proc/41758/task/43034/fd (opendir: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43035 tp_osd_tp ceph cwd unknown /proc/41758/task/43035/cwd (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43035 tp_osd_tp ceph rtd unknown /proc/41758/task/43035/root (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43035 tp_osd_tp ceph txt unknown /proc/41758/task/43035/exe (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43035 tp_osd_tp ceph NOFD /proc/41758/task/43035/fd (opendir: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43036 tp_osd_tp ceph cwd unknown /proc/41758/task/43036/cwd (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43036 tp_osd_tp ceph rtd unknown /proc/41758/task/43036/root (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43036 tp_osd_tp ceph txt unknown /proc/41758/task/43036/exe (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43036 tp_osd_tp ceph NOFD /proc/41758/task/43036/fd (opendir: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43037 tp_osd_tp ceph cwd unknown /proc/41758/task/43037/cwd (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43037 tp_osd_tp ceph rtd unknown /proc/41758/task/43037/root (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43037 tp_osd_tp ceph txt unknown /proc/41758/task/43037/exe (readlink: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43037 tp_osd_tp ceph NOFD /proc/41758/task/43037/fd (opendir: Permission denied) 2024-04-25T00:24:20.892 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43038 tp_osd_tp ceph cwd unknown /proc/41758/task/43038/cwd (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43038 tp_osd_tp ceph rtd unknown /proc/41758/task/43038/root (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43038 tp_osd_tp ceph txt unknown /proc/41758/task/43038/exe (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43038 tp_osd_tp ceph NOFD /proc/41758/task/43038/fd (opendir: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43039 tp_osd_tp ceph cwd unknown /proc/41758/task/43039/cwd (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43039 tp_osd_tp ceph rtd unknown /proc/41758/task/43039/root (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43039 tp_osd_tp ceph txt unknown /proc/41758/task/43039/exe (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43039 tp_osd_tp ceph NOFD /proc/41758/task/43039/fd (opendir: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43040 osd_srv_h ceph cwd unknown /proc/41758/task/43040/cwd (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43040 osd_srv_h ceph rtd unknown /proc/41758/task/43040/root (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43040 osd_srv_h ceph txt unknown /proc/41758/task/43040/exe (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 41758 43040 osd_srv_h ceph NOFD /proc/41758/task/43040/fd (opendir: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 43041 root cwd unknown /proc/43041/cwd (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 43041 root rtd unknown /proc/43041/root (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 43041 root txt unknown /proc/43041/exe (readlink: Permission denied) 2024-04-25T00:24:20.893 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 43041 root NOFD /proc/43041/fd (opendir: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 43059 root cwd unknown /proc/43059/cwd (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 43059 root rtd unknown /proc/43059/root (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 43059 root txt unknown /proc/43059/exe (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 43059 root NOFD /proc/43059/fd (opendir: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 48023 root cwd unknown /proc/48023/cwd (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 48023 root rtd unknown /proc/48023/root (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 48023 root txt unknown /proc/48023/exe (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 48023 root NOFD /proc/48023/fd (opendir: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 48319 root cwd unknown /proc/48319/cwd (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 48319 root rtd unknown /proc/48319/root (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 48319 root txt unknown /proc/48319/exe (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 48319 root NOFD /proc/48319/fd (opendir: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 48321 root cwd unknown /proc/48321/cwd (readlink: Permission denied) 2024-04-25T00:24:20.894 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 48321 root rtd unknown /proc/48321/root (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 48321 root txt unknown /proc/48321/exe (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 48321 root NOFD /proc/48321/fd (opendir: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 ceph cwd unknown /proc/48323/cwd (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 ceph rtd unknown /proc/48323/root (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 ceph txt unknown /proc/48323/exe (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 ceph NOFD /proc/48323/fd (opendir: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48324 log ceph cwd unknown /proc/48323/task/48324/cwd (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48324 log ceph rtd unknown /proc/48323/task/48324/root (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48324 log ceph txt unknown /proc/48323/task/48324/exe (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48324 log ceph NOFD /proc/48323/task/48324/fd (opendir: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48326 msgr-work ceph cwd unknown /proc/48323/task/48326/cwd (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48326 msgr-work ceph rtd unknown /proc/48323/task/48326/root (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48326 msgr-work ceph txt unknown /proc/48323/task/48326/exe (readlink: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48326 msgr-work ceph NOFD /proc/48323/task/48326/fd (opendir: Permission denied) 2024-04-25T00:24:20.895 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48327 msgr-work ceph cwd unknown /proc/48323/task/48327/cwd (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48327 msgr-work ceph rtd unknown /proc/48323/task/48327/root (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48327 msgr-work ceph txt unknown /proc/48323/task/48327/exe (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48327 msgr-work ceph NOFD /proc/48323/task/48327/fd (opendir: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48328 msgr-work ceph cwd unknown /proc/48323/task/48328/cwd (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48328 msgr-work ceph rtd unknown /proc/48323/task/48328/root (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48328 msgr-work ceph txt unknown /proc/48323/task/48328/exe (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48328 msgr-work ceph NOFD /proc/48323/task/48328/fd (opendir: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48332 service ceph cwd unknown /proc/48323/task/48332/cwd (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48332 service ceph rtd unknown /proc/48323/task/48332/root (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48332 service ceph txt unknown /proc/48323/task/48332/exe (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48332 service ceph NOFD /proc/48323/task/48332/fd (opendir: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48333 admin_soc ceph cwd unknown /proc/48323/task/48333/cwd (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48333 admin_soc ceph rtd unknown /proc/48323/task/48333/root (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48333 admin_soc ceph txt unknown /proc/48323/task/48333/exe (readlink: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48333 admin_soc ceph NOFD /proc/48323/task/48333/fd (opendir: Permission denied) 2024-04-25T00:24:20.896 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48475 signal_ha ceph cwd unknown /proc/48323/task/48475/cwd (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48475 signal_ha ceph rtd unknown /proc/48323/task/48475/root (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48475 signal_ha ceph txt unknown /proc/48323/task/48475/exe (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48475 signal_ha ceph NOFD /proc/48323/task/48475/fd (opendir: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48476 io_contex ceph cwd unknown /proc/48323/task/48476/cwd (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48476 io_contex ceph rtd unknown /proc/48323/task/48476/root (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48476 io_contex ceph txt unknown /proc/48323/task/48476/exe (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48476 io_contex ceph NOFD /proc/48323/task/48476/fd (opendir: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48477 io_contex ceph cwd unknown /proc/48323/task/48477/cwd (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48477 io_contex ceph rtd unknown /proc/48323/task/48477/root (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48477 io_contex ceph txt unknown /proc/48323/task/48477/exe (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48477 io_contex ceph NOFD /proc/48323/task/48477/fd (opendir: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48594 OpHistory ceph cwd unknown /proc/48323/task/48594/cwd (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48594 OpHistory ceph rtd unknown /proc/48323/task/48594/root (readlink: Permission denied) 2024-04-25T00:24:20.897 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48594 OpHistory ceph txt unknown /proc/48323/task/48594/exe (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48594 OpHistory ceph NOFD /proc/48323/task/48594/fd (opendir: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48595 ceph_time ceph cwd unknown /proc/48323/task/48595/cwd (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48595 ceph_time ceph rtd unknown /proc/48323/task/48595/root (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48595 ceph_time ceph txt unknown /proc/48323/task/48595/exe (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48595 ceph_time ceph NOFD /proc/48323/task/48595/fd (opendir: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48770 ceph-osd ceph cwd unknown /proc/48323/task/48770/cwd (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48770 ceph-osd ceph rtd unknown /proc/48323/task/48770/root (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48770 ceph-osd ceph txt unknown /proc/48323/task/48770/exe (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48770 ceph-osd ceph NOFD /proc/48323/task/48770/fd (opendir: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48771 safe_time ceph cwd unknown /proc/48323/task/48771/cwd (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48771 safe_time ceph rtd unknown /proc/48323/task/48771/root (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48771 safe_time ceph txt unknown /proc/48323/task/48771/exe (readlink: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48771 safe_time ceph NOFD /proc/48323/task/48771/fd (opendir: Permission denied) 2024-04-25T00:24:20.898 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48773 safe_time ceph cwd unknown /proc/48323/task/48773/cwd (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48773 safe_time ceph rtd unknown /proc/48323/task/48773/root (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48773 safe_time ceph txt unknown /proc/48323/task/48773/exe (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48773 safe_time ceph NOFD /proc/48323/task/48773/fd (opendir: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48774 safe_time ceph cwd unknown /proc/48323/task/48774/cwd (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48774 safe_time ceph rtd unknown /proc/48323/task/48774/root (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48774 safe_time ceph txt unknown /proc/48323/task/48774/exe (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48774 safe_time ceph NOFD /proc/48323/task/48774/fd (opendir: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48775 safe_time ceph cwd unknown /proc/48323/task/48775/cwd (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48775 safe_time ceph rtd unknown /proc/48323/task/48775/root (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48775 safe_time ceph txt unknown /proc/48323/task/48775/exe (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48775 safe_time ceph NOFD /proc/48323/task/48775/fd (opendir: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48776 fn_anonym ceph cwd unknown /proc/48323/task/48776/cwd (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48776 fn_anonym ceph rtd unknown /proc/48323/task/48776/root (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48776 fn_anonym ceph txt unknown /proc/48323/task/48776/exe (readlink: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48776 fn_anonym ceph NOFD /proc/48323/task/48776/fd (opendir: Permission denied) 2024-04-25T00:24:20.899 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48779 rocksdb:l ceph cwd unknown /proc/48323/task/48779/cwd (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48779 rocksdb:l ceph rtd unknown /proc/48323/task/48779/root (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48779 rocksdb:l ceph txt unknown /proc/48323/task/48779/exe (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48779 rocksdb:l ceph NOFD /proc/48323/task/48779/fd (opendir: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48780 rocksdb:l ceph cwd unknown /proc/48323/task/48780/cwd (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48780 rocksdb:l ceph rtd unknown /proc/48323/task/48780/root (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48780 rocksdb:l ceph txt unknown /proc/48323/task/48780/exe (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48780 rocksdb:l ceph NOFD /proc/48323/task/48780/fd (opendir: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48781 rocksdb:l ceph cwd unknown /proc/48323/task/48781/cwd (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48781 rocksdb:l ceph rtd unknown /proc/48323/task/48781/root (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48781 rocksdb:l ceph txt unknown /proc/48323/task/48781/exe (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48781 rocksdb:l ceph NOFD /proc/48323/task/48781/fd (opendir: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48782 rocksdb:h ceph cwd unknown /proc/48323/task/48782/cwd (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48782 rocksdb:h ceph rtd unknown /proc/48323/task/48782/root (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48782 rocksdb:h ceph txt unknown /proc/48323/task/48782/exe (readlink: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 48782 rocksdb:h ceph NOFD /proc/48323/task/48782/fd (opendir: Permission denied) 2024-04-25T00:24:20.900 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49190 bstore_ai ceph cwd unknown /proc/48323/task/49190/cwd (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49190 bstore_ai ceph rtd unknown /proc/48323/task/49190/root (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49190 bstore_ai ceph txt unknown /proc/48323/task/49190/exe (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49190 bstore_ai ceph NOFD /proc/48323/task/49190/fd (opendir: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49389 bstore_ai ceph cwd unknown /proc/48323/task/49389/cwd (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49389 bstore_ai ceph rtd unknown /proc/48323/task/49389/root (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49389 bstore_ai ceph txt unknown /proc/48323/task/49389/exe (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49389 bstore_ai ceph NOFD /proc/48323/task/49389/fd (opendir: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49571 ceph-osd ceph cwd unknown /proc/48323/task/49571/cwd (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49571 ceph-osd ceph rtd unknown /proc/48323/task/49571/root (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49571 ceph-osd ceph txt unknown /proc/48323/task/49571/exe (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49571 ceph-osd ceph NOFD /proc/48323/task/49571/fd (opendir: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49572 cfin ceph cwd unknown /proc/48323/task/49572/cwd (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49572 cfin ceph rtd unknown /proc/48323/task/49572/root (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49572 cfin ceph txt unknown /proc/48323/task/49572/exe (readlink: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49572 cfin ceph NOFD /proc/48323/task/49572/fd (opendir: Permission denied) 2024-04-25T00:24:20.901 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49573 bstore_kv ceph cwd unknown /proc/48323/task/49573/cwd (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49573 bstore_kv ceph rtd unknown /proc/48323/task/49573/root (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49573 bstore_kv ceph txt unknown /proc/48323/task/49573/exe (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49573 bstore_kv ceph NOFD /proc/48323/task/49573/fd (opendir: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49574 bstore_kv ceph cwd unknown /proc/48323/task/49574/cwd (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49574 bstore_kv ceph rtd unknown /proc/48323/task/49574/root (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49574 bstore_kv ceph txt unknown /proc/48323/task/49574/exe (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49574 bstore_kv ceph NOFD /proc/48323/task/49574/fd (opendir: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49575 bstore_me ceph cwd unknown /proc/48323/task/49575/cwd (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49575 bstore_me ceph rtd unknown /proc/48323/task/49575/root (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49575 bstore_me ceph txt unknown /proc/48323/task/49575/exe (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49575 bstore_me ceph NOFD /proc/48323/task/49575/fd (opendir: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49576 ms_dispat ceph cwd unknown /proc/48323/task/49576/cwd (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49576 ms_dispat ceph rtd unknown /proc/48323/task/49576/root (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49576 ms_dispat ceph txt unknown /proc/48323/task/49576/exe (readlink: Permission denied) 2024-04-25T00:24:20.902 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49576 ms_dispat ceph NOFD /proc/48323/task/49576/fd (opendir: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49577 ms_local ceph cwd unknown /proc/48323/task/49577/cwd (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49577 ms_local ceph rtd unknown /proc/48323/task/49577/root (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49577 ms_local ceph txt unknown /proc/48323/task/49577/exe (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49577 ms_local ceph NOFD /proc/48323/task/49577/fd (opendir: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49578 safe_time ceph cwd unknown /proc/48323/task/49578/cwd (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49578 safe_time ceph rtd unknown /proc/48323/task/49578/root (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49578 safe_time ceph txt unknown /proc/48323/task/49578/exe (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49578 safe_time ceph NOFD /proc/48323/task/49578/fd (opendir: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49579 safe_time ceph cwd unknown /proc/48323/task/49579/cwd (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49579 safe_time ceph rtd unknown /proc/48323/task/49579/root (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49579 safe_time ceph txt unknown /proc/48323/task/49579/exe (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49579 safe_time ceph NOFD /proc/48323/task/49579/fd (opendir: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49580 ms_dispat ceph cwd unknown /proc/48323/task/49580/cwd (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49580 ms_dispat ceph rtd unknown /proc/48323/task/49580/root (readlink: Permission denied) 2024-04-25T00:24:20.903 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49580 ms_dispat ceph txt unknown /proc/48323/task/49580/exe (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49580 ms_dispat ceph NOFD /proc/48323/task/49580/fd (opendir: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49581 ms_local ceph cwd unknown /proc/48323/task/49581/cwd (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49581 ms_local ceph rtd unknown /proc/48323/task/49581/root (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49581 ms_local ceph txt unknown /proc/48323/task/49581/exe (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49581 ms_local ceph NOFD /proc/48323/task/49581/fd (opendir: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49582 ms_dispat ceph cwd unknown /proc/48323/task/49582/cwd (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49582 ms_dispat ceph rtd unknown /proc/48323/task/49582/root (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49582 ms_dispat ceph txt unknown /proc/48323/task/49582/exe (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49582 ms_dispat ceph NOFD /proc/48323/task/49582/fd (opendir: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49583 ms_local ceph cwd unknown /proc/48323/task/49583/cwd (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49583 ms_local ceph rtd unknown /proc/48323/task/49583/root (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49583 ms_local ceph txt unknown /proc/48323/task/49583/exe (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49583 ms_local ceph NOFD /proc/48323/task/49583/fd (opendir: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49584 ms_dispat ceph cwd unknown /proc/48323/task/49584/cwd (readlink: Permission denied) 2024-04-25T00:24:20.904 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49584 ms_dispat ceph rtd unknown /proc/48323/task/49584/root (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49584 ms_dispat ceph txt unknown /proc/48323/task/49584/exe (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49584 ms_dispat ceph NOFD /proc/48323/task/49584/fd (opendir: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49585 ms_local ceph cwd unknown /proc/48323/task/49585/cwd (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49585 ms_local ceph rtd unknown /proc/48323/task/49585/root (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49585 ms_local ceph txt unknown /proc/48323/task/49585/exe (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49585 ms_local ceph NOFD /proc/48323/task/49585/fd (opendir: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49586 ms_dispat ceph cwd unknown /proc/48323/task/49586/cwd (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49586 ms_dispat ceph rtd unknown /proc/48323/task/49586/root (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49586 ms_dispat ceph txt unknown /proc/48323/task/49586/exe (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49586 ms_dispat ceph NOFD /proc/48323/task/49586/fd (opendir: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49587 ms_local ceph cwd unknown /proc/48323/task/49587/cwd (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49587 ms_local ceph rtd unknown /proc/48323/task/49587/root (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49587 ms_local ceph txt unknown /proc/48323/task/49587/exe (readlink: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49587 ms_local ceph NOFD /proc/48323/task/49587/fd (opendir: Permission denied) 2024-04-25T00:24:20.905 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49588 ms_dispat ceph cwd unknown /proc/48323/task/49588/cwd (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49588 ms_dispat ceph rtd unknown /proc/48323/task/49588/root (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49588 ms_dispat ceph txt unknown /proc/48323/task/49588/exe (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49588 ms_dispat ceph NOFD /proc/48323/task/49588/fd (opendir: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49589 ms_local ceph cwd unknown /proc/48323/task/49589/cwd (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49589 ms_local ceph rtd unknown /proc/48323/task/49589/root (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49589 ms_local ceph txt unknown /proc/48323/task/49589/exe (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49589 ms_local ceph NOFD /proc/48323/task/49589/fd (opendir: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49590 ms_dispat ceph cwd unknown /proc/48323/task/49590/cwd (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49590 ms_dispat ceph rtd unknown /proc/48323/task/49590/root (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49590 ms_dispat ceph txt unknown /proc/48323/task/49590/exe (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49590 ms_dispat ceph NOFD /proc/48323/task/49590/fd (opendir: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49591 ms_local ceph cwd unknown /proc/48323/task/49591/cwd (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49591 ms_local ceph rtd unknown /proc/48323/task/49591/root (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49591 ms_local ceph txt unknown /proc/48323/task/49591/exe (readlink: Permission denied) 2024-04-25T00:24:20.906 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49591 ms_local ceph NOFD /proc/48323/task/49591/fd (opendir: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49592 fn_anonym ceph cwd unknown /proc/48323/task/49592/cwd (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49592 fn_anonym ceph rtd unknown /proc/48323/task/49592/root (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49592 fn_anonym ceph txt unknown /proc/48323/task/49592/exe (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49592 fn_anonym ceph NOFD /proc/48323/task/49592/fd (opendir: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49593 finisher ceph cwd unknown /proc/48323/task/49593/cwd (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49593 finisher ceph rtd unknown /proc/48323/task/49593/root (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49593 finisher ceph txt unknown /proc/48323/task/49593/exe (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49593 finisher ceph NOFD /proc/48323/task/49593/fd (opendir: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49594 safe_time ceph cwd unknown /proc/48323/task/49594/cwd (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49594 safe_time ceph rtd unknown /proc/48323/task/49594/root (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49594 safe_time ceph txt unknown /proc/48323/task/49594/exe (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49594 safe_time ceph NOFD /proc/48323/task/49594/fd (opendir: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49595 safe_time ceph cwd unknown /proc/48323/task/49595/cwd (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49595 safe_time ceph rtd unknown /proc/48323/task/49595/root (readlink: Permission denied) 2024-04-25T00:24:20.907 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49595 safe_time ceph txt unknown /proc/48323/task/49595/exe (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49595 safe_time ceph NOFD /proc/48323/task/49595/fd (opendir: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49596 ceph-osd ceph cwd unknown /proc/48323/task/49596/cwd (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49596 ceph-osd ceph rtd unknown /proc/48323/task/49596/root (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49596 ceph-osd ceph txt unknown /proc/48323/task/49596/exe (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49596 ceph-osd ceph NOFD /proc/48323/task/49596/fd (opendir: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49597 osd_srv_a ceph cwd unknown /proc/48323/task/49597/cwd (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49597 osd_srv_a ceph rtd unknown /proc/48323/task/49597/root (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49597 osd_srv_a ceph txt unknown /proc/48323/task/49597/exe (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49597 osd_srv_a ceph NOFD /proc/48323/task/49597/fd (opendir: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49598 tp_osd_tp ceph cwd unknown /proc/48323/task/49598/cwd (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49598 tp_osd_tp ceph rtd unknown /proc/48323/task/49598/root (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49598 tp_osd_tp ceph txt unknown /proc/48323/task/49598/exe (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49598 tp_osd_tp ceph NOFD /proc/48323/task/49598/fd (opendir: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49599 tp_osd_tp ceph cwd unknown /proc/48323/task/49599/cwd (readlink: Permission denied) 2024-04-25T00:24:20.908 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49599 tp_osd_tp ceph rtd unknown /proc/48323/task/49599/root (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49599 tp_osd_tp ceph txt unknown /proc/48323/task/49599/exe (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49599 tp_osd_tp ceph NOFD /proc/48323/task/49599/fd (opendir: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49600 tp_osd_tp ceph cwd unknown /proc/48323/task/49600/cwd (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49600 tp_osd_tp ceph rtd unknown /proc/48323/task/49600/root (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49600 tp_osd_tp ceph txt unknown /proc/48323/task/49600/exe (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49600 tp_osd_tp ceph NOFD /proc/48323/task/49600/fd (opendir: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49601 tp_osd_tp ceph cwd unknown /proc/48323/task/49601/cwd (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49601 tp_osd_tp ceph rtd unknown /proc/48323/task/49601/root (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49601 tp_osd_tp ceph txt unknown /proc/48323/task/49601/exe (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49601 tp_osd_tp ceph NOFD /proc/48323/task/49601/fd (opendir: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49602 tp_osd_tp ceph cwd unknown /proc/48323/task/49602/cwd (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49602 tp_osd_tp ceph rtd unknown /proc/48323/task/49602/root (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49602 tp_osd_tp ceph txt unknown /proc/48323/task/49602/exe (readlink: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49602 tp_osd_tp ceph NOFD /proc/48323/task/49602/fd (opendir: Permission denied) 2024-04-25T00:24:20.909 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49603 tp_osd_tp ceph cwd unknown /proc/48323/task/49603/cwd (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49603 tp_osd_tp ceph rtd unknown /proc/48323/task/49603/root (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49603 tp_osd_tp ceph txt unknown /proc/48323/task/49603/exe (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49603 tp_osd_tp ceph NOFD /proc/48323/task/49603/fd (opendir: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49604 tp_osd_tp ceph cwd unknown /proc/48323/task/49604/cwd (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49604 tp_osd_tp ceph rtd unknown /proc/48323/task/49604/root (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49604 tp_osd_tp ceph txt unknown /proc/48323/task/49604/exe (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49604 tp_osd_tp ceph NOFD /proc/48323/task/49604/fd (opendir: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49605 tp_osd_tp ceph cwd unknown /proc/48323/task/49605/cwd (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49605 tp_osd_tp ceph rtd unknown /proc/48323/task/49605/root (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49605 tp_osd_tp ceph txt unknown /proc/48323/task/49605/exe (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49605 tp_osd_tp ceph NOFD /proc/48323/task/49605/fd (opendir: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49606 tp_osd_tp ceph cwd unknown /proc/48323/task/49606/cwd (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49606 tp_osd_tp ceph rtd unknown /proc/48323/task/49606/root (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49606 tp_osd_tp ceph txt unknown /proc/48323/task/49606/exe (readlink: Permission denied) 2024-04-25T00:24:20.910 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49606 tp_osd_tp ceph NOFD /proc/48323/task/49606/fd (opendir: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49607 tp_osd_tp ceph cwd unknown /proc/48323/task/49607/cwd (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49607 tp_osd_tp ceph rtd unknown /proc/48323/task/49607/root (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49607 tp_osd_tp ceph txt unknown /proc/48323/task/49607/exe (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49607 tp_osd_tp ceph NOFD /proc/48323/task/49607/fd (opendir: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49608 tp_osd_tp ceph cwd unknown /proc/48323/task/49608/cwd (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49608 tp_osd_tp ceph rtd unknown /proc/48323/task/49608/root (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49608 tp_osd_tp ceph txt unknown /proc/48323/task/49608/exe (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49608 tp_osd_tp ceph NOFD /proc/48323/task/49608/fd (opendir: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49609 tp_osd_tp ceph cwd unknown /proc/48323/task/49609/cwd (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49609 tp_osd_tp ceph rtd unknown /proc/48323/task/49609/root (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49609 tp_osd_tp ceph txt unknown /proc/48323/task/49609/exe (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49609 tp_osd_tp ceph NOFD /proc/48323/task/49609/fd (opendir: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49610 tp_osd_tp ceph cwd unknown /proc/48323/task/49610/cwd (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49610 tp_osd_tp ceph rtd unknown /proc/48323/task/49610/root (readlink: Permission denied) 2024-04-25T00:24:20.911 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49610 tp_osd_tp ceph txt unknown /proc/48323/task/49610/exe (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49610 tp_osd_tp ceph NOFD /proc/48323/task/49610/fd (opendir: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49611 tp_osd_tp ceph cwd unknown /proc/48323/task/49611/cwd (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49611 tp_osd_tp ceph rtd unknown /proc/48323/task/49611/root (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49611 tp_osd_tp ceph txt unknown /proc/48323/task/49611/exe (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49611 tp_osd_tp ceph NOFD /proc/48323/task/49611/fd (opendir: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49612 tp_osd_tp ceph cwd unknown /proc/48323/task/49612/cwd (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49612 tp_osd_tp ceph rtd unknown /proc/48323/task/49612/root (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49612 tp_osd_tp ceph txt unknown /proc/48323/task/49612/exe (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49612 tp_osd_tp ceph NOFD /proc/48323/task/49612/fd (opendir: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49613 tp_osd_tp ceph cwd unknown /proc/48323/task/49613/cwd (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49613 tp_osd_tp ceph rtd unknown /proc/48323/task/49613/root (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49613 tp_osd_tp ceph txt unknown /proc/48323/task/49613/exe (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49613 tp_osd_tp ceph NOFD /proc/48323/task/49613/fd (opendir: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49614 osd_srv_h ceph cwd unknown /proc/48323/task/49614/cwd (readlink: Permission denied) 2024-04-25T00:24:20.912 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49614 osd_srv_h ceph rtd unknown /proc/48323/task/49614/root (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49614 osd_srv_h ceph txt unknown /proc/48323/task/49614/exe (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 48323 49614 osd_srv_h ceph NOFD /proc/48323/task/49614/fd (opendir: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 49615 root cwd unknown /proc/49615/cwd (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 49615 root rtd unknown /proc/49615/root (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 49615 root txt unknown /proc/49615/exe (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 49615 root NOFD /proc/49615/fd (opendir: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 49634 root cwd unknown /proc/49634/cwd (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 49634 root rtd unknown /proc/49634/root (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 49634 root txt unknown /proc/49634/exe (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 49634 root NOFD /proc/49634/fd (opendir: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 52017 root cwd unknown /proc/52017/cwd (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 52017 root rtd unknown /proc/52017/root (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 52017 root txt unknown /proc/52017/exe (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 52017 root NOFD /proc/52017/fd (opendir: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 54141 root cwd unknown /proc/54141/cwd (readlink: Permission denied) 2024-04-25T00:24:20.913 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 54141 root rtd unknown /proc/54141/root (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 54141 root txt unknown /proc/54141/exe (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 54141 root NOFD /proc/54141/fd (opendir: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 54873 root cwd unknown /proc/54873/cwd (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 54873 root rtd unknown /proc/54873/root (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 54873 root txt unknown /proc/54873/exe (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 54873 root NOFD /proc/54873/fd (opendir: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 54875 root cwd unknown /proc/54875/cwd (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 54875 root rtd unknown /proc/54875/root (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 54875 root txt unknown /proc/54875/exe (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 54875 root NOFD /proc/54875/fd (opendir: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 ceph cwd unknown /proc/54877/cwd (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 ceph rtd unknown /proc/54877/root (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 ceph txt unknown /proc/54877/exe (readlink: Permission denied) 2024-04-25T00:24:20.914 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 ceph NOFD /proc/54877/fd (opendir: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54878 log ceph cwd unknown /proc/54877/task/54878/cwd (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54878 log ceph rtd unknown /proc/54877/task/54878/root (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54878 log ceph txt unknown /proc/54877/task/54878/exe (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54878 log ceph NOFD /proc/54877/task/54878/fd (opendir: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54880 msgr-work ceph cwd unknown /proc/54877/task/54880/cwd (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54880 msgr-work ceph rtd unknown /proc/54877/task/54880/root (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54880 msgr-work ceph txt unknown /proc/54877/task/54880/exe (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54880 msgr-work ceph NOFD /proc/54877/task/54880/fd (opendir: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54881 msgr-work ceph cwd unknown /proc/54877/task/54881/cwd (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54881 msgr-work ceph rtd unknown /proc/54877/task/54881/root (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54881 msgr-work ceph txt unknown /proc/54877/task/54881/exe (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54881 msgr-work ceph NOFD /proc/54877/task/54881/fd (opendir: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54882 msgr-work ceph cwd unknown /proc/54877/task/54882/cwd (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54882 msgr-work ceph rtd unknown /proc/54877/task/54882/root (readlink: Permission denied) 2024-04-25T00:24:20.915 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54882 msgr-work ceph txt unknown /proc/54877/task/54882/exe (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54882 msgr-work ceph NOFD /proc/54877/task/54882/fd (opendir: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54886 service ceph cwd unknown /proc/54877/task/54886/cwd (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54886 service ceph rtd unknown /proc/54877/task/54886/root (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54886 service ceph txt unknown /proc/54877/task/54886/exe (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54886 service ceph NOFD /proc/54877/task/54886/fd (opendir: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54887 admin_soc ceph cwd unknown /proc/54877/task/54887/cwd (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54887 admin_soc ceph rtd unknown /proc/54877/task/54887/root (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54887 admin_soc ceph txt unknown /proc/54877/task/54887/exe (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 54887 admin_soc ceph NOFD /proc/54877/task/54887/fd (opendir: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55029 signal_ha ceph cwd unknown /proc/54877/task/55029/cwd (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55029 signal_ha ceph rtd unknown /proc/54877/task/55029/root (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55029 signal_ha ceph txt unknown /proc/54877/task/55029/exe (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55029 signal_ha ceph NOFD /proc/54877/task/55029/fd (opendir: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55030 io_contex ceph cwd unknown /proc/54877/task/55030/cwd (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55030 io_contex ceph rtd unknown /proc/54877/task/55030/root (readlink: Permission denied) 2024-04-25T00:24:20.916 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55030 io_contex ceph txt unknown /proc/54877/task/55030/exe (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55030 io_contex ceph NOFD /proc/54877/task/55030/fd (opendir: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55031 io_contex ceph cwd unknown /proc/54877/task/55031/cwd (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55031 io_contex ceph rtd unknown /proc/54877/task/55031/root (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55031 io_contex ceph txt unknown /proc/54877/task/55031/exe (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55031 io_contex ceph NOFD /proc/54877/task/55031/fd (opendir: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55144 OpHistory ceph cwd unknown /proc/54877/task/55144/cwd (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55144 OpHistory ceph rtd unknown /proc/54877/task/55144/root (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55144 OpHistory ceph txt unknown /proc/54877/task/55144/exe (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55144 OpHistory ceph NOFD /proc/54877/task/55144/fd (opendir: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55145 ceph_time ceph cwd unknown /proc/54877/task/55145/cwd (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55145 ceph_time ceph rtd unknown /proc/54877/task/55145/root (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55145 ceph_time ceph txt unknown /proc/54877/task/55145/exe (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55145 ceph_time ceph NOFD /proc/54877/task/55145/fd (opendir: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55166 ceph-osd ceph cwd unknown /proc/54877/task/55166/cwd (readlink: Permission denied) 2024-04-25T00:24:20.917 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55166 ceph-osd ceph rtd unknown /proc/54877/task/55166/root (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55166 ceph-osd ceph txt unknown /proc/54877/task/55166/exe (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55166 ceph-osd ceph NOFD /proc/54877/task/55166/fd (opendir: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55183 ceph-osd ceph cwd unknown /proc/54877/task/55183/cwd (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55183 ceph-osd ceph rtd unknown /proc/54877/task/55183/root (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55183 ceph-osd ceph txt unknown /proc/54877/task/55183/exe (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55183 ceph-osd ceph NOFD /proc/54877/task/55183/fd (opendir: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55203 ceph-osd ceph cwd unknown /proc/54877/task/55203/cwd (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55203 ceph-osd ceph rtd unknown /proc/54877/task/55203/root (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55203 ceph-osd ceph txt unknown /proc/54877/task/55203/exe (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55203 ceph-osd ceph NOFD /proc/54877/task/55203/fd (opendir: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55225 ceph-osd ceph cwd unknown /proc/54877/task/55225/cwd (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55225 ceph-osd ceph rtd unknown /proc/54877/task/55225/root (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55225 ceph-osd ceph txt unknown /proc/54877/task/55225/exe (readlink: Permission denied) 2024-04-25T00:24:20.918 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55225 ceph-osd ceph NOFD /proc/54877/task/55225/fd (opendir: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55246 ceph-osd ceph cwd unknown /proc/54877/task/55246/cwd (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55246 ceph-osd ceph rtd unknown /proc/54877/task/55246/root (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55246 ceph-osd ceph txt unknown /proc/54877/task/55246/exe (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55246 ceph-osd ceph NOFD /proc/54877/task/55246/fd (opendir: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55249 ceph-osd ceph cwd unknown /proc/54877/task/55249/cwd (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55249 ceph-osd ceph rtd unknown /proc/54877/task/55249/root (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55249 ceph-osd ceph txt unknown /proc/54877/task/55249/exe (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55249 ceph-osd ceph NOFD /proc/54877/task/55249/fd (opendir: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55292 ceph-osd ceph cwd unknown /proc/54877/task/55292/cwd (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55292 ceph-osd ceph rtd unknown /proc/54877/task/55292/root (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55292 ceph-osd ceph txt unknown /proc/54877/task/55292/exe (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55292 ceph-osd ceph NOFD /proc/54877/task/55292/fd (opendir: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55301 ceph-osd ceph cwd unknown /proc/54877/task/55301/cwd (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55301 ceph-osd ceph rtd unknown /proc/54877/task/55301/root (readlink: Permission denied) 2024-04-25T00:24:20.919 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55301 ceph-osd ceph txt unknown /proc/54877/task/55301/exe (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55301 ceph-osd ceph NOFD /proc/54877/task/55301/fd (opendir: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55302 ceph-osd ceph cwd unknown /proc/54877/task/55302/cwd (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55302 ceph-osd ceph rtd unknown /proc/54877/task/55302/root (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55302 ceph-osd ceph txt unknown /proc/54877/task/55302/exe (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55302 ceph-osd ceph NOFD /proc/54877/task/55302/fd (opendir: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55303 safe_time ceph cwd unknown /proc/54877/task/55303/cwd (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55303 safe_time ceph rtd unknown /proc/54877/task/55303/root (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55303 safe_time ceph txt unknown /proc/54877/task/55303/exe (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55303 safe_time ceph NOFD /proc/54877/task/55303/fd (opendir: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55304 safe_time ceph cwd unknown /proc/54877/task/55304/cwd (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55304 safe_time ceph rtd unknown /proc/54877/task/55304/root (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55304 safe_time ceph txt unknown /proc/54877/task/55304/exe (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55304 safe_time ceph NOFD /proc/54877/task/55304/fd (opendir: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55305 safe_time ceph cwd unknown /proc/54877/task/55305/cwd (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55305 safe_time ceph rtd unknown /proc/54877/task/55305/root (readlink: Permission denied) 2024-04-25T00:24:20.920 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55305 safe_time ceph txt unknown /proc/54877/task/55305/exe (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55305 safe_time ceph NOFD /proc/54877/task/55305/fd (opendir: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55306 safe_time ceph cwd unknown /proc/54877/task/55306/cwd (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55306 safe_time ceph rtd unknown /proc/54877/task/55306/root (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55306 safe_time ceph txt unknown /proc/54877/task/55306/exe (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55306 safe_time ceph NOFD /proc/54877/task/55306/fd (opendir: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55307 fn_anonym ceph cwd unknown /proc/54877/task/55307/cwd (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55307 fn_anonym ceph rtd unknown /proc/54877/task/55307/root (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55307 fn_anonym ceph txt unknown /proc/54877/task/55307/exe (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55307 fn_anonym ceph NOFD /proc/54877/task/55307/fd (opendir: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55311 rocksdb:l ceph cwd unknown /proc/54877/task/55311/cwd (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55311 rocksdb:l ceph rtd unknown /proc/54877/task/55311/root (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55311 rocksdb:l ceph txt unknown /proc/54877/task/55311/exe (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55311 rocksdb:l ceph NOFD /proc/54877/task/55311/fd (opendir: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55312 rocksdb:l ceph cwd unknown /proc/54877/task/55312/cwd (readlink: Permission denied) 2024-04-25T00:24:20.921 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55312 rocksdb:l ceph rtd unknown /proc/54877/task/55312/root (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55312 rocksdb:l ceph txt unknown /proc/54877/task/55312/exe (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55312 rocksdb:l ceph NOFD /proc/54877/task/55312/fd (opendir: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55313 rocksdb:l ceph cwd unknown /proc/54877/task/55313/cwd (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55313 rocksdb:l ceph rtd unknown /proc/54877/task/55313/root (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55313 rocksdb:l ceph txt unknown /proc/54877/task/55313/exe (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55313 rocksdb:l ceph NOFD /proc/54877/task/55313/fd (opendir: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55314 rocksdb:h ceph cwd unknown /proc/54877/task/55314/cwd (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55314 rocksdb:h ceph rtd unknown /proc/54877/task/55314/root (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55314 rocksdb:h ceph txt unknown /proc/54877/task/55314/exe (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55314 rocksdb:h ceph NOFD /proc/54877/task/55314/fd (opendir: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55801 bstore_ai ceph cwd unknown /proc/54877/task/55801/cwd (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55801 bstore_ai ceph rtd unknown /proc/54877/task/55801/root (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55801 bstore_ai ceph txt unknown /proc/54877/task/55801/exe (readlink: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55801 bstore_ai ceph NOFD /proc/54877/task/55801/fd (opendir: Permission denied) 2024-04-25T00:24:20.922 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55986 bstore_ai ceph cwd unknown /proc/54877/task/55986/cwd (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55986 bstore_ai ceph rtd unknown /proc/54877/task/55986/root (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55986 bstore_ai ceph txt unknown /proc/54877/task/55986/exe (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 55986 bstore_ai ceph NOFD /proc/54877/task/55986/fd (opendir: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56168 ceph-osd ceph cwd unknown /proc/54877/task/56168/cwd (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56168 ceph-osd ceph rtd unknown /proc/54877/task/56168/root (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56168 ceph-osd ceph txt unknown /proc/54877/task/56168/exe (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56168 ceph-osd ceph NOFD /proc/54877/task/56168/fd (opendir: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56169 cfin ceph cwd unknown /proc/54877/task/56169/cwd (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56169 cfin ceph rtd unknown /proc/54877/task/56169/root (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56169 cfin ceph txt unknown /proc/54877/task/56169/exe (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56169 cfin ceph NOFD /proc/54877/task/56169/fd (opendir: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56170 bstore_kv ceph cwd unknown /proc/54877/task/56170/cwd (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56170 bstore_kv ceph rtd unknown /proc/54877/task/56170/root (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56170 bstore_kv ceph txt unknown /proc/54877/task/56170/exe (readlink: Permission denied) 2024-04-25T00:24:20.923 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56170 bstore_kv ceph NOFD /proc/54877/task/56170/fd (opendir: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56171 bstore_kv ceph cwd unknown /proc/54877/task/56171/cwd (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56171 bstore_kv ceph rtd unknown /proc/54877/task/56171/root (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56171 bstore_kv ceph txt unknown /proc/54877/task/56171/exe (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56171 bstore_kv ceph NOFD /proc/54877/task/56171/fd (opendir: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56172 bstore_me ceph cwd unknown /proc/54877/task/56172/cwd (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56172 bstore_me ceph rtd unknown /proc/54877/task/56172/root (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56172 bstore_me ceph txt unknown /proc/54877/task/56172/exe (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56172 bstore_me ceph NOFD /proc/54877/task/56172/fd (opendir: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56173 ms_dispat ceph cwd unknown /proc/54877/task/56173/cwd (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56173 ms_dispat ceph rtd unknown /proc/54877/task/56173/root (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56173 ms_dispat ceph txt unknown /proc/54877/task/56173/exe (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56173 ms_dispat ceph NOFD /proc/54877/task/56173/fd (opendir: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56174 ms_local ceph cwd unknown /proc/54877/task/56174/cwd (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56174 ms_local ceph rtd unknown /proc/54877/task/56174/root (readlink: Permission denied) 2024-04-25T00:24:20.924 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56174 ms_local ceph txt unknown /proc/54877/task/56174/exe (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56174 ms_local ceph NOFD /proc/54877/task/56174/fd (opendir: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56175 safe_time ceph cwd unknown /proc/54877/task/56175/cwd (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56175 safe_time ceph rtd unknown /proc/54877/task/56175/root (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56175 safe_time ceph txt unknown /proc/54877/task/56175/exe (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56175 safe_time ceph NOFD /proc/54877/task/56175/fd (opendir: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56176 safe_time ceph cwd unknown /proc/54877/task/56176/cwd (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56176 safe_time ceph rtd unknown /proc/54877/task/56176/root (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56176 safe_time ceph txt unknown /proc/54877/task/56176/exe (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56176 safe_time ceph NOFD /proc/54877/task/56176/fd (opendir: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56177 ms_dispat ceph cwd unknown /proc/54877/task/56177/cwd (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56177 ms_dispat ceph rtd unknown /proc/54877/task/56177/root (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56177 ms_dispat ceph txt unknown /proc/54877/task/56177/exe (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56177 ms_dispat ceph NOFD /proc/54877/task/56177/fd (opendir: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56178 ms_local ceph cwd unknown /proc/54877/task/56178/cwd (readlink: Permission denied) 2024-04-25T00:24:20.925 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56178 ms_local ceph rtd unknown /proc/54877/task/56178/root (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56178 ms_local ceph txt unknown /proc/54877/task/56178/exe (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56178 ms_local ceph NOFD /proc/54877/task/56178/fd (opendir: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56179 ms_dispat ceph cwd unknown /proc/54877/task/56179/cwd (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56179 ms_dispat ceph rtd unknown /proc/54877/task/56179/root (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56179 ms_dispat ceph txt unknown /proc/54877/task/56179/exe (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56179 ms_dispat ceph NOFD /proc/54877/task/56179/fd (opendir: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56180 ms_local ceph cwd unknown /proc/54877/task/56180/cwd (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56180 ms_local ceph rtd unknown /proc/54877/task/56180/root (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56180 ms_local ceph txt unknown /proc/54877/task/56180/exe (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56180 ms_local ceph NOFD /proc/54877/task/56180/fd (opendir: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56181 ms_dispat ceph cwd unknown /proc/54877/task/56181/cwd (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56181 ms_dispat ceph rtd unknown /proc/54877/task/56181/root (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56181 ms_dispat ceph txt unknown /proc/54877/task/56181/exe (readlink: Permission denied) 2024-04-25T00:24:20.926 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56181 ms_dispat ceph NOFD /proc/54877/task/56181/fd (opendir: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56182 ms_local ceph cwd unknown /proc/54877/task/56182/cwd (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56182 ms_local ceph rtd unknown /proc/54877/task/56182/root (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56182 ms_local ceph txt unknown /proc/54877/task/56182/exe (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56182 ms_local ceph NOFD /proc/54877/task/56182/fd (opendir: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56183 ms_dispat ceph cwd unknown /proc/54877/task/56183/cwd (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56183 ms_dispat ceph rtd unknown /proc/54877/task/56183/root (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56183 ms_dispat ceph txt unknown /proc/54877/task/56183/exe (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56183 ms_dispat ceph NOFD /proc/54877/task/56183/fd (opendir: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56184 ms_local ceph cwd unknown /proc/54877/task/56184/cwd (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56184 ms_local ceph rtd unknown /proc/54877/task/56184/root (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56184 ms_local ceph txt unknown /proc/54877/task/56184/exe (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56184 ms_local ceph NOFD /proc/54877/task/56184/fd (opendir: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56185 ms_dispat ceph cwd unknown /proc/54877/task/56185/cwd (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56185 ms_dispat ceph rtd unknown /proc/54877/task/56185/root (readlink: Permission denied) 2024-04-25T00:24:20.927 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56185 ms_dispat ceph txt unknown /proc/54877/task/56185/exe (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56185 ms_dispat ceph NOFD /proc/54877/task/56185/fd (opendir: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56186 ms_local ceph cwd unknown /proc/54877/task/56186/cwd (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56186 ms_local ceph rtd unknown /proc/54877/task/56186/root (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56186 ms_local ceph txt unknown /proc/54877/task/56186/exe (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56186 ms_local ceph NOFD /proc/54877/task/56186/fd (opendir: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56187 ms_dispat ceph cwd unknown /proc/54877/task/56187/cwd (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56187 ms_dispat ceph rtd unknown /proc/54877/task/56187/root (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56187 ms_dispat ceph txt unknown /proc/54877/task/56187/exe (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56187 ms_dispat ceph NOFD /proc/54877/task/56187/fd (opendir: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56188 ms_local ceph cwd unknown /proc/54877/task/56188/cwd (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56188 ms_local ceph rtd unknown /proc/54877/task/56188/root (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56188 ms_local ceph txt unknown /proc/54877/task/56188/exe (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56188 ms_local ceph NOFD /proc/54877/task/56188/fd (opendir: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56189 fn_anonym ceph cwd unknown /proc/54877/task/56189/cwd (readlink: Permission denied) 2024-04-25T00:24:20.928 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56189 fn_anonym ceph rtd unknown /proc/54877/task/56189/root (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56189 fn_anonym ceph txt unknown /proc/54877/task/56189/exe (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56189 fn_anonym ceph NOFD /proc/54877/task/56189/fd (opendir: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56190 finisher ceph cwd unknown /proc/54877/task/56190/cwd (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56190 finisher ceph rtd unknown /proc/54877/task/56190/root (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56190 finisher ceph txt unknown /proc/54877/task/56190/exe (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56190 finisher ceph NOFD /proc/54877/task/56190/fd (opendir: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56191 safe_time ceph cwd unknown /proc/54877/task/56191/cwd (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56191 safe_time ceph rtd unknown /proc/54877/task/56191/root (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56191 safe_time ceph txt unknown /proc/54877/task/56191/exe (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56191 safe_time ceph NOFD /proc/54877/task/56191/fd (opendir: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56192 safe_time ceph cwd unknown /proc/54877/task/56192/cwd (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56192 safe_time ceph rtd unknown /proc/54877/task/56192/root (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56192 safe_time ceph txt unknown /proc/54877/task/56192/exe (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56192 safe_time ceph NOFD /proc/54877/task/56192/fd (opendir: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56193 ceph-osd ceph cwd unknown /proc/54877/task/56193/cwd (readlink: Permission denied) 2024-04-25T00:24:20.929 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56193 ceph-osd ceph rtd unknown /proc/54877/task/56193/root (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56193 ceph-osd ceph txt unknown /proc/54877/task/56193/exe (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56193 ceph-osd ceph NOFD /proc/54877/task/56193/fd (opendir: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56194 osd_srv_a ceph cwd unknown /proc/54877/task/56194/cwd (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56194 osd_srv_a ceph rtd unknown /proc/54877/task/56194/root (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56194 osd_srv_a ceph txt unknown /proc/54877/task/56194/exe (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56194 osd_srv_a ceph NOFD /proc/54877/task/56194/fd (opendir: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56195 tp_osd_tp ceph cwd unknown /proc/54877/task/56195/cwd (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56195 tp_osd_tp ceph rtd unknown /proc/54877/task/56195/root (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56195 tp_osd_tp ceph txt unknown /proc/54877/task/56195/exe (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56195 tp_osd_tp ceph NOFD /proc/54877/task/56195/fd (opendir: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56196 tp_osd_tp ceph cwd unknown /proc/54877/task/56196/cwd (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56196 tp_osd_tp ceph rtd unknown /proc/54877/task/56196/root (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56196 tp_osd_tp ceph txt unknown /proc/54877/task/56196/exe (readlink: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56196 tp_osd_tp ceph NOFD /proc/54877/task/56196/fd (opendir: Permission denied) 2024-04-25T00:24:20.930 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56197 tp_osd_tp ceph cwd unknown /proc/54877/task/56197/cwd (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56197 tp_osd_tp ceph rtd unknown /proc/54877/task/56197/root (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56197 tp_osd_tp ceph txt unknown /proc/54877/task/56197/exe (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56197 tp_osd_tp ceph NOFD /proc/54877/task/56197/fd (opendir: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56198 tp_osd_tp ceph cwd unknown /proc/54877/task/56198/cwd (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56198 tp_osd_tp ceph rtd unknown /proc/54877/task/56198/root (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56198 tp_osd_tp ceph txt unknown /proc/54877/task/56198/exe (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56198 tp_osd_tp ceph NOFD /proc/54877/task/56198/fd (opendir: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56199 tp_osd_tp ceph cwd unknown /proc/54877/task/56199/cwd (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56199 tp_osd_tp ceph rtd unknown /proc/54877/task/56199/root (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56199 tp_osd_tp ceph txt unknown /proc/54877/task/56199/exe (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56199 tp_osd_tp ceph NOFD /proc/54877/task/56199/fd (opendir: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56200 tp_osd_tp ceph cwd unknown /proc/54877/task/56200/cwd (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56200 tp_osd_tp ceph rtd unknown /proc/54877/task/56200/root (readlink: Permission denied) 2024-04-25T00:24:20.931 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56200 tp_osd_tp ceph txt unknown /proc/54877/task/56200/exe (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56200 tp_osd_tp ceph NOFD /proc/54877/task/56200/fd (opendir: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56201 tp_osd_tp ceph cwd unknown /proc/54877/task/56201/cwd (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56201 tp_osd_tp ceph rtd unknown /proc/54877/task/56201/root (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56201 tp_osd_tp ceph txt unknown /proc/54877/task/56201/exe (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56201 tp_osd_tp ceph NOFD /proc/54877/task/56201/fd (opendir: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56202 tp_osd_tp ceph cwd unknown /proc/54877/task/56202/cwd (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56202 tp_osd_tp ceph rtd unknown /proc/54877/task/56202/root (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56202 tp_osd_tp ceph txt unknown /proc/54877/task/56202/exe (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56202 tp_osd_tp ceph NOFD /proc/54877/task/56202/fd (opendir: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56203 tp_osd_tp ceph cwd unknown /proc/54877/task/56203/cwd (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56203 tp_osd_tp ceph rtd unknown /proc/54877/task/56203/root (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56203 tp_osd_tp ceph txt unknown /proc/54877/task/56203/exe (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56203 tp_osd_tp ceph NOFD /proc/54877/task/56203/fd (opendir: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56204 tp_osd_tp ceph cwd unknown /proc/54877/task/56204/cwd (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56204 tp_osd_tp ceph rtd unknown /proc/54877/task/56204/root (readlink: Permission denied) 2024-04-25T00:24:20.932 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56204 tp_osd_tp ceph txt unknown /proc/54877/task/56204/exe (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56204 tp_osd_tp ceph NOFD /proc/54877/task/56204/fd (opendir: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56205 tp_osd_tp ceph cwd unknown /proc/54877/task/56205/cwd (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56205 tp_osd_tp ceph rtd unknown /proc/54877/task/56205/root (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56205 tp_osd_tp ceph txt unknown /proc/54877/task/56205/exe (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56205 tp_osd_tp ceph NOFD /proc/54877/task/56205/fd (opendir: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56206 tp_osd_tp ceph cwd unknown /proc/54877/task/56206/cwd (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56206 tp_osd_tp ceph rtd unknown /proc/54877/task/56206/root (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56206 tp_osd_tp ceph txt unknown /proc/54877/task/56206/exe (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56206 tp_osd_tp ceph NOFD /proc/54877/task/56206/fd (opendir: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56207 tp_osd_tp ceph cwd unknown /proc/54877/task/56207/cwd (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56207 tp_osd_tp ceph rtd unknown /proc/54877/task/56207/root (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56207 tp_osd_tp ceph txt unknown /proc/54877/task/56207/exe (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56207 tp_osd_tp ceph NOFD /proc/54877/task/56207/fd (opendir: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56208 tp_osd_tp ceph cwd unknown /proc/54877/task/56208/cwd (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56208 tp_osd_tp ceph rtd unknown /proc/54877/task/56208/root (readlink: Permission denied) 2024-04-25T00:24:20.933 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56208 tp_osd_tp ceph txt unknown /proc/54877/task/56208/exe (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56208 tp_osd_tp ceph NOFD /proc/54877/task/56208/fd (opendir: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56209 tp_osd_tp ceph cwd unknown /proc/54877/task/56209/cwd (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56209 tp_osd_tp ceph rtd unknown /proc/54877/task/56209/root (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56209 tp_osd_tp ceph txt unknown /proc/54877/task/56209/exe (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56209 tp_osd_tp ceph NOFD /proc/54877/task/56209/fd (opendir: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56210 tp_osd_tp ceph cwd unknown /proc/54877/task/56210/cwd (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56210 tp_osd_tp ceph rtd unknown /proc/54877/task/56210/root (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56210 tp_osd_tp ceph txt unknown /proc/54877/task/56210/exe (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56210 tp_osd_tp ceph NOFD /proc/54877/task/56210/fd (opendir: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56211 osd_srv_h ceph cwd unknown /proc/54877/task/56211/cwd (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56211 osd_srv_h ceph rtd unknown /proc/54877/task/56211/root (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56211 osd_srv_h ceph txt unknown /proc/54877/task/56211/exe (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 54877 56211 osd_srv_h ceph NOFD /proc/54877/task/56211/fd (opendir: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 56323 root cwd unknown /proc/56323/cwd (readlink: Permission denied) 2024-04-25T00:24:20.934 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 56323 root rtd unknown /proc/56323/root (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 56323 root txt unknown /proc/56323/exe (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 56323 root NOFD /proc/56323/fd (opendir: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 56350 root cwd unknown /proc/56350/cwd (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 56350 root rtd unknown /proc/56350/root (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 56350 root txt unknown /proc/56350/exe (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 56350 root NOFD /proc/56350/fd (opendir: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 61914 root cwd unknown /proc/61914/cwd (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 61914 root rtd unknown /proc/61914/root (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 61914 root txt unknown /proc/61914/exe (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/6 61914 root NOFD /proc/61914/fd (opendir: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 61917 root cwd unknown /proc/61917/cwd (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 61917 root rtd unknown /proc/61917/root (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 61917 root txt unknown /proc/61917/exe (readlink: Permission denied) 2024-04-25T00:24:20.935 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 61917 root NOFD /proc/61917/fd (opendir: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 61919 root cwd unknown /proc/61919/cwd (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 61919 root rtd unknown /proc/61919/root (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 61919 root txt unknown /proc/61919/exe (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 61919 root NOFD /proc/61919/fd (opendir: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 ceph cwd unknown /proc/61921/cwd (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 ceph rtd unknown /proc/61921/root (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 ceph txt unknown /proc/61921/exe (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 ceph NOFD /proc/61921/fd (opendir: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61922 log ceph cwd unknown /proc/61921/task/61922/cwd (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61922 log ceph rtd unknown /proc/61921/task/61922/root (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61922 log ceph txt unknown /proc/61921/task/61922/exe (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61922 log ceph NOFD /proc/61921/task/61922/fd (opendir: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61924 msgr-work ceph cwd unknown /proc/61921/task/61924/cwd (readlink: Permission denied) 2024-04-25T00:24:20.936 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61924 msgr-work ceph rtd unknown /proc/61921/task/61924/root (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61924 msgr-work ceph txt unknown /proc/61921/task/61924/exe (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61924 msgr-work ceph NOFD /proc/61921/task/61924/fd (opendir: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61925 msgr-work ceph cwd unknown /proc/61921/task/61925/cwd (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61925 msgr-work ceph rtd unknown /proc/61921/task/61925/root (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61925 msgr-work ceph txt unknown /proc/61921/task/61925/exe (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61925 msgr-work ceph NOFD /proc/61921/task/61925/fd (opendir: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61926 msgr-work ceph cwd unknown /proc/61921/task/61926/cwd (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61926 msgr-work ceph rtd unknown /proc/61921/task/61926/root (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61926 msgr-work ceph txt unknown /proc/61921/task/61926/exe (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61926 msgr-work ceph NOFD /proc/61921/task/61926/fd (opendir: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61930 service ceph cwd unknown /proc/61921/task/61930/cwd (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61930 service ceph rtd unknown /proc/61921/task/61930/root (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61930 service ceph txt unknown /proc/61921/task/61930/exe (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61930 service ceph NOFD /proc/61921/task/61930/fd (opendir: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61931 admin_soc ceph cwd unknown /proc/61921/task/61931/cwd (readlink: Permission denied) 2024-04-25T00:24:20.937 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61931 admin_soc ceph rtd unknown /proc/61921/task/61931/root (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61931 admin_soc ceph txt unknown /proc/61921/task/61931/exe (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 61931 admin_soc ceph NOFD /proc/61921/task/61931/fd (opendir: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62073 signal_ha ceph cwd unknown /proc/61921/task/62073/cwd (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62073 signal_ha ceph rtd unknown /proc/61921/task/62073/root (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62073 signal_ha ceph txt unknown /proc/61921/task/62073/exe (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62073 signal_ha ceph NOFD /proc/61921/task/62073/fd (opendir: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62074 io_contex ceph cwd unknown /proc/61921/task/62074/cwd (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62074 io_contex ceph rtd unknown /proc/61921/task/62074/root (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62074 io_contex ceph txt unknown /proc/61921/task/62074/exe (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62074 io_contex ceph NOFD /proc/61921/task/62074/fd (opendir: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62075 io_contex ceph cwd unknown /proc/61921/task/62075/cwd (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62075 io_contex ceph rtd unknown /proc/61921/task/62075/root (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62075 io_contex ceph txt unknown /proc/61921/task/62075/exe (readlink: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62075 io_contex ceph NOFD /proc/61921/task/62075/fd (opendir: Permission denied) 2024-04-25T00:24:20.938 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62192 OpHistory ceph cwd unknown /proc/61921/task/62192/cwd (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62192 OpHistory ceph rtd unknown /proc/61921/task/62192/root (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62192 OpHistory ceph txt unknown /proc/61921/task/62192/exe (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62192 OpHistory ceph NOFD /proc/61921/task/62192/fd (opendir: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62193 ceph_time ceph cwd unknown /proc/61921/task/62193/cwd (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62193 ceph_time ceph rtd unknown /proc/61921/task/62193/root (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62193 ceph_time ceph txt unknown /proc/61921/task/62193/exe (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62193 ceph_time ceph NOFD /proc/61921/task/62193/fd (opendir: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62214 ceph-osd ceph cwd unknown /proc/61921/task/62214/cwd (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62214 ceph-osd ceph rtd unknown /proc/61921/task/62214/root (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62214 ceph-osd ceph txt unknown /proc/61921/task/62214/exe (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62214 ceph-osd ceph NOFD /proc/61921/task/62214/fd (opendir: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62217 ceph-osd ceph cwd unknown /proc/61921/task/62217/cwd (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62217 ceph-osd ceph rtd unknown /proc/61921/task/62217/root (readlink: Permission denied) 2024-04-25T00:24:20.939 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62217 ceph-osd ceph txt unknown /proc/61921/task/62217/exe (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62217 ceph-osd ceph NOFD /proc/61921/task/62217/fd (opendir: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62224 ceph-osd ceph cwd unknown /proc/61921/task/62224/cwd (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62224 ceph-osd ceph rtd unknown /proc/61921/task/62224/root (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62224 ceph-osd ceph txt unknown /proc/61921/task/62224/exe (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62224 ceph-osd ceph NOFD /proc/61921/task/62224/fd (opendir: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62238 ceph-osd ceph cwd unknown /proc/61921/task/62238/cwd (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62238 ceph-osd ceph rtd unknown /proc/61921/task/62238/root (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62238 ceph-osd ceph txt unknown /proc/61921/task/62238/exe (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62238 ceph-osd ceph NOFD /proc/61921/task/62238/fd (opendir: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62288 ceph-osd ceph cwd unknown /proc/61921/task/62288/cwd (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62288 ceph-osd ceph rtd unknown /proc/61921/task/62288/root (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62288 ceph-osd ceph txt unknown /proc/61921/task/62288/exe (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62288 ceph-osd ceph NOFD /proc/61921/task/62288/fd (opendir: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62306 ceph-osd ceph cwd unknown /proc/61921/task/62306/cwd (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62306 ceph-osd ceph rtd unknown /proc/61921/task/62306/root (readlink: Permission denied) 2024-04-25T00:24:20.940 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62306 ceph-osd ceph txt unknown /proc/61921/task/62306/exe (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62306 ceph-osd ceph NOFD /proc/61921/task/62306/fd (opendir: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62343 ceph-osd ceph cwd unknown /proc/61921/task/62343/cwd (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62343 ceph-osd ceph rtd unknown /proc/61921/task/62343/root (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62343 ceph-osd ceph txt unknown /proc/61921/task/62343/exe (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62343 ceph-osd ceph NOFD /proc/61921/task/62343/fd (opendir: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62372 ceph-osd ceph cwd unknown /proc/61921/task/62372/cwd (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62372 ceph-osd ceph rtd unknown /proc/61921/task/62372/root (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62372 ceph-osd ceph txt unknown /proc/61921/task/62372/exe (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62372 ceph-osd ceph NOFD /proc/61921/task/62372/fd (opendir: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62374 ceph-osd ceph cwd unknown /proc/61921/task/62374/cwd (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62374 ceph-osd ceph rtd unknown /proc/61921/task/62374/root (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62374 ceph-osd ceph txt unknown /proc/61921/task/62374/exe (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62374 ceph-osd ceph NOFD /proc/61921/task/62374/fd (opendir: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62375 safe_time ceph cwd unknown /proc/61921/task/62375/cwd (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62375 safe_time ceph rtd unknown /proc/61921/task/62375/root (readlink: Permission denied) 2024-04-25T00:24:20.941 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62375 safe_time ceph txt unknown /proc/61921/task/62375/exe (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62375 safe_time ceph NOFD /proc/61921/task/62375/fd (opendir: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62376 safe_time ceph cwd unknown /proc/61921/task/62376/cwd (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62376 safe_time ceph rtd unknown /proc/61921/task/62376/root (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62376 safe_time ceph txt unknown /proc/61921/task/62376/exe (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62376 safe_time ceph NOFD /proc/61921/task/62376/fd (opendir: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62377 safe_time ceph cwd unknown /proc/61921/task/62377/cwd (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62377 safe_time ceph rtd unknown /proc/61921/task/62377/root (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62377 safe_time ceph txt unknown /proc/61921/task/62377/exe (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62377 safe_time ceph NOFD /proc/61921/task/62377/fd (opendir: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62378 safe_time ceph cwd unknown /proc/61921/task/62378/cwd (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62378 safe_time ceph rtd unknown /proc/61921/task/62378/root (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62378 safe_time ceph txt unknown /proc/61921/task/62378/exe (readlink: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62378 safe_time ceph NOFD /proc/61921/task/62378/fd (opendir: Permission denied) 2024-04-25T00:24:20.942 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62379 fn_anonym ceph cwd unknown /proc/61921/task/62379/cwd (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62379 fn_anonym ceph rtd unknown /proc/61921/task/62379/root (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62379 fn_anonym ceph txt unknown /proc/61921/task/62379/exe (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62379 fn_anonym ceph NOFD /proc/61921/task/62379/fd (opendir: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62382 rocksdb:l ceph cwd unknown /proc/61921/task/62382/cwd (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62382 rocksdb:l ceph rtd unknown /proc/61921/task/62382/root (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62382 rocksdb:l ceph txt unknown /proc/61921/task/62382/exe (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62382 rocksdb:l ceph NOFD /proc/61921/task/62382/fd (opendir: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62383 rocksdb:l ceph cwd unknown /proc/61921/task/62383/cwd (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62383 rocksdb:l ceph rtd unknown /proc/61921/task/62383/root (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62383 rocksdb:l ceph txt unknown /proc/61921/task/62383/exe (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62383 rocksdb:l ceph NOFD /proc/61921/task/62383/fd (opendir: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62384 rocksdb:l ceph cwd unknown /proc/61921/task/62384/cwd (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62384 rocksdb:l ceph rtd unknown /proc/61921/task/62384/root (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62384 rocksdb:l ceph txt unknown /proc/61921/task/62384/exe (readlink: Permission denied) 2024-04-25T00:24:20.943 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62384 rocksdb:l ceph NOFD /proc/61921/task/62384/fd (opendir: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62385 rocksdb:h ceph cwd unknown /proc/61921/task/62385/cwd (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62385 rocksdb:h ceph rtd unknown /proc/61921/task/62385/root (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62385 rocksdb:h ceph txt unknown /proc/61921/task/62385/exe (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62385 rocksdb:h ceph NOFD /proc/61921/task/62385/fd (opendir: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62862 bstore_ai ceph cwd unknown /proc/61921/task/62862/cwd (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62862 bstore_ai ceph rtd unknown /proc/61921/task/62862/root (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62862 bstore_ai ceph txt unknown /proc/61921/task/62862/exe (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 62862 bstore_ai ceph NOFD /proc/61921/task/62862/fd (opendir: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63048 bstore_ai ceph cwd unknown /proc/61921/task/63048/cwd (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63048 bstore_ai ceph rtd unknown /proc/61921/task/63048/root (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63048 bstore_ai ceph txt unknown /proc/61921/task/63048/exe (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63048 bstore_ai ceph NOFD /proc/61921/task/63048/fd (opendir: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63230 ceph-osd ceph cwd unknown /proc/61921/task/63230/cwd (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63230 ceph-osd ceph rtd unknown /proc/61921/task/63230/root (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63230 ceph-osd ceph txt unknown /proc/61921/task/63230/exe (readlink: Permission denied) 2024-04-25T00:24:20.944 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63230 ceph-osd ceph NOFD /proc/61921/task/63230/fd (opendir: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63231 cfin ceph cwd unknown /proc/61921/task/63231/cwd (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63231 cfin ceph rtd unknown /proc/61921/task/63231/root (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63231 cfin ceph txt unknown /proc/61921/task/63231/exe (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63231 cfin ceph NOFD /proc/61921/task/63231/fd (opendir: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63232 bstore_kv ceph cwd unknown /proc/61921/task/63232/cwd (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63232 bstore_kv ceph rtd unknown /proc/61921/task/63232/root (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63232 bstore_kv ceph txt unknown /proc/61921/task/63232/exe (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63232 bstore_kv ceph NOFD /proc/61921/task/63232/fd (opendir: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63233 bstore_kv ceph cwd unknown /proc/61921/task/63233/cwd (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63233 bstore_kv ceph rtd unknown /proc/61921/task/63233/root (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63233 bstore_kv ceph txt unknown /proc/61921/task/63233/exe (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63233 bstore_kv ceph NOFD /proc/61921/task/63233/fd (opendir: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63234 bstore_me ceph cwd unknown /proc/61921/task/63234/cwd (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63234 bstore_me ceph rtd unknown /proc/61921/task/63234/root (readlink: Permission denied) 2024-04-25T00:24:20.945 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63234 bstore_me ceph txt unknown /proc/61921/task/63234/exe (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63234 bstore_me ceph NOFD /proc/61921/task/63234/fd (opendir: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63235 ms_dispat ceph cwd unknown /proc/61921/task/63235/cwd (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63235 ms_dispat ceph rtd unknown /proc/61921/task/63235/root (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63235 ms_dispat ceph txt unknown /proc/61921/task/63235/exe (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63235 ms_dispat ceph NOFD /proc/61921/task/63235/fd (opendir: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63236 ms_local ceph cwd unknown /proc/61921/task/63236/cwd (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63236 ms_local ceph rtd unknown /proc/61921/task/63236/root (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63236 ms_local ceph txt unknown /proc/61921/task/63236/exe (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63236 ms_local ceph NOFD /proc/61921/task/63236/fd (opendir: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63237 safe_time ceph cwd unknown /proc/61921/task/63237/cwd (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63237 safe_time ceph rtd unknown /proc/61921/task/63237/root (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63237 safe_time ceph txt unknown /proc/61921/task/63237/exe (readlink: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63237 safe_time ceph NOFD /proc/61921/task/63237/fd (opendir: Permission denied) 2024-04-25T00:24:20.946 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63238 safe_time ceph cwd unknown /proc/61921/task/63238/cwd (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63238 safe_time ceph rtd unknown /proc/61921/task/63238/root (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63238 safe_time ceph txt unknown /proc/61921/task/63238/exe (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63238 safe_time ceph NOFD /proc/61921/task/63238/fd (opendir: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63239 ms_dispat ceph cwd unknown /proc/61921/task/63239/cwd (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63239 ms_dispat ceph rtd unknown /proc/61921/task/63239/root (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63239 ms_dispat ceph txt unknown /proc/61921/task/63239/exe (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63239 ms_dispat ceph NOFD /proc/61921/task/63239/fd (opendir: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63240 ms_local ceph cwd unknown /proc/61921/task/63240/cwd (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63240 ms_local ceph rtd unknown /proc/61921/task/63240/root (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63240 ms_local ceph txt unknown /proc/61921/task/63240/exe (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63240 ms_local ceph NOFD /proc/61921/task/63240/fd (opendir: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63241 ms_dispat ceph cwd unknown /proc/61921/task/63241/cwd (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63241 ms_dispat ceph rtd unknown /proc/61921/task/63241/root (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63241 ms_dispat ceph txt unknown /proc/61921/task/63241/exe (readlink: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63241 ms_dispat ceph NOFD /proc/61921/task/63241/fd (opendir: Permission denied) 2024-04-25T00:24:20.947 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63242 ms_local ceph cwd unknown /proc/61921/task/63242/cwd (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63242 ms_local ceph rtd unknown /proc/61921/task/63242/root (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63242 ms_local ceph txt unknown /proc/61921/task/63242/exe (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63242 ms_local ceph NOFD /proc/61921/task/63242/fd (opendir: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63243 ms_dispat ceph cwd unknown /proc/61921/task/63243/cwd (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63243 ms_dispat ceph rtd unknown /proc/61921/task/63243/root (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63243 ms_dispat ceph txt unknown /proc/61921/task/63243/exe (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63243 ms_dispat ceph NOFD /proc/61921/task/63243/fd (opendir: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63244 ms_local ceph cwd unknown /proc/61921/task/63244/cwd (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63244 ms_local ceph rtd unknown /proc/61921/task/63244/root (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63244 ms_local ceph txt unknown /proc/61921/task/63244/exe (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63244 ms_local ceph NOFD /proc/61921/task/63244/fd (opendir: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63245 ms_dispat ceph cwd unknown /proc/61921/task/63245/cwd (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63245 ms_dispat ceph rtd unknown /proc/61921/task/63245/root (readlink: Permission denied) 2024-04-25T00:24:20.948 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63245 ms_dispat ceph txt unknown /proc/61921/task/63245/exe (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63245 ms_dispat ceph NOFD /proc/61921/task/63245/fd (opendir: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63246 ms_local ceph cwd unknown /proc/61921/task/63246/cwd (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63246 ms_local ceph rtd unknown /proc/61921/task/63246/root (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63246 ms_local ceph txt unknown /proc/61921/task/63246/exe (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63246 ms_local ceph NOFD /proc/61921/task/63246/fd (opendir: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63247 ms_dispat ceph cwd unknown /proc/61921/task/63247/cwd (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63247 ms_dispat ceph rtd unknown /proc/61921/task/63247/root (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63247 ms_dispat ceph txt unknown /proc/61921/task/63247/exe (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63247 ms_dispat ceph NOFD /proc/61921/task/63247/fd (opendir: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63248 ms_local ceph cwd unknown /proc/61921/task/63248/cwd (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63248 ms_local ceph rtd unknown /proc/61921/task/63248/root (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63248 ms_local ceph txt unknown /proc/61921/task/63248/exe (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63248 ms_local ceph NOFD /proc/61921/task/63248/fd (opendir: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63249 ms_dispat ceph cwd unknown /proc/61921/task/63249/cwd (readlink: Permission denied) 2024-04-25T00:24:20.949 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63249 ms_dispat ceph rtd unknown /proc/61921/task/63249/root (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63249 ms_dispat ceph txt unknown /proc/61921/task/63249/exe (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63249 ms_dispat ceph NOFD /proc/61921/task/63249/fd (opendir: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63250 ms_local ceph cwd unknown /proc/61921/task/63250/cwd (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63250 ms_local ceph rtd unknown /proc/61921/task/63250/root (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63250 ms_local ceph txt unknown /proc/61921/task/63250/exe (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63250 ms_local ceph NOFD /proc/61921/task/63250/fd (opendir: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63251 fn_anonym ceph cwd unknown /proc/61921/task/63251/cwd (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63251 fn_anonym ceph rtd unknown /proc/61921/task/63251/root (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63251 fn_anonym ceph txt unknown /proc/61921/task/63251/exe (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63251 fn_anonym ceph NOFD /proc/61921/task/63251/fd (opendir: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63252 finisher ceph cwd unknown /proc/61921/task/63252/cwd (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63252 finisher ceph rtd unknown /proc/61921/task/63252/root (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63252 finisher ceph txt unknown /proc/61921/task/63252/exe (readlink: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63252 finisher ceph NOFD /proc/61921/task/63252/fd (opendir: Permission denied) 2024-04-25T00:24:20.950 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63253 safe_time ceph cwd unknown /proc/61921/task/63253/cwd (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63253 safe_time ceph rtd unknown /proc/61921/task/63253/root (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63253 safe_time ceph txt unknown /proc/61921/task/63253/exe (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63253 safe_time ceph NOFD /proc/61921/task/63253/fd (opendir: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63254 safe_time ceph cwd unknown /proc/61921/task/63254/cwd (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63254 safe_time ceph rtd unknown /proc/61921/task/63254/root (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63254 safe_time ceph txt unknown /proc/61921/task/63254/exe (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63254 safe_time ceph NOFD /proc/61921/task/63254/fd (opendir: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63255 ceph-osd ceph cwd unknown /proc/61921/task/63255/cwd (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63255 ceph-osd ceph rtd unknown /proc/61921/task/63255/root (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63255 ceph-osd ceph txt unknown /proc/61921/task/63255/exe (readlink: Permission denied) 2024-04-25T00:24:20.951 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63255 ceph-osd ceph NOFD /proc/61921/task/63255/fd (opendir: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63256 osd_srv_a ceph cwd unknown /proc/61921/task/63256/cwd (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63256 osd_srv_a ceph rtd unknown /proc/61921/task/63256/root (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63256 osd_srv_a ceph txt unknown /proc/61921/task/63256/exe (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63256 osd_srv_a ceph NOFD /proc/61921/task/63256/fd (opendir: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63257 tp_osd_tp ceph cwd unknown /proc/61921/task/63257/cwd (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63257 tp_osd_tp ceph rtd unknown /proc/61921/task/63257/root (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63257 tp_osd_tp ceph txt unknown /proc/61921/task/63257/exe (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63257 tp_osd_tp ceph NOFD /proc/61921/task/63257/fd (opendir: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63258 tp_osd_tp ceph cwd unknown /proc/61921/task/63258/cwd (readlink: Permission denied) 2024-04-25T00:24:20.952 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63258 tp_osd_tp ceph rtd unknown /proc/61921/task/63258/root (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63258 tp_osd_tp ceph txt unknown /proc/61921/task/63258/exe (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63258 tp_osd_tp ceph NOFD /proc/61921/task/63258/fd (opendir: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63259 tp_osd_tp ceph cwd unknown /proc/61921/task/63259/cwd (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63259 tp_osd_tp ceph rtd unknown /proc/61921/task/63259/root (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63259 tp_osd_tp ceph txt unknown /proc/61921/task/63259/exe (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63259 tp_osd_tp ceph NOFD /proc/61921/task/63259/fd (opendir: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63260 tp_osd_tp ceph cwd unknown /proc/61921/task/63260/cwd (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63260 tp_osd_tp ceph rtd unknown /proc/61921/task/63260/root (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63260 tp_osd_tp ceph txt unknown /proc/61921/task/63260/exe (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63260 tp_osd_tp ceph NOFD /proc/61921/task/63260/fd (opendir: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63261 tp_osd_tp ceph cwd unknown /proc/61921/task/63261/cwd (readlink: Permission denied) 2024-04-25T00:24:20.953 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63261 tp_osd_tp ceph rtd unknown /proc/61921/task/63261/root (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63261 tp_osd_tp ceph txt unknown /proc/61921/task/63261/exe (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63261 tp_osd_tp ceph NOFD /proc/61921/task/63261/fd (opendir: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63262 tp_osd_tp ceph cwd unknown /proc/61921/task/63262/cwd (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63262 tp_osd_tp ceph rtd unknown /proc/61921/task/63262/root (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63262 tp_osd_tp ceph txt unknown /proc/61921/task/63262/exe (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63262 tp_osd_tp ceph NOFD /proc/61921/task/63262/fd (opendir: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63263 tp_osd_tp ceph cwd unknown /proc/61921/task/63263/cwd (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63263 tp_osd_tp ceph rtd unknown /proc/61921/task/63263/root (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63263 tp_osd_tp ceph txt unknown /proc/61921/task/63263/exe (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63263 tp_osd_tp ceph NOFD /proc/61921/task/63263/fd (opendir: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63264 tp_osd_tp ceph cwd unknown /proc/61921/task/63264/cwd (readlink: Permission denied) 2024-04-25T00:24:20.954 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63264 tp_osd_tp ceph rtd unknown /proc/61921/task/63264/root (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63264 tp_osd_tp ceph txt unknown /proc/61921/task/63264/exe (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63264 tp_osd_tp ceph NOFD /proc/61921/task/63264/fd (opendir: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63265 tp_osd_tp ceph cwd unknown /proc/61921/task/63265/cwd (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63265 tp_osd_tp ceph rtd unknown /proc/61921/task/63265/root (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63265 tp_osd_tp ceph txt unknown /proc/61921/task/63265/exe (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63265 tp_osd_tp ceph NOFD /proc/61921/task/63265/fd (opendir: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63266 tp_osd_tp ceph cwd unknown /proc/61921/task/63266/cwd (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63266 tp_osd_tp ceph rtd unknown /proc/61921/task/63266/root (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63266 tp_osd_tp ceph txt unknown /proc/61921/task/63266/exe (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63266 tp_osd_tp ceph NOFD /proc/61921/task/63266/fd (opendir: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63267 tp_osd_tp ceph cwd unknown /proc/61921/task/63267/cwd (readlink: Permission denied) 2024-04-25T00:24:20.955 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63267 tp_osd_tp ceph rtd unknown /proc/61921/task/63267/root (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63267 tp_osd_tp ceph txt unknown /proc/61921/task/63267/exe (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63267 tp_osd_tp ceph NOFD /proc/61921/task/63267/fd (opendir: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63268 tp_osd_tp ceph cwd unknown /proc/61921/task/63268/cwd (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63268 tp_osd_tp ceph rtd unknown /proc/61921/task/63268/root (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63268 tp_osd_tp ceph txt unknown /proc/61921/task/63268/exe (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63268 tp_osd_tp ceph NOFD /proc/61921/task/63268/fd (opendir: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63269 tp_osd_tp ceph cwd unknown /proc/61921/task/63269/cwd (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63269 tp_osd_tp ceph rtd unknown /proc/61921/task/63269/root (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63269 tp_osd_tp ceph txt unknown /proc/61921/task/63269/exe (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63269 tp_osd_tp ceph NOFD /proc/61921/task/63269/fd (opendir: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63270 tp_osd_tp ceph cwd unknown /proc/61921/task/63270/cwd (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63270 tp_osd_tp ceph rtd unknown /proc/61921/task/63270/root (readlink: Permission denied) 2024-04-25T00:24:20.956 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63270 tp_osd_tp ceph txt unknown /proc/61921/task/63270/exe (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63270 tp_osd_tp ceph NOFD /proc/61921/task/63270/fd (opendir: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63271 tp_osd_tp ceph cwd unknown /proc/61921/task/63271/cwd (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63271 tp_osd_tp ceph rtd unknown /proc/61921/task/63271/root (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63271 tp_osd_tp ceph txt unknown /proc/61921/task/63271/exe (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63271 tp_osd_tp ceph NOFD /proc/61921/task/63271/fd (opendir: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63272 tp_osd_tp ceph cwd unknown /proc/61921/task/63272/cwd (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63272 tp_osd_tp ceph rtd unknown /proc/61921/task/63272/root (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63272 tp_osd_tp ceph txt unknown /proc/61921/task/63272/exe (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63272 tp_osd_tp ceph NOFD /proc/61921/task/63272/fd (opendir: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63273 osd_srv_h ceph cwd unknown /proc/61921/task/63273/cwd (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63273 osd_srv_h ceph rtd unknown /proc/61921/task/63273/root (readlink: Permission denied) 2024-04-25T00:24:20.957 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63273 osd_srv_h ceph txt unknown /proc/61921/task/63273/exe (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-osd 61921 63273 osd_srv_h ceph NOFD /proc/61921/task/63273/fd (opendir: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 63376 root cwd unknown /proc/63376/cwd (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 63376 root rtd unknown /proc/63376/root (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 63376 root txt unknown /proc/63376/exe (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 63376 root NOFD /proc/63376/fd (opendir: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 63400 root cwd unknown /proc/63400/cwd (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 63400 root rtd unknown /proc/63400/root (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 63400 root txt unknown /proc/63400/exe (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 63400 root NOFD /proc/63400/fd (opendir: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 66417 root cwd unknown /proc/66417/cwd (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 66417 root rtd unknown /proc/66417/root (readlink: Permission denied) 2024-04-25T00:24:20.958 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 66417 root txt unknown /proc/66417/exe (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/0 66417 root NOFD /proc/66417/fd (opendir: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67006 root cwd unknown /proc/67006/cwd (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67006 root rtd unknown /proc/67006/root (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67006 root txt unknown /proc/67006/exe (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67006 root NOFD /proc/67006/fd (opendir: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67023 root cwd unknown /proc/67023/cwd (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67023 root rtd unknown /proc/67023/root (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67023 root txt unknown /proc/67023/exe (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67023 root NOFD /proc/67023/fd (opendir: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67024 root cwd unknown /proc/67024/cwd (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67024 root rtd unknown /proc/67024/root (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67024 root txt unknown /proc/67024/exe (readlink: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67024 root NOFD /proc/67024/fd (opendir: Permission denied) 2024-04-25T00:24:20.959 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67041 root cwd unknown /proc/67041/cwd (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67041 root rtd unknown /proc/67041/root (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67041 root txt unknown /proc/67041/exe (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67041 root NOFD /proc/67041/fd (opendir: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67042 root cwd unknown /proc/67042/cwd (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67042 root rtd unknown /proc/67042/root (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67042 root txt unknown /proc/67042/exe (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67042 root NOFD /proc/67042/fd (opendir: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67059 root cwd unknown /proc/67059/cwd (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67059 root rtd unknown /proc/67059/root (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67059 root txt unknown /proc/67059/exe (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67059 root NOFD /proc/67059/fd (opendir: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67060 root cwd unknown /proc/67060/cwd (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67060 root rtd unknown /proc/67060/root (readlink: Permission denied) 2024-04-25T00:24:20.960 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67060 root txt unknown /proc/67060/exe (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:sudo 67060 root NOFD /proc/67060/fd (opendir: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67077 root cwd unknown /proc/67077/cwd (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67077 root rtd unknown /proc/67077/root (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67077 root txt unknown /proc/67077/exe (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:journalct 67077 root NOFD /proc/67077/fd (opendir: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67616 root cwd unknown /proc/67616/cwd (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67616 root rtd unknown /proc/67616/root (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67616 root txt unknown /proc/67616/exe (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67616 root NOFD /proc/67616/fd (opendir: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67618 root cwd unknown /proc/67618/cwd (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67618 root rtd unknown /proc/67618/root (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67618 root txt unknown /proc/67618/exe (readlink: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67618 root NOFD /proc/67618/fd (opendir: Permission denied) 2024-04-25T00:24:20.961 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 ceph cwd unknown /proc/67622/cwd (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 ceph rtd unknown /proc/67622/root (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 ceph txt unknown /proc/67622/exe (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 ceph NOFD /proc/67622/fd (opendir: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67623 log ceph cwd unknown /proc/67622/task/67623/cwd (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67623 log ceph rtd unknown /proc/67622/task/67623/root (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67623 log ceph txt unknown /proc/67622/task/67623/exe (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67623 log ceph NOFD /proc/67622/task/67623/fd (opendir: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67625 msgr-work ceph cwd unknown /proc/67622/task/67625/cwd (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67625 msgr-work ceph rtd unknown /proc/67622/task/67625/root (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67625 msgr-work ceph txt unknown /proc/67622/task/67625/exe (readlink: Permission denied) 2024-04-25T00:24:20.962 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67625 msgr-work ceph NOFD /proc/67622/task/67625/fd (opendir: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67626 msgr-work ceph cwd unknown /proc/67622/task/67626/cwd (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67626 msgr-work ceph rtd unknown /proc/67622/task/67626/root (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67626 msgr-work ceph txt unknown /proc/67622/task/67626/exe (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67626 msgr-work ceph NOFD /proc/67622/task/67626/fd (opendir: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67627 msgr-work ceph cwd unknown /proc/67622/task/67627/cwd (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67627 msgr-work ceph rtd unknown /proc/67622/task/67627/root (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67627 msgr-work ceph txt unknown /proc/67622/task/67627/exe (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67627 msgr-work ceph NOFD /proc/67622/task/67627/fd (opendir: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67631 service ceph cwd unknown /proc/67622/task/67631/cwd (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67631 service ceph rtd unknown /proc/67622/task/67631/root (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67631 service ceph txt unknown /proc/67622/task/67631/exe (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67631 service ceph NOFD /proc/67622/task/67631/fd (opendir: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67632 admin_soc ceph cwd unknown /proc/67622/task/67632/cwd (readlink: Permission denied) 2024-04-25T00:24:20.963 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67632 admin_soc ceph rtd unknown /proc/67622/task/67632/root (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67632 admin_soc ceph txt unknown /proc/67622/task/67632/exe (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67632 admin_soc ceph NOFD /proc/67622/task/67632/fd (opendir: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67633 signal_ha ceph cwd unknown /proc/67622/task/67633/cwd (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67633 signal_ha ceph rtd unknown /proc/67622/task/67633/root (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67633 signal_ha ceph txt unknown /proc/67622/task/67633/exe (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67633 signal_ha ceph NOFD /proc/67622/task/67633/fd (opendir: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67634 io_contex ceph cwd unknown /proc/67622/task/67634/cwd (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67634 io_contex ceph rtd unknown /proc/67622/task/67634/root (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67634 io_contex ceph txt unknown /proc/67622/task/67634/exe (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67634 io_contex ceph NOFD /proc/67622/task/67634/fd (opendir: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67635 io_contex ceph cwd unknown /proc/67622/task/67635/cwd (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67635 io_contex ceph rtd unknown /proc/67622/task/67635/root (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67635 io_contex ceph txt unknown /proc/67622/task/67635/exe (readlink: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67635 io_contex ceph NOFD /proc/67622/task/67635/fd (opendir: Permission denied) 2024-04-25T00:24:20.964 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67636 ms_dispat ceph cwd unknown /proc/67622/task/67636/cwd (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67636 ms_dispat ceph rtd unknown /proc/67622/task/67636/root (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67636 ms_dispat ceph txt unknown /proc/67622/task/67636/exe (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67636 ms_dispat ceph NOFD /proc/67622/task/67636/fd (opendir: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67637 ms_local ceph cwd unknown /proc/67622/task/67637/cwd (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67637 ms_local ceph rtd unknown /proc/67622/task/67637/root (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67637 ms_local ceph txt unknown /proc/67622/task/67637/exe (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67637 ms_local ceph NOFD /proc/67622/task/67637/fd (opendir: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67638 safe_time ceph cwd unknown /proc/67622/task/67638/cwd (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67638 safe_time ceph rtd unknown /proc/67622/task/67638/root (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67638 safe_time ceph txt unknown /proc/67622/task/67638/exe (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67638 safe_time ceph NOFD /proc/67622/task/67638/fd (opendir: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67639 safe_time ceph cwd unknown /proc/67622/task/67639/cwd (readlink: Permission denied) 2024-04-25T00:24:20.965 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67639 safe_time ceph rtd unknown /proc/67622/task/67639/root (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67639 safe_time ceph txt unknown /proc/67622/task/67639/exe (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67639 safe_time ceph NOFD /proc/67622/task/67639/fd (opendir: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67640 ceph-mds ceph cwd unknown /proc/67622/task/67640/cwd (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67640 ceph-mds ceph rtd unknown /proc/67622/task/67640/root (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67640 ceph-mds ceph txt unknown /proc/67622/task/67640/exe (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67640 ceph-mds ceph NOFD /proc/67622/task/67640/fd (opendir: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67711 safe_time ceph cwd unknown /proc/67622/task/67711/cwd (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67711 safe_time ceph rtd unknown /proc/67622/task/67711/root (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67711 safe_time ceph txt unknown /proc/67622/task/67711/exe (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67622 67711 safe_time ceph NOFD /proc/67622/task/67711/fd (opendir: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67966 root cwd unknown /proc/67966/cwd (readlink: Permission denied) 2024-04-25T00:24:20.966 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67966 root rtd unknown /proc/67966/root (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67966 root txt unknown /proc/67966/exe (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 67966 root NOFD /proc/67966/fd (opendir: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67968 root cwd unknown /proc/67968/cwd (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67968 root rtd unknown /proc/67968/root (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67968 root txt unknown /proc/67968/exe (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 67968 root NOFD /proc/67968/fd (opendir: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 ceph cwd unknown /proc/67970/cwd (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 ceph rtd unknown /proc/67970/root (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 ceph txt unknown /proc/67970/exe (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 ceph NOFD /proc/67970/fd (opendir: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67971 log ceph cwd unknown /proc/67970/task/67971/cwd (readlink: Permission denied) 2024-04-25T00:24:20.967 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67971 log ceph rtd unknown /proc/67970/task/67971/root (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67971 log ceph txt unknown /proc/67970/task/67971/exe (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67971 log ceph NOFD /proc/67970/task/67971/fd (opendir: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67973 msgr-work ceph cwd unknown /proc/67970/task/67973/cwd (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67973 msgr-work ceph rtd unknown /proc/67970/task/67973/root (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67973 msgr-work ceph txt unknown /proc/67970/task/67973/exe (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67973 msgr-work ceph NOFD /proc/67970/task/67973/fd (opendir: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67974 msgr-work ceph cwd unknown /proc/67970/task/67974/cwd (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67974 msgr-work ceph rtd unknown /proc/67970/task/67974/root (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67974 msgr-work ceph txt unknown /proc/67970/task/67974/exe (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67974 msgr-work ceph NOFD /proc/67970/task/67974/fd (opendir: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67975 msgr-work ceph cwd unknown /proc/67970/task/67975/cwd (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67975 msgr-work ceph rtd unknown /proc/67970/task/67975/root (readlink: Permission denied) 2024-04-25T00:24:20.968 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67975 msgr-work ceph txt unknown /proc/67970/task/67975/exe (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67975 msgr-work ceph NOFD /proc/67970/task/67975/fd (opendir: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67979 service ceph cwd unknown /proc/67970/task/67979/cwd (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67979 service ceph rtd unknown /proc/67970/task/67979/root (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67979 service ceph txt unknown /proc/67970/task/67979/exe (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67979 service ceph NOFD /proc/67970/task/67979/fd (opendir: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67980 admin_soc ceph cwd unknown /proc/67970/task/67980/cwd (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67980 admin_soc ceph rtd unknown /proc/67970/task/67980/root (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67980 admin_soc ceph txt unknown /proc/67970/task/67980/exe (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67980 admin_soc ceph NOFD /proc/67970/task/67980/fd (opendir: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67981 signal_ha ceph cwd unknown /proc/67970/task/67981/cwd (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67981 signal_ha ceph rtd unknown /proc/67970/task/67981/root (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67981 signal_ha ceph txt unknown /proc/67970/task/67981/exe (readlink: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67981 signal_ha ceph NOFD /proc/67970/task/67981/fd (opendir: Permission denied) 2024-04-25T00:24:20.969 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67982 io_contex ceph cwd unknown /proc/67970/task/67982/cwd (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67982 io_contex ceph rtd unknown /proc/67970/task/67982/root (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67982 io_contex ceph txt unknown /proc/67970/task/67982/exe (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67982 io_contex ceph NOFD /proc/67970/task/67982/fd (opendir: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67983 io_contex ceph cwd unknown /proc/67970/task/67983/cwd (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67983 io_contex ceph rtd unknown /proc/67970/task/67983/root (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67983 io_contex ceph txt unknown /proc/67970/task/67983/exe (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67983 io_contex ceph NOFD /proc/67970/task/67983/fd (opendir: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67984 ms_dispat ceph cwd unknown /proc/67970/task/67984/cwd (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67984 ms_dispat ceph rtd unknown /proc/67970/task/67984/root (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67984 ms_dispat ceph txt unknown /proc/67970/task/67984/exe (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67984 ms_dispat ceph NOFD /proc/67970/task/67984/fd (opendir: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67985 ms_local ceph cwd unknown /proc/67970/task/67985/cwd (readlink: Permission denied) 2024-04-25T00:24:20.970 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67985 ms_local ceph rtd unknown /proc/67970/task/67985/root (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67985 ms_local ceph txt unknown /proc/67970/task/67985/exe (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67985 ms_local ceph NOFD /proc/67970/task/67985/fd (opendir: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67986 safe_time ceph cwd unknown /proc/67970/task/67986/cwd (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67986 safe_time ceph rtd unknown /proc/67970/task/67986/root (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67986 safe_time ceph txt unknown /proc/67970/task/67986/exe (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67986 safe_time ceph NOFD /proc/67970/task/67986/fd (opendir: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67987 safe_time ceph cwd unknown /proc/67970/task/67987/cwd (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67987 safe_time ceph rtd unknown /proc/67970/task/67987/root (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67987 safe_time ceph txt unknown /proc/67970/task/67987/exe (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67987 safe_time ceph NOFD /proc/67970/task/67987/fd (opendir: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67988 ceph-mds ceph cwd unknown /proc/67970/task/67988/cwd (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67988 ceph-mds ceph rtd unknown /proc/67970/task/67988/root (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67988 ceph-mds ceph txt unknown /proc/67970/task/67988/exe (readlink: Permission denied) 2024-04-25T00:24:20.971 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67988 ceph-mds ceph NOFD /proc/67970/task/67988/fd (opendir: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67993 safe_time ceph cwd unknown /proc/67970/task/67993/cwd (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67993 safe_time ceph rtd unknown /proc/67970/task/67993/root (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67993 safe_time ceph txt unknown /proc/67970/task/67993/exe (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 67970 67993 safe_time ceph NOFD /proc/67970/task/67993/fd (opendir: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 68076 root cwd unknown /proc/68076/cwd (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 68076 root rtd unknown /proc/68076/root (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 68076 root txt unknown /proc/68076/exe (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 68076 root NOFD /proc/68076/fd (opendir: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68564 root cwd unknown /proc/68564/cwd (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68564 root rtd unknown /proc/68564/root (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68564 root txt unknown /proc/68564/exe (readlink: Permission denied) 2024-04-25T00:24:20.972 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68564 root NOFD /proc/68564/fd (opendir: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68566 root cwd unknown /proc/68566/cwd (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68566 root rtd unknown /proc/68566/root (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68566 root txt unknown /proc/68566/exe (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68566 root NOFD /proc/68566/fd (opendir: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 ceph cwd unknown /proc/68568/cwd (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 ceph rtd unknown /proc/68568/root (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 ceph txt unknown /proc/68568/exe (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 ceph NOFD /proc/68568/fd (opendir: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68569 log ceph cwd unknown /proc/68568/task/68569/cwd (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68569 log ceph rtd unknown /proc/68568/task/68569/root (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68569 log ceph txt unknown /proc/68568/task/68569/exe (readlink: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68569 log ceph NOFD /proc/68568/task/68569/fd (opendir: Permission denied) 2024-04-25T00:24:20.973 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68571 msgr-work ceph cwd unknown /proc/68568/task/68571/cwd (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68571 msgr-work ceph rtd unknown /proc/68568/task/68571/root (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68571 msgr-work ceph txt unknown /proc/68568/task/68571/exe (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68571 msgr-work ceph NOFD /proc/68568/task/68571/fd (opendir: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68572 msgr-work ceph cwd unknown /proc/68568/task/68572/cwd (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68572 msgr-work ceph rtd unknown /proc/68568/task/68572/root (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68572 msgr-work ceph txt unknown /proc/68568/task/68572/exe (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68572 msgr-work ceph NOFD /proc/68568/task/68572/fd (opendir: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68573 msgr-work ceph cwd unknown /proc/68568/task/68573/cwd (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68573 msgr-work ceph rtd unknown /proc/68568/task/68573/root (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68573 msgr-work ceph txt unknown /proc/68568/task/68573/exe (readlink: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68573 msgr-work ceph NOFD /proc/68568/task/68573/fd (opendir: Permission denied) 2024-04-25T00:24:20.974 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68577 service ceph cwd unknown /proc/68568/task/68577/cwd (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68577 service ceph rtd unknown /proc/68568/task/68577/root (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68577 service ceph txt unknown /proc/68568/task/68577/exe (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68577 service ceph NOFD /proc/68568/task/68577/fd (opendir: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68578 admin_soc ceph cwd unknown /proc/68568/task/68578/cwd (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68578 admin_soc ceph rtd unknown /proc/68568/task/68578/root (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68578 admin_soc ceph txt unknown /proc/68568/task/68578/exe (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68578 admin_soc ceph NOFD /proc/68568/task/68578/fd (opendir: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68579 signal_ha ceph cwd unknown /proc/68568/task/68579/cwd (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68579 signal_ha ceph rtd unknown /proc/68568/task/68579/root (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68579 signal_ha ceph txt unknown /proc/68568/task/68579/exe (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68579 signal_ha ceph NOFD /proc/68568/task/68579/fd (opendir: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68580 io_contex ceph cwd unknown /proc/68568/task/68580/cwd (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68580 io_contex ceph rtd unknown /proc/68568/task/68580/root (readlink: Permission denied) 2024-04-25T00:24:20.975 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68580 io_contex ceph txt unknown /proc/68568/task/68580/exe (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68580 io_contex ceph NOFD /proc/68568/task/68580/fd (opendir: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68581 io_contex ceph cwd unknown /proc/68568/task/68581/cwd (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68581 io_contex ceph rtd unknown /proc/68568/task/68581/root (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68581 io_contex ceph txt unknown /proc/68568/task/68581/exe (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68581 io_contex ceph NOFD /proc/68568/task/68581/fd (opendir: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68582 ms_dispat ceph cwd unknown /proc/68568/task/68582/cwd (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68582 ms_dispat ceph rtd unknown /proc/68568/task/68582/root (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68582 ms_dispat ceph txt unknown /proc/68568/task/68582/exe (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68582 ms_dispat ceph NOFD /proc/68568/task/68582/fd (opendir: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68583 ms_local ceph cwd unknown /proc/68568/task/68583/cwd (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68583 ms_local ceph rtd unknown /proc/68568/task/68583/root (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68583 ms_local ceph txt unknown /proc/68568/task/68583/exe (readlink: Permission denied) 2024-04-25T00:24:20.976 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68583 ms_local ceph NOFD /proc/68568/task/68583/fd (opendir: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68584 safe_time ceph cwd unknown /proc/68568/task/68584/cwd (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68584 safe_time ceph rtd unknown /proc/68568/task/68584/root (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68584 safe_time ceph txt unknown /proc/68568/task/68584/exe (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68584 safe_time ceph NOFD /proc/68568/task/68584/fd (opendir: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68585 safe_time ceph cwd unknown /proc/68568/task/68585/cwd (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68585 safe_time ceph rtd unknown /proc/68568/task/68585/root (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68585 safe_time ceph txt unknown /proc/68568/task/68585/exe (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68585 safe_time ceph NOFD /proc/68568/task/68585/fd (opendir: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68586 ceph-mds ceph cwd unknown /proc/68568/task/68586/cwd (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68586 ceph-mds ceph rtd unknown /proc/68568/task/68586/root (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68586 ceph-mds ceph txt unknown /proc/68568/task/68586/exe (readlink: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68586 ceph-mds ceph NOFD /proc/68568/task/68586/fd (opendir: Permission denied) 2024-04-25T00:24:20.977 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68657 safe_time ceph cwd unknown /proc/68568/task/68657/cwd (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68657 safe_time ceph rtd unknown /proc/68568/task/68657/root (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68657 safe_time ceph txt unknown /proc/68568/task/68657/exe (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 68657 safe_time ceph NOFD /proc/68568/task/68657/fd (opendir: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72217 ceph_time ceph cwd unknown /proc/68568/task/72217/cwd (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72217 ceph_time ceph rtd unknown /proc/68568/task/72217/root (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72217 ceph_time ceph txt unknown /proc/68568/task/72217/exe (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72217 ceph_time ceph NOFD /proc/68568/task/72217/fd (opendir: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72219 OpHistory ceph cwd unknown /proc/68568/task/72219/cwd (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72219 OpHistory ceph rtd unknown /proc/68568/task/72219/root (readlink: Permission denied) 2024-04-25T00:24:20.978 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72219 OpHistory ceph txt unknown /proc/68568/task/72219/exe (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72219 OpHistory ceph NOFD /proc/68568/task/72219/fd (opendir: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72221 ms_dispat ceph cwd unknown /proc/68568/task/72221/cwd (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72221 ms_dispat ceph rtd unknown /proc/68568/task/72221/root (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72221 ms_dispat ceph txt unknown /proc/68568/task/72221/exe (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72221 ms_dispat ceph NOFD /proc/68568/task/72221/fd (opendir: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72223 ms_dispat ceph cwd unknown /proc/68568/task/72223/cwd (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72223 ms_dispat ceph rtd unknown /proc/68568/task/72223/root (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72223 ms_dispat ceph txt unknown /proc/68568/task/72223/exe (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72223 ms_dispat ceph NOFD /proc/68568/task/72223/fd (opendir: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72225 mds_rank_ ceph cwd unknown /proc/68568/task/72225/cwd (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72225 mds_rank_ ceph rtd unknown /proc/68568/task/72225/root (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72225 mds_rank_ ceph txt unknown /proc/68568/task/72225/exe (readlink: Permission denied) 2024-04-25T00:24:20.979 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72225 mds_rank_ ceph NOFD /proc/68568/task/72225/fd (opendir: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72227 PQ_Finish ceph cwd unknown /proc/68568/task/72227/cwd (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72227 PQ_Finish ceph rtd unknown /proc/68568/task/72227/root (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72227 PQ_Finish ceph txt unknown /proc/68568/task/72227/exe (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72227 PQ_Finish ceph NOFD /proc/68568/task/72227/fd (opendir: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72229 safe_time ceph cwd unknown /proc/68568/task/72229/cwd (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72229 safe_time ceph rtd unknown /proc/68568/task/72229/root (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72229 safe_time ceph txt unknown /proc/68568/task/72229/exe (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72229 safe_time ceph NOFD /proc/68568/task/72229/fd (opendir: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72231 MR_Finish ceph cwd unknown /proc/68568/task/72231/cwd (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72231 MR_Finish ceph rtd unknown /proc/68568/task/72231/root (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72231 MR_Finish ceph txt unknown /proc/68568/task/72231/exe (readlink: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72231 MR_Finish ceph NOFD /proc/68568/task/72231/fd (opendir: Permission denied) 2024-04-25T00:24:20.980 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72234 md_submit ceph cwd unknown /proc/68568/task/72234/cwd (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72234 md_submit ceph rtd unknown /proc/68568/task/72234/root (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72234 md_submit ceph txt unknown /proc/68568/task/72234/exe (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68568 72234 md_submit ceph NOFD /proc/68568/task/72234/fd (opendir: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68918 root cwd unknown /proc/68918/cwd (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68918 root rtd unknown /proc/68918/root (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68918 root txt unknown /proc/68918/exe (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:conmon 68918 root NOFD /proc/68918/fd (opendir: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68920 root cwd unknown /proc/68920/cwd (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68920 root rtd unknown /proc/68920/root (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68920 root txt unknown /proc/68920/exe (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:podman-in 68920 root NOFD /proc/68920/fd (opendir: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 ceph cwd unknown /proc/68922/cwd (readlink: Permission denied) 2024-04-25T00:24:20.981 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 ceph rtd unknown /proc/68922/root (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 ceph txt unknown /proc/68922/exe (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 ceph NOFD /proc/68922/fd (opendir: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68925 log ceph cwd unknown /proc/68922/task/68925/cwd (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68925 log ceph rtd unknown /proc/68922/task/68925/root (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68925 log ceph txt unknown /proc/68922/task/68925/exe (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68925 log ceph NOFD /proc/68922/task/68925/fd (opendir: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68927 msgr-work ceph cwd unknown /proc/68922/task/68927/cwd (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68927 msgr-work ceph rtd unknown /proc/68922/task/68927/root (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68927 msgr-work ceph txt unknown /proc/68922/task/68927/exe (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68927 msgr-work ceph NOFD /proc/68922/task/68927/fd (opendir: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68928 msgr-work ceph cwd unknown /proc/68922/task/68928/cwd (readlink: Permission denied) 2024-04-25T00:24:20.982 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68928 msgr-work ceph rtd unknown /proc/68922/task/68928/root (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68928 msgr-work ceph txt unknown /proc/68922/task/68928/exe (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68928 msgr-work ceph NOFD /proc/68922/task/68928/fd (opendir: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68929 msgr-work ceph cwd unknown /proc/68922/task/68929/cwd (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68929 msgr-work ceph rtd unknown /proc/68922/task/68929/root (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68929 msgr-work ceph txt unknown /proc/68922/task/68929/exe (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68929 msgr-work ceph NOFD /proc/68922/task/68929/fd (opendir: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68933 service ceph cwd unknown /proc/68922/task/68933/cwd (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68933 service ceph rtd unknown /proc/68922/task/68933/root (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68933 service ceph txt unknown /proc/68922/task/68933/exe (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68933 service ceph NOFD /proc/68922/task/68933/fd (opendir: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68934 admin_soc ceph cwd unknown /proc/68922/task/68934/cwd (readlink: Permission denied) 2024-04-25T00:24:20.983 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68934 admin_soc ceph rtd unknown /proc/68922/task/68934/root (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68934 admin_soc ceph txt unknown /proc/68922/task/68934/exe (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68934 admin_soc ceph NOFD /proc/68922/task/68934/fd (opendir: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68935 signal_ha ceph cwd unknown /proc/68922/task/68935/cwd (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68935 signal_ha ceph rtd unknown /proc/68922/task/68935/root (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68935 signal_ha ceph txt unknown /proc/68922/task/68935/exe (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68935 signal_ha ceph NOFD /proc/68922/task/68935/fd (opendir: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68936 io_contex ceph cwd unknown /proc/68922/task/68936/cwd (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68936 io_contex ceph rtd unknown /proc/68922/task/68936/root (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68936 io_contex ceph txt unknown /proc/68922/task/68936/exe (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68936 io_contex ceph NOFD /proc/68922/task/68936/fd (opendir: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68937 io_contex ceph cwd unknown /proc/68922/task/68937/cwd (readlink: Permission denied) 2024-04-25T00:24:20.984 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68937 io_contex ceph rtd unknown /proc/68922/task/68937/root (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68937 io_contex ceph txt unknown /proc/68922/task/68937/exe (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68937 io_contex ceph NOFD /proc/68922/task/68937/fd (opendir: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68938 ms_dispat ceph cwd unknown /proc/68922/task/68938/cwd (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68938 ms_dispat ceph rtd unknown /proc/68922/task/68938/root (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68938 ms_dispat ceph txt unknown /proc/68922/task/68938/exe (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68938 ms_dispat ceph NOFD /proc/68922/task/68938/fd (opendir: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68939 ms_local ceph cwd unknown /proc/68922/task/68939/cwd (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68939 ms_local ceph rtd unknown /proc/68922/task/68939/root (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68939 ms_local ceph txt unknown /proc/68922/task/68939/exe (readlink: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68939 ms_local ceph NOFD /proc/68922/task/68939/fd (opendir: Permission denied) 2024-04-25T00:24:20.985 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68940 safe_time ceph cwd unknown /proc/68922/task/68940/cwd (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68940 safe_time ceph rtd unknown /proc/68922/task/68940/root (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68940 safe_time ceph txt unknown /proc/68922/task/68940/exe (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68940 safe_time ceph NOFD /proc/68922/task/68940/fd (opendir: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68941 safe_time ceph cwd unknown /proc/68922/task/68941/cwd (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68941 safe_time ceph rtd unknown /proc/68922/task/68941/root (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68941 safe_time ceph txt unknown /proc/68922/task/68941/exe (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68941 safe_time ceph NOFD /proc/68922/task/68941/fd (opendir: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68942 ceph-mds ceph cwd unknown /proc/68922/task/68942/cwd (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68942 ceph-mds ceph rtd unknown /proc/68922/task/68942/root (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68942 ceph-mds ceph txt unknown /proc/68922/task/68942/exe (readlink: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68942 ceph-mds ceph NOFD /proc/68922/task/68942/fd (opendir: Permission denied) 2024-04-25T00:24:20.986 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68947 safe_time ceph cwd unknown /proc/68922/task/68947/cwd (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68947 safe_time ceph rtd unknown /proc/68922/task/68947/root (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68947 safe_time ceph txt unknown /proc/68922/task/68947/exe (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 68947 safe_time ceph NOFD /proc/68922/task/68947/fd (opendir: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72218 ceph_time ceph cwd unknown /proc/68922/task/72218/cwd (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72218 ceph_time ceph rtd unknown /proc/68922/task/72218/root (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72218 ceph_time ceph txt unknown /proc/68922/task/72218/exe (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72218 ceph_time ceph NOFD /proc/68922/task/72218/fd (opendir: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72220 OpHistory ceph cwd unknown /proc/68922/task/72220/cwd (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72220 OpHistory ceph rtd unknown /proc/68922/task/72220/root (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72220 OpHistory ceph txt unknown /proc/68922/task/72220/exe (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72220 OpHistory ceph NOFD /proc/68922/task/72220/fd (opendir: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72222 ms_dispat ceph cwd unknown /proc/68922/task/72222/cwd (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72222 ms_dispat ceph rtd unknown /proc/68922/task/72222/root (readlink: Permission denied) 2024-04-25T00:24:20.987 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72222 ms_dispat ceph txt unknown /proc/68922/task/72222/exe (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72222 ms_dispat ceph NOFD /proc/68922/task/72222/fd (opendir: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72224 ms_dispat ceph cwd unknown /proc/68922/task/72224/cwd (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72224 ms_dispat ceph rtd unknown /proc/68922/task/72224/root (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72224 ms_dispat ceph txt unknown /proc/68922/task/72224/exe (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72224 ms_dispat ceph NOFD /proc/68922/task/72224/fd (opendir: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72226 mds_rank_ ceph cwd unknown /proc/68922/task/72226/cwd (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72226 mds_rank_ ceph rtd unknown /proc/68922/task/72226/root (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72226 mds_rank_ ceph txt unknown /proc/68922/task/72226/exe (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72226 mds_rank_ ceph NOFD /proc/68922/task/72226/fd (opendir: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72228 PQ_Finish ceph cwd unknown /proc/68922/task/72228/cwd (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72228 PQ_Finish ceph rtd unknown /proc/68922/task/72228/root (readlink: Permission denied) 2024-04-25T00:24:20.988 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72228 PQ_Finish ceph txt unknown /proc/68922/task/72228/exe (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72228 PQ_Finish ceph NOFD /proc/68922/task/72228/fd (opendir: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72230 safe_time ceph cwd unknown /proc/68922/task/72230/cwd (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72230 safe_time ceph rtd unknown /proc/68922/task/72230/root (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72230 safe_time ceph txt unknown /proc/68922/task/72230/exe (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72230 safe_time ceph NOFD /proc/68922/task/72230/fd (opendir: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72232 MR_Finish ceph cwd unknown /proc/68922/task/72232/cwd (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72232 MR_Finish ceph rtd unknown /proc/68922/task/72232/root (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72232 MR_Finish ceph txt unknown /proc/68922/task/72232/exe (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72232 MR_Finish ceph NOFD /proc/68922/task/72232/fd (opendir: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72236 md_submit ceph cwd unknown /proc/68922/task/72236/cwd (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72236 md_submit ceph rtd unknown /proc/68922/task/72236/root (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72236 md_submit ceph txt unknown /proc/68922/task/72236/exe (readlink: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-mds 68922 72236 md_submit ceph NOFD /proc/68922/task/72236/fd (opendir: Permission denied) 2024-04-25T00:24:20.989 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70295 root cwd unknown /proc/70295/cwd (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70295 root rtd unknown /proc/70295/root (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70295 root txt unknown /proc/70295/exe (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70295 root NOFD /proc/70295/fd (opendir: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70296 root cwd unknown /proc/70296/cwd (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70296 root rtd unknown /proc/70296/root (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70296 root txt unknown /proc/70296/exe (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/1 70296 root NOFD /proc/70296/fd (opendir: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 73104 root cwd unknown /proc/73104/cwd (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 73104 root rtd unknown /proc/73104/root (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 73104 root txt unknown /proc/73104/exe (readlink: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/2 73104 root NOFD /proc/73104/fd (opendir: Permission denied) 2024-04-25T00:24:20.990 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 77434 root cwd unknown /proc/77434/cwd (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 77434 root rtd unknown /proc/77434/root (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 77434 root txt unknown /proc/77434/exe (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 77434 root NOFD /proc/77434/fd (opendir: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 79611 root cwd unknown /proc/79611/cwd (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 79611 root rtd unknown /proc/79611/root (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 79611 root txt unknown /proc/79611/exe (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/7 79611 root NOFD /proc/79611/fd (opendir: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 80810 root cwd unknown /proc/80810/cwd (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 80810 root rtd unknown /proc/80810/root (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 80810 root txt unknown /proc/80810/exe (readlink: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/4 80810 root NOFD /proc/80810/fd (opendir: Permission denied) 2024-04-25T00:24:20.991 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-msgr 81336 root cwd unknown /proc/81336/cwd (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-msgr 81336 root rtd unknown /proc/81336/root (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-msgr 81336 root txt unknown /proc/81336/exe (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-msgr 81336 root NOFD /proc/81336/fd (opendir: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-watc 81339 root cwd unknown /proc/81339/cwd (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-watc 81339 root rtd unknown /proc/81339/root (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-watc 81339 root txt unknown /proc/81339/exe (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-watc 81339 root NOFD /proc/81339/fd (opendir: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-comp 81340 root cwd unknown /proc/81340/cwd (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-comp 81340 root rtd unknown /proc/81340/root (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-comp 81340 root txt unknown /proc/81340/exe (readlink: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:ceph-comp 81340 root NOFD /proc/81340/fd (opendir: Permission denied) 2024-04-25T00:24:20.992 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81845 root cwd unknown /proc/81845/cwd (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81845 root rtd unknown /proc/81845/root (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81845 root txt unknown /proc/81845/exe (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81845 root NOFD /proc/81845/fd (opendir: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81958 root cwd unknown /proc/81958/cwd (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81958 root rtd unknown /proc/81958/root (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81958 root txt unknown /proc/81958/exe (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:sshd 81958 root NOFD /proc/81958/fd (opendir: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 84352 root cwd unknown /proc/84352/cwd (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 84352 root rtd unknown /proc/84352/root (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 84352 root txt unknown /proc/84352/exe (readlink: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/5 84352 root NOFD /proc/84352/fd (opendir: Permission denied) 2024-04-25T00:24:20.993 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu rtd DIR 8,1 4096 2 / 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-25T00:24:20.994 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu 0r FIFO 0,13 0t0 235806 pipe 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu 1w FIFO 0,13 0t0 235807 pipe 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:timeout 85816 ubuntu 2w FIFO 0,13 0t0 235808 pipe 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu rtd DIR 8,1 4096 2 / 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-25T00:24:20.995 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-25T00:24:20.996 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu 0r FIFO 0,13 0t0 235806 pipe 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu 1w FIFO 0,13 0t0 235807 pipe 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu 2w FIFO 0,13 0t0 235808 pipe 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:fsync-tes 85834 ubuntu 255r REG 8,1 207 530742 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 87107 root cwd unknown /proc/87107/cwd (readlink: Permission denied) 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 87107 root rtd unknown /proc/87107/root (readlink: Permission denied) 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 87107 root txt unknown /proc/87107/exe (readlink: Permission denied) 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:kworker/u 87107 root NOFD /proc/87107/fd (opendir: Permission denied) 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu rtd DIR 8,1 4096 2 / 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-25T00:24:20.997 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 5431656 62279 /usr/lib64/libcrypto.so.3.2.1 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-25T00:24:20.998 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 0r FIFO 0,13 0t0 235806 pipe 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 1w FIFO 0,13 0t0 235807 pipe 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 2w FIFO 0,13 0t0 235808 pipe 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 3r DIR 0,20 0 1 /proc 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 4r DIR 0,20 0 239684 /proc/87109/fd 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 5w FIFO 0,13 0t0 239689 pipe 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87109 ubuntu 6r FIFO 0,13 0t0 239690 pipe 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:24:20.999 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu rtd DIR 8,1 4096 2 / 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 5431656 62279 /usr/lib64/libcrypto.so.3.2.1 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-25T00:24:21.000 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu 4r FIFO 0,13 0t0 239689 pipe 2024-04-25T00:24:21.001 INFO:tasks.workunit.client.0.smithi045.stdout:lsof 87110 ubuntu 7w FIFO 0,13 0t0 239690 pipe 2024-04-25T00:24:21.001 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:24:21.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:24:21.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:20 smithi045 ceph-mon[29194]: pgmap v74: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.85k op/s 2024-04-25T00:24:21.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:20 smithi071 ceph-mon[32667]: pgmap v74: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.85k op/s 2024-04-25T00:24:21.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:20 smithi122 ceph-mon[32959]: pgmap v74: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.85k op/s 2024-04-25T00:24:21.046 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-04-25T00:24:21.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T00:24:21.384 DEBUG:teuthology.parallel:result is None 2024-04-25T00:24:21.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:24:21.418 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:24:21.418 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T00:24:21.429 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T00:24:21.429 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:24:21.429 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:24:21.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T00:24:21.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T00:24:22.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:22 smithi122 ceph-mon[32959]: pgmap v75: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.63k op/s 2024-04-25T00:24:22.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:22 smithi071 ceph-mon[32667]: pgmap v75: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.63k op/s 2024-04-25T00:24:22.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:22 smithi045 ceph-mon[29194]: pgmap v75: 129 pgs: 129 active+clean; 338 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 1.63k op/s 2024-04-25T00:24:24.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:24 smithi122 ceph-mon[32959]: pgmap v76: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-04-25T00:24:24.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:24 smithi071 ceph-mon[32667]: pgmap v76: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-04-25T00:24:24.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:24 smithi045 ceph-mon[29194]: pgmap v76: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.64k op/s 2024-04-25T00:24:26.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:26 smithi122 ceph-mon[32959]: pgmap v77: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 1.04k op/s 2024-04-25T00:24:26.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:26 smithi071 ceph-mon[32667]: pgmap v77: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 1.04k op/s 2024-04-25T00:24:26.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:26 smithi045 ceph-mon[29194]: pgmap v77: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.7 MiB/s wr, 1.04k op/s 2024-04-25T00:24:28.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:28 smithi122 ceph-mon[32959]: pgmap v78: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 620 op/s 2024-04-25T00:24:28.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:28 smithi071 ceph-mon[32667]: pgmap v78: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 620 op/s 2024-04-25T00:24:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:28 smithi045 ceph-mon[29194]: pgmap v78: 129 pgs: 129 active+clean; 339 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 620 op/s 2024-04-25T00:24:30.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:30 smithi122 ceph-mon[32959]: pgmap v79: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-25T00:24:30.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:30 smithi071 ceph-mon[32667]: pgmap v79: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-25T00:24:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:30 smithi045 ceph-mon[29194]: pgmap v79: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 626 op/s 2024-04-25T00:24:32.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:32 smithi122 ceph-mon[32959]: pgmap v80: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:24:32.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:32 smithi071 ceph-mon[32667]: pgmap v80: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:24:32.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:32 smithi045 ceph-mon[29194]: pgmap v80: 129 pgs: 129 active+clean; 343 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 11 op/s 2024-04-25T00:24:34.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:34 smithi122 ceph-mon[32959]: pgmap v81: 129 pgs: 129 active+clean; 423 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-04-25T00:24:34.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:34 smithi071 ceph-mon[32667]: pgmap v81: 129 pgs: 129 active+clean; 423 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-04-25T00:24:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:34 smithi045 ceph-mon[29194]: pgmap v81: 129 pgs: 129 active+clean; 423 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-04-25T00:24:36.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:36 smithi122 ceph-mon[32959]: pgmap v82: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-04-25T00:24:36.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:36 smithi071 ceph-mon[32667]: pgmap v82: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-04-25T00:24:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:36 smithi045 ceph-mon[29194]: pgmap v82: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-04-25T00:24:38.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:38 smithi122 ceph-mon[32959]: pgmap v83: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-04-25T00:24:38.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:38 smithi071 ceph-mon[32667]: pgmap v83: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-04-25T00:24:38.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:38 smithi045 ceph-mon[29194]: pgmap v83: 129 pgs: 129 active+clean; 470 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-04-25T00:24:40.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:40 smithi122 ceph-mon[32959]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 MiB/s wr, 25 op/s 2024-04-25T00:24:40.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:40 smithi071 ceph-mon[32667]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 MiB/s wr, 25 op/s 2024-04-25T00:24:40.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:40 smithi045 ceph-mon[29194]: pgmap v84: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 MiB/s wr, 25 op/s 2024-04-25T00:24:42.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:42 smithi122 ceph-mon[32959]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-04-25T00:24:42.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:42 smithi071 ceph-mon[32667]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-04-25T00:24:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:42 smithi045 ceph-mon[29194]: pgmap v85: 129 pgs: 129 active+clean; 259 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-04-25T00:24:44.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:44.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 -- 172.21.15.45:0/1111220112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 msgr2=0x7fb7fc10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 --2- 172.21.15.45:0/1111220112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc10e1a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009ca0 tx=0x7fb7f402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 -- 172.21.15.45:0/1111220112 shutdown_connections 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 --2- 172.21.15.45:0/1111220112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc10e1a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 --2- 172.21.15.45:0/1111220112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 0x7fb7fc10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 --2- 172.21.15.45:0/1111220112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 0x7fb7fc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 -- 172.21.15.45:0/1111220112 >> 172.21.15.45:0/1111220112 conn(0x7fb7fc077740 msgr2=0x7fb7fc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 -- 172.21.15.45:0/1111220112 shutdown_connections 2024-04-25T00:24:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.392+0000 7fb800ef7640 1 -- 172.21.15.45:0/1111220112 wait complete. 2024-04-25T00:24:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 Processor -- start 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 -- start start 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 0x7fb7fc198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 0x7fb7fc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc111b00 con 0x7fb7fc10bd00 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc111980 con 0x7fb7fc1029c0 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb800ef7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc111c80 con 0x7fb7fc1032e0 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37806/0 (socket says 172.21.15.45:37806) 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.393+0000 7fb7fad9d640 1 -- 172.21.15.45:0/528603681 learned_addr learned my addr 172.21.15.45:0/528603681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 0x7fb7fc198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 0x7fb7fc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 -- 172.21.15.45:0/528603681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 msgr2=0x7fb7fc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 0x7fb7fc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 -- 172.21.15.45:0/528603681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 msgr2=0x7fb7fc198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 0x7fb7fc198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 -- 172.21.15.45:0/528603681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc19cfa0 con 0x7fb7fc10bd00 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4006010 tx=0x7fb7f4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7eb7fe640 1 -- 172.21.15.45:0/528603681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4047070 con 0x7fb7fc10bd00 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc19d230 con 0x7fb7fc10bd00 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7eb7fe640 1 -- 172.21.15.45:0/528603681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4003e50 con 0x7fb7fc10bd00 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb7eb7fe640 1 -- 172.21.15.45:0/528603681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4033050 con 0x7fb7fc10bd00 2024-04-25T00:24:44.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.394+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc1a4b20 con 0x7fb7fc10bd00 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.396+0000 7fb7eb7fe640 1 -- 172.21.15.45:0/528603681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb7f404a070 con 0x7fb7fc10bd00 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 Processor -- start 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 -- start start 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7eb7fe640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5592168fc6b0 con 0x559216997530 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5592168a4880 con 0x559216997e50 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb803bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592168f8670 con 0x5592169a6a30 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7eb7fe640 1 -- 172.21.15.45:0/528603681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb7f40f0520 con 0x7fb7fc10bd00 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37810/0 (socket says 172.21.15.45:37810) 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7fa59c640 1 -- 172.21.15.45:0/172689278 learned_addr learned my addr 172.21.15.45:0/172689278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.397+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 -- 172.21.15.45:0/172689278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 msgr2=0x5592169a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 -- 172.21.15.45:0/172689278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 msgr2=0x5592169a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 -- 172.21.15.45:0/172689278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592166da7d0 con 0x559216997530 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005fd0 tx=0x7fb7f000b760 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00bcb0 tx=0x7fb7ec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457c65dd3e16d6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7e8ff9640 1 -- 172.21.15.45:0/172689278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec015070 con 0x559216997530 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7e8ff9640 1 -- 172.21.15.45:0/172689278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec011920 con 0x559216997530 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.398+0000 7fb7e8ff9640 1 -- 172.21.15.45:0/172689278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0108d0 con 0x559216997530 2024-04-25T00:24:44.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- 172.21.15.45:0/172689278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 msgr2=0x559216997910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00bcb0 tx=0x7fb7ec00f730 comp rx=0 tx=0).stop 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- 172.21.15.45:0/172689278 shutdown_connections 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 --2- 172.21.15.45:0/172689278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x559216997910 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- 172.21.15.45:0/172689278 >> 172.21.15.45:0/172689278 conn(0x559216993950 msgr2=0x559216993d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- 172.21.15.45:0/172689278 shutdown_connections 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- 172.21.15.45:0/172689278 wait complete. 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 -- start start 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.399+0000 7fb803bfd740 1 Processor -- start 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5592169a97f0 con 0x559216997530 2024-04-25T00:24:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5592168b7f50 con 0x559216997e50 2024-04-25T00:24:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592169a9970 con 0x5592169a6a30 2024-04-25T00:24:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb803bfd740 10 client.0 _ll_register_callbacks cb 0x5592168f3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37814/0 (socket says 172.21.15.45:37814) 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 -- 172.21.15.45:0/1084968058 learned_addr learned my addr 172.21.15.45:0/1084968058 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 -- 172.21.15.45:0/1084968058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 msgr2=0x5592169a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 -- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 msgr2=0x5592169a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 -- 172.21.15.45:0/1084968058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592166103f0 con 0x559216997530 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec010410 tx=0x7fb7ec00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa8d4aa79a3128f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.400+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec015070 con 0x559216997530 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec011920 con 0x559216997530 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb7ec011ad0 con 0x559216997530 2024-04-25T00:24:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec02e050 con 0x559216997530 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 10 client.16407 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592165d00f0 con 0x559216997530 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec011920 con 0x559216997530 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7ec004880 con 0x559216997530 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 20 client.16407 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169b6c60 0x5592169b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 4 client.16407 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559216909bf0 con 0x5592169b6c60 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.401+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169b6c60 0x5592169b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.402+0000 7fb7e1ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.402+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169b6c60 0x5592169b9050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90002ee9b84431b server_cookie=727f78b853a10c86 in_seq=0 out_seq=0 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.402+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/1084968058 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559216909bf0 con 0x5592169b6c60 2024-04-25T00:24:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.402+0000 7fb7e1ffb640 10 client.16407 handle_command_reply: tid=0 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 client.16407 shutdown 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 msgr2=0x5592169a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec010410 tx=0x7fb7ec00b9f0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 shutdown_connections 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592169a6a30 0x5592169a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169b6c60 0x5592169b9050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 --2- 172.21.15.45:0/1084968058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a39c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 >> 172.21.15.45:0/1084968058 conn(0x559216993950 msgr2=0x5592169996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb7e1ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 shutdown_connections 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 1 -- 172.21.15.45:0/1084968058 wait complete. 2024-04-25T00:24:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.403+0000 7fb803bfd740 20 client.16407 trim_cache size 0 max 16384 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 Processor -- start 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 -- start start 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 0x5592169b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216997530 0x559216999820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1ac8c0 con 0x559216997e50 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1ae400 con 0x559216993950 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb803bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1acf90 con 0x559216997530 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37820/0 (socket says 172.21.15.45:37820) 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 -- 172.21.15.45:0/983407907 learned_addr learned my addr 172.21.15.45:0/983407907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216997530 0x559216999820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 0x5592169b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 -- 172.21.15.45:0/983407907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216997530 msgr2=0x559216999820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216997530 0x559216999820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 -- 172.21.15.45:0/983407907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 msgr2=0x5592169b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 0x5592169b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fad9d640 1 -- 172.21.15.45:0/983407907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559216985090 con 0x559216997e50 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.421+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 0x5592169b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4004af0 tx=0x7fb7f4009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d42a7b7b11919dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb7e3fff640 1 -- 172.21.15.45:0/983407907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40bcd80 con 0x559216997e50 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb7e3fff640 1 -- 172.21.15.45:0/983407907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40879a0 con 0x559216997e50 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb7e3fff640 1 -- 172.21.15.45:0/983407907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40b6380 con 0x559216997e50 2024-04-25T00:24:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 -- 172.21.15.45:0/983407907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 msgr2=0x5592169be740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4004af0 tx=0x7fb7f4009980 comp rx=0 tx=0).stop 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 -- 172.21.15.45:0/983407907 shutdown_connections 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997e50 0x5592169be740 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216997530 0x559216999820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 --2- 172.21.15.45:0/983407907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216993950 0x5592169b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 -- 172.21.15.45:0/983407907 >> 172.21.15.45:0/983407907 conn(0x5592169a2620 msgr2=0x559216992af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 -- 172.21.15.45:0/983407907 shutdown_connections 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.422+0000 7fb803bfd740 1 -- 172.21.15.45:0/983407907 wait complete. 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 -- start start 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 Processor -- start 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216993950 0x5592169a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1add30 con 0x559216997530 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1ad660 con 0x559216997e50 2024-04-25T00:24:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1aa8a0 con 0x559216993950 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb803bfd740 10 client.0 _ll_register_callbacks cb 0x7fb7f402fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37822/0 (socket says 172.21.15.45:37822) 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 -- 172.21.15.45:0/228105142 learned_addr learned my addr 172.21.15.45:0/228105142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216993950 0x5592169a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7fad9d640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 -- 172.21.15.45:0/228105142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216993950 msgr2=0x5592169a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216993950 0x5592169a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 -- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 msgr2=0x5592169a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.423+0000 7fb7f9d9b640 1 -- 172.21.15.45:0/228105142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559216a1bfb0 con 0x559216997530 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb7f9d9b640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005d80 tx=0x7fb7f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e762a8c4dee515c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f002f060 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb7f0011db0 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f40879a0 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7f00292d0 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb803bfd740 10 client.16413 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.424+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559216a1d2a0 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f40879a0 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7f0021b10 con 0x559216997530 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 10 client.16413 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 20 client.16413 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169a6a30 0x559216a1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 4 client.16413 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559216903f40 con 0x5592169a6a30 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169a6a30 0x559216a1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb7e1ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.425+0000 7fb7fa59c640 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169a6a30 0x559216a1fa70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b72af5cdee50ec1 server_cookie=775be1e8852b0962 in_seq=0 out_seq=0 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb7e1ffb640 1 -- 172.21.15.45:0/228105142 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559216903f40 con 0x5592169a6a30 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb7e1ffb640 10 client.16413 handle_command_reply: tid=0 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 client.16413 shutdown 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 msgr2=0x5592169a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005d80 tx=0x7fb7f0004730 comp rx=0 tx=0).stop 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 shutdown_connections 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559216997e50 0x5592169a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5592169a6a30 0x559216a1fa70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559216997530 0x5592169a0bd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 --2- 172.21.15.45:0/228105142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559216993950 0x5592169a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb7e1ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 >> 172.21.15.45:0/228105142 conn(0x5592169a2620 msgr2=0x5592169a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 shutdown_connections 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 1 -- 172.21.15.45:0/228105142 wait complete. 2024-04-25T00:24:44.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb803bfd740 20 client.16413 trim_cache size 0 max 16384 2024-04-25T00:24:44.433 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:44.433 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:24:44.433 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:24:44.433 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 msgr2=0x7fb7e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005fd0 tx=0x7fb7f000b760 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 msgr2=0x7fb7fc19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.426+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4006010 tx=0x7fb7f4003ce0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 shutdown_connections 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7fc10bd00 0x7fb7fc19c870 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1032e0 0x7fb7fc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 --2- 172.21.15.45:0/528603681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1029c0 0x7fb7fc198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 >> 172.21.15.45:0/528603681 conn(0x7fb7fc077740 msgr2=0x7fb7fc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 shutdown_connections 2024-04-25T00:24:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.427+0000 7fb800ef7640 1 -- 172.21.15.45:0/528603681 wait complete. 2024-04-25T00:24:44.438 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:24:44.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 -- 172.21.15.45:0/3007831539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650107720 msgr2=0x7f4650107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 --2- 172.21.15.45:0/3007831539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650107720 0x7f4650107b00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4638009930 tx=0x7f463802a420 comp rx=0 tx=0).stop 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 -- 172.21.15.45:0/3007831539 shutdown_connections 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 --2- 172.21.15.45:0/3007831539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 0x7f465010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 --2- 172.21.15.45:0/3007831539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650101710 0x7f4650101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 --2- 172.21.15.45:0/3007831539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650107720 0x7f4650107b00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 -- 172.21.15.45:0/3007831539 >> 172.21.15.45:0/3007831539 conn(0x7f4650078b00 msgr2=0x7f4650078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 -- 172.21.15.45:0/3007831539 shutdown_connections 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.540+0000 7f4656a4d640 1 -- 172.21.15.45:0/3007831539 wait complete. 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 Processor -- start 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 -- start start 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 0x7f4650073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 0x7f46500744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4650111b40 con 0x7f4650101710 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46501119c0 con 0x7f4650107720 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4656a4d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4650111cc0 con 0x7f46501020d0 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f464ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f464ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37840/0 (socket says 172.21.15.45:37840) 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f464ffff640 1 -- 172.21.15.45:0/2243674811 learned_addr learned my addr 172.21.15.45:0/2243674811 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f464f7fe640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 0x7f4650073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.541+0000 7f4654fea640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 0x7f46500744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 -- 172.21.15.45:0/2243674811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 msgr2=0x7f4650073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 0x7f4650073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 -- 172.21.15.45:0/2243674811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 msgr2=0x7f46500744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 0x7f46500744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 -- 172.21.15.45:0/2243674811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46501a64c0 con 0x7f4650101710 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f4654fea640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 0x7f46500744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464ffff640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f463802a940 tx=0x7f4638003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803f070 con 0x7f4650101710 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46501a6750 con 0x7f4650101710 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638031c90 con 0x7f4650101710 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638035030 con 0x7f4650101710 2024-04-25T00:24:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.542+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501a6c40 con 0x7f4650101710 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.543+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46500689b0 con 0x7f4650101710 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.544+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4638042070 con 0x7f4650101710 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.545+0000 7f464d7fa640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 0x7f463c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.545+0000 7f464f7fe640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 0x7f463c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.545+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f46380f0080 con 0x7f4650101710 2024-04-25T00:24:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.545+0000 7f464f7fe640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 0x7f463c0a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46500754d0 tx=0x7f464000a2d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.549+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46380bc8e0 con 0x7f4650101710 2024-04-25T00:24:44.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:44 smithi045 ceph-mon[29194]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-04-25T00:24:44.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.774+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4650109c80 con 0x7f4650101710 2024-04-25T00:24:44.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.775+0000 7f464d7fa640 1 -- 172.21.15.45:0/2243674811 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f46380dae20 con 0x7f4650101710 2024-04-25T00:24:44.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:24:44.776 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:24:44.776 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 msgr2=0x7f463c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 0x7f463c0a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46500754d0 tx=0x7f464000a2d0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 msgr2=0x7f4650073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f463802a940 tx=0x7f4638003c00 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 shutdown_connections 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f463c0a2d00 0x7f463c0a51c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4650107720 0x7f46500744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46501020d0 0x7f4650073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 --2- 172.21.15.45:0/2243674811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4650101710 0x7f4650073a40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 >> 172.21.15.45:0/2243674811 conn(0x7f4650078b00 msgr2=0x7f46501040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 shutdown_connections 2024-04-25T00:24:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.777+0000 7f4656a4d640 1 -- 172.21.15.45:0/2243674811 wait complete. 2024-04-25T00:24:44.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:24:44.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:44 smithi122 ceph-mon[32959]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-04-25T00:24:44.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:44 smithi071 ceph-mon[32667]: pgmap v86: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 -- 172.21.15.45:0/3264649089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 msgr2=0x7f785810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 --2- 172.21.15.45:0/3264649089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f785810d4f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7850009cd0 tx=0x7f785002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 -- 172.21.15.45:0/3264649089 shutdown_connections 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 --2- 172.21.15.45:0/3264649089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f785810d4f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 --2- 172.21.15.45:0/3264649089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858100620 0x7f7858100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 --2- 172.21.15.45:0/3264649089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858106630 0x7f7858106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 -- 172.21.15.45:0/3264649089 >> 172.21.15.45:0/3264649089 conn(0x7f78580776a0 msgr2=0x7f7858077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 -- 172.21.15.45:0/3264649089 shutdown_connections 2024-04-25T00:24:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 -- 172.21.15.45:0/3264649089 wait complete. 2024-04-25T00:24:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.894+0000 7f785d8c4640 1 Processor -- start 2024-04-25T00:24:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 -- start start 2024-04-25T00:24:44.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858100620 0x7f7858197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858106630 0x7f785819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7858110a40 con 0x7f7858100fe0 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78581108c0 con 0x7f7858100620 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f785d8c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7858110bc0 con 0x7f7858106630 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37854/0 (socket says 172.21.15.45:37854) 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 -- 172.21.15.45:0/887956306 learned_addr learned my addr 172.21.15.45:0/887956306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f7856ffd640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858100620 0x7f7858197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78577fe640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858106630 0x7f785819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 -- 172.21.15.45:0/887956306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858106630 msgr2=0x7f785819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858106630 0x7f785819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 -- 172.21.15.45:0/887956306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858100620 msgr2=0x7f7858197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858100620 0x7f7858197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 -- 172.21.15.45:0/887956306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785819cf70 con 0x7f7858100fe0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.895+0000 7f78567fc640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.896+0000 7f785c8c2640 1 -- 172.21.15.45:0/887956306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c00cd80 con 0x7f7858100fe0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.896+0000 7f785c8c2640 1 -- 172.21.15.45:0/887956306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c005500 con 0x7f7858100fe0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.896+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785819d260 con 0x7f7858100fe0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.896+0000 7f785c8c2640 1 -- 172.21.15.45:0/887956306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c010690 con 0x7f7858100fe0 2024-04-25T00:24:44.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.896+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a4af0 con 0x7f7858100fe0 2024-04-25T00:24:44.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f785c8c2640 1 -- 172.21.15.45:0/887956306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f784c0046c0 con 0x7f7858100fe0 2024-04-25T00:24:44.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f78605d8740 1 Processor -- start 2024-04-25T00:24:44.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f78605d8740 1 -- start start 2024-04-25T00:24:44.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f785c8c2640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 0x7f78400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f78605d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b1a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f78605d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.898+0000 7f78605d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b29980 0x560309b2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78605d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560309a7f670 con 0x560309b1ac60 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78605d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560309a27970 con 0x560309b29980 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78605d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560309a7b630 con 0x560309b1a2b0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78577fe640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 0x7f78400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f785c8c2640 1 -- 172.21.15.45:0/887956306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f784c0cfc80 con 0x7f7858100fe0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37864/0 (socket says 172.21.15.45:37864) 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78567fc640 1 -- 172.21.15.45:0/390178415 learned_addr learned my addr 172.21.15.45:0/390178415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b1a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78577fe640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b29980 0x560309b2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 -- 172.21.15.45:0/390178415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b29980 msgr2=0x560309b2bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b29980 0x560309b2bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 -- 172.21.15.45:0/390178415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 msgr2=0x560309b29440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 -- 172.21.15.45:0/390178415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560309a4b400 con 0x560309b1a2b0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78567fc640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f78577fe640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 0x7f78400a51c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7850009a60 tx=0x7f78500316c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f7856ffd640 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b1a690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7848009c50 tx=0x7f7848011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be5034fa0e4f155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f783dffb640 1 -- 172.21.15.45:0/390178415 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848004050 con 0x560309b1a2b0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.899+0000 7f783dffb640 1 -- 172.21.15.45:0/390178415 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848005500 con 0x560309b1a2b0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f783dffb640 1 -- 172.21.15.45:0/390178415 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848004dd0 con 0x560309b1a2b0 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- 172.21.15.45:0/390178415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 msgr2=0x560309b1a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b1a690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7848009c50 tx=0x7f7848011810 comp rx=0 tx=0).stop 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- 172.21.15.45:0/390178415 shutdown_connections 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b29980 0x560309b2bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b29440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 --2- 172.21.15.45:0/390178415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b1a690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- 172.21.15.45:0/390178415 >> 172.21.15.45:0/390178415 conn(0x560309b168a0 msgr2=0x560309b16c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- 172.21.15.45:0/390178415 shutdown_connections 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- 172.21.15.45:0/390178415 wait complete. 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.900+0000 7f78605d8740 1 -- start start 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 Processor -- start 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b26b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560309b2c740 con 0x560309b1ac60 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560309a3b050 con 0x560309b1a2b0 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560309b2c8c0 con 0x560309b29980 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78605d8740 10 client.0 _ll_register_callbacks cb 0x560309792be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37868/0 (socket says 172.21.15.45:37868) 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 -- 172.21.15.45:0/3153390209 learned_addr learned my addr 172.21.15.45:0/3153390209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78577fe640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b26b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 -- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 msgr2=0x560309b26b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b26b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 -- 172.21.15.45:0/3153390209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 msgr2=0x560309b292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f78567fc640 1 -- 172.21.15.45:0/3153390209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560309a0cdc0 con 0x560309b1ac60 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.901+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78567fc640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f784c098810 tx=0x7f784c04a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4864f4ff968ee136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c09e070 con 0x560309b1ac60 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f784c0a4820 con 0x560309b1ac60 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7848005500 con 0x560309b1ac60 2024-04-25T00:24:44.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f784c09c540 con 0x560309b1ac60 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 10 client.16431 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603097933f0 con 0x560309b1ac60 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848005500 con 0x560309b1ac60 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f784c09b290 con 0x560309b1ac60 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 10 client.16431 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 10 client.16431 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 20 client.16431 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b39a30 0x560309b3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 4 client.16431 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.902+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560309a8cbb0 con 0x560309b39a30 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.903+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b39a30 0x560309b3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.903+0000 7f7836ffd640 10 client.16431 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.903+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b39a30 0x560309b3bea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a8662c1ef643c20 server_cookie=a9dc5b7d5c866634 in_seq=0 out_seq=0 2024-04-25T00:24:44.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f7836ffd640 1 -- 172.21.15.45:0/3153390209 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560309a8cbb0 con 0x560309b39a30 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f7836ffd640 10 client.16431 handle_command_reply: tid=0 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 client.16431 shutdown 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 msgr2=0x560309b265c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f784c098810 tx=0x7f784c04a7f0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 shutdown_connections 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b39a30 0x560309b3bea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b26b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b265c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f7836ffd640 0 client.16431 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 --2- 172.21.15.45:0/3153390209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 >> 172.21.15.45:0/3153390209 conn(0x560309b168a0 msgr2=0x560309b1c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 shutdown_connections 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 1 -- 172.21.15.45:0/3153390209 wait complete. 2024-04-25T00:24:44.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.904+0000 7f78605d8740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.917+0000 7f78605d8740 1 Processor -- start 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.917+0000 7f78605d8740 1 -- start start 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b35640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 0x560309b28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ac870 con 0x560309b29980 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ae3b0 con 0x560309b1ac60 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78605d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78581acf40 con 0x560309b1a2b0 2024-04-25T00:24:44.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37880/0 (socket says 172.21.15.45:37880) 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 -- 172.21.15.45:0/1789424234 learned_addr learned my addr 172.21.15.45:0/1789424234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78567fc640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 0x560309b28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f7856ffd640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b35640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 -- 172.21.15.45:0/1789424234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 msgr2=0x560309b35640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b35640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 -- 172.21.15.45:0/1789424234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 msgr2=0x560309b28e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 0x560309b28e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 -- 172.21.15.45:0/1789424234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030985d7d0 con 0x560309b29980 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78567fc640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 0x560309b28e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f78577fe640 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7850009cd0 tx=0x7f7850009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ee6ac6f5b5a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.918+0000 7f783d7fa640 1 -- 172.21.15.45:0/1789424234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785004f070 con 0x560309b29980 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f783d7fa640 1 -- 172.21.15.45:0/1789424234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7850036040 con 0x560309b29980 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f783d7fa640 1 -- 172.21.15.45:0/1789424234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850030d80 con 0x560309b29980 2024-04-25T00:24:44.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- 172.21.15.45:0/1789424234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 msgr2=0x560309b41740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7850009cd0 tx=0x7f7850009360 comp rx=0 tx=0).stop 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- 172.21.15.45:0/1789424234 shutdown_connections 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b29980 0x560309b41740 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1ac60 0x560309b28e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 --2- 172.21.15.45:0/1789424234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b1a2b0 0x560309b35640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- 172.21.15.45:0/1789424234 >> 172.21.15.45:0/1789424234 conn(0x560309b168a0 msgr2=0x560309b255f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- 172.21.15.45:0/1789424234 shutdown_connections 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- 172.21.15.45:0/1789424234 wait complete. 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.919+0000 7f78605d8740 1 -- start start 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 Processor -- start 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b97f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78581adce0 con 0x560309b1ac60 2024-04-25T00:24:44.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78581ad610 con 0x560309b1a2b0 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78581aa850 con 0x560309b29980 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78605d8740 10 client.0 _ll_register_callbacks cb 0x7f784c04a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37886/0 (socket says 172.21.15.45:37886) 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 -- 172.21.15.45:0/3405765777 learned_addr learned my addr 172.21.15.45:0/3405765777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 -- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 msgr2=0x560309b97f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78577fe640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b97f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b97f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 -- 172.21.15.45:0/3405765777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 msgr2=0x560309b2d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b2d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 -- 172.21.15.45:0/3405765777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560309b985e0 con 0x560309b1ac60 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:44.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78577fe640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f78567fc640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f784c09b600 tx=0x7f784c004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4503d806ee7a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.920+0000 7f7836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c09e070 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f784c0a4830 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7850036040 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f784c0a01f0 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 10 client.16437 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560309b9f9a0 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7850036040 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f784c04ace0 con 0x560309b1ac60 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 10 client.16437 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 20 client.16437 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b9fe80 0x560309ba2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 4 client.16437 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.921+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560309a86f00 con 0x560309b9fe80 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b9fe80 0x560309ba2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7856ffd640 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b9fe80 0x560309ba2270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6377242eb830ea6 server_cookie=33015d2071109a0 in_seq=0 out_seq=0 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7836ffd640 10 client.16437 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7836ffd640 1 -- 172.21.15.45:0/3405765777 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560309a86f00 con 0x560309b9fe80 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7836ffd640 10 client.16437 handle_command_reply: tid=0 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 client.16437 shutdown 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 msgr2=0x560309b2d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f784c09b600 tx=0x7f784c004ae0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 shutdown_connections 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x560309b9fe80 0x560309ba2270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560309b29980 0x560309b97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f7836ffd640 0 client.16437 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560309b1ac60 0x560309b2d590 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 --2- 172.21.15.45:0/3405765777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560309b1a2b0 0x560309b2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.922+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 >> 172.21.15.45:0/3405765777 conn(0x560309b168a0 msgr2=0x560309b1be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 shutdown_connections 2024-04-25T00:24:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f78605d8740 1 -- 172.21.15.45:0/3405765777 wait complete. 2024-04-25T00:24:44.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f78605d8740 20 client.16437 trim_cache size 0 max 16384 2024-04-25T00:24:44.930 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 msgr2=0x7f78400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 0x7f78400a51c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7850009a60 tx=0x7f78500316c0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 msgr2=0x7f7858198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f784c00e9f0 tx=0x7f784c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 shutdown_connections 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78400a2d00 0x7f78400a51c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7858106630 0x7f785819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7858100fe0 0x7f7858198510 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 --2- 172.21.15.45:0/887956306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7858100620 0x7f7858197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 >> 172.21.15.45:0/887956306 conn(0x7f78580776a0 msgr2=0x7f78580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 shutdown_connections 2024-04-25T00:24:44.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:44.923+0000 7f785d8c4640 1 -- 172.21.15.45:0/887956306 wait complete. 2024-04-25T00:24:44.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:24:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/799147055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb4101710 msgr2=0x7f6cb4101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/799147055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb4101710 0x7f6cb4101b90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009930 tx=0x7f6ca402a420 comp rx=0 tx=0).stop 2024-04-25T00:24:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/799147055 shutdown_connections 2024-04-25T00:24:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/799147055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb41020d0 0x7f6cb410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/799147055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb4101710 0x7f6cb4101b90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/799147055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/799147055 >> 172.21.15.45:0/799147055 conn(0x7f6cb4078b90 msgr2=0x7f6cb4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/799147055 shutdown_connections 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.038+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/799147055 wait complete. 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 Processor -- start 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 -- start start 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4101710 0x7f6cb41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4111b80 con 0x7f6cb41020d0 2024-04-25T00:24:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4111a00 con 0x7f6cb4101710 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cbaa5d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4111d00 con 0x7f6cb4107720 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.039+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37912/0 (socket says 172.21.15.45:37912) 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 -- 172.21.15.45:0/470044699 learned_addr learned my addr 172.21.15.45:0/470044699 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4101710 0x7f6cb41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 -- 172.21.15.45:0/470044699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 msgr2=0x7f6cb419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 -- 172.21.15.45:0/470044699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4101710 msgr2=0x7f6cb41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4101710 0x7f6cb41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 -- 172.21.15.45:0/470044699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb419d0b0 con 0x7f6cb41020d0 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402a940 tx=0x7f6ca4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb17fa640 1 -- 172.21.15.45:0/470044699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca403f070 con 0x7f6cb41020d0 2024-04-25T00:24:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb17fa640 1 -- 172.21.15.45:0/470044699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca4031c90 con 0x7f6cb41020d0 2024-04-25T00:24:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb419d340 con 0x7f6cb41020d0 2024-04-25T00:24:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cb17fa640 1 -- 172.21.15.45:0/470044699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4035030 con 0x7f6cb41020d0 2024-04-25T00:24:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.040+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a4c30 con 0x7f6cb41020d0 2024-04-25T00:24:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb17fa640 1 -- 172.21.15.45:0/470044699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ca4042070 con 0x7f6cb41020d0 2024-04-25T00:24:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 Processor -- start 2024-04-25T00:24:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 -- start start 2024-04-25T00:24:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7b27fb670 con 0x55d7b2896480 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d7b27a3970 con 0x55d7b2896da0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7b27f7630 con 0x55d7b28a5980 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb17fa640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 0x7f6c9c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb8ffa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb8ffa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37922/0 (socket says 172.21.15.45:37922) 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb8ffa640 1 -- 172.21.15.45:0/1804921881 learned_addr learned my addr 172.21.15.45:0/1804921881 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb17fa640 1 -- 172.21.15.45:0/470044699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6ca40eff40 con 0x7f6cb41020d0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 0x7f6c9c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.043+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 -- 172.21.15.45:0/1804921881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 msgr2=0x55d7b28a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 -- 172.21.15.45:0/1804921881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 msgr2=0x55d7b28a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 -- 172.21.15.45:0/1804921881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b27c7400 con 0x55d7b2896480 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6cac014a90 tx=0x7f6cac014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e053d831bfe18b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6c9a7fc640 1 -- 172.21.15.45:0/1804921881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac014530 con 0x55d7b2896480 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6c9a7fc640 1 -- 172.21.15.45:0/1804921881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cac023070 con 0x55d7b2896480 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6c9a7fc640 1 -- 172.21.15.45:0/1804921881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00cca0 con 0x55d7b2896480 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.044+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 0x7f6c9c0a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6cac022000 tx=0x7f6cac020000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1804921881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 msgr2=0x55d7b2896860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6cac014a90 tx=0x7f6cac014f50 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1804921881 shutdown_connections 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1804921881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b2896860 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1804921881 >> 172.21.15.45:0/1804921881 conn(0x55d7b28928a0 msgr2=0x55d7b2892c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1804921881 shutdown_connections 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1804921881 wait complete. 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 -- start start 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.045+0000 7f6cbd76f740 1 Processor -- start 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d7b28a8740 con 0x55d7b2896480 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d7b27b7050 con 0x55d7b2896da0 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7b28a88c0 con 0x55d7b28a5980 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cbd76f740 10 client.0 _ll_register_callbacks cb 0x55d7b250ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37928/0 (socket says 172.21.15.45:37928) 2024-04-25T00:24:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 -- 172.21.15.45:0/190143836 learned_addr learned my addr 172.21.15.45:0/190143836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 -- 172.21.15.45:0/190143836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 msgr2=0x55d7b28a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 -- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 msgr2=0x55d7b28a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 -- 172.21.15.45:0/190143836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b2788dc0 con 0x55d7b2896480 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012700 tx=0x7f6ca8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395a859066844b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6c937fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8004050 con 0x55d7b2896480 2024-04-25T00:24:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cac023070 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.046+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6ca80041f0 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ca8025050 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7b250f3f0 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cac023070 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ca8003a10 con 0x55d7b2896480 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 10 client.16449 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 20 client.16449 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b28b59f0 0x55d7b28b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 4 client.16449 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7b2808bb0 con 0x55d7b28b59f0 2024-04-25T00:24:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.047+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b28b59f0 0x55d7b28b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.048+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b28b59f0 0x55d7b28b7e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cab2940d796514cf server_cookie=d44c74b37dd500a4 in_seq=0 out_seq=0 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.048+0000 7f6c937fe640 10 client.16449 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.048+0000 7f6c937fe640 1 -- 172.21.15.45:0/190143836 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7b2808bb0 con 0x55d7b28b59f0 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.048+0000 7f6c937fe640 10 client.16449 handle_command_reply: tid=0 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 client.16449 shutdown 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 msgr2=0x55d7b28a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012700 tx=0x7f6ca8012bc0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 shutdown_connections 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b28b59f0 0x55d7b28b7e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896da0 0x55d7b28a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6c937fe640 0 client.16449 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/190143836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896480 0x55d7b28a52a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 >> 172.21.15.45:0/190143836 conn(0x55d7b28928a0 msgr2=0x55d7b2891b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 shutdown_connections 2024-04-25T00:24:45.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 1 -- 172.21.15.45:0/190143836 wait complete. 2024-04-25T00:24:45.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.049+0000 7f6cbd76f740 20 client.16449 trim_cache size 0 max 16384 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 Processor -- start 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 -- start start 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 0x55d7b28b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ac9b0 con 0x55d7b2896da0 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ae4f0 con 0x55d7b2896480 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ad080 con 0x55d7b28a5980 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37934/0 (socket says 172.21.15.45:37934) 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.062+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1377203542 learned_addr learned my addr 172.21.15.45:0/1377203542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 0x55d7b28b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1377203542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 msgr2=0x55d7b28bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1377203542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 msgr2=0x55d7b28b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 0x55d7b28b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1377203542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7b25d97d0 con 0x55d7b2896da0 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 0x55d7b28b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40b8ad0 tx=0x7f6ca4031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186c5aab81b852c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6c99ffb640 1 -- 172.21.15.45:0/1377203542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4049040 con 0x55d7b2896da0 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6c99ffb640 1 -- 172.21.15.45:0/1377203542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca40c04b0 con 0x55d7b2896da0 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.063+0000 7f6c99ffb640 1 -- 172.21.15.45:0/1377203542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40ed5b0 con 0x55d7b2896da0 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1377203542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 msgr2=0x55d7b28a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40b8ad0 tx=0x7f6ca4031490 comp rx=0 tx=0).stop 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1377203542 shutdown_connections 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b28a5980 0x55d7b28bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a4dd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1377203542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b2896480 0x55d7b28b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1377203542 >> 172.21.15.45:0/1377203542 conn(0x55d7b28928a0 msgr2=0x55d7b28a26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1377203542 shutdown_connections 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1377203542 wait complete. 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- start start 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 Processor -- start 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b2896480 0x55d7b28a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 0x55d7b28aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ade20 con 0x55d7b2896da0 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ad750 con 0x55d7b28a5980 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.064+0000 7f6cbd76f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41aa990 con 0x55d7b2896480 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cbd76f740 10 client.0 _ll_register_callbacks cb 0x7f6ca8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37948/0 (socket says 172.21.15.45:37948) 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1652257674 learned_addr learned my addr 172.21.15.45:0/1652257674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b2896480 0x55d7b28a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 0x55d7b28aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1652257674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b2896480 msgr2=0x55d7b28a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b2896480 0x55d7b28a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 msgr2=0x55d7b28aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 0x55d7b28aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 -- 172.21.15.45:0/1652257674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7b291b0a0 con 0x55d7b2896da0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb8ffa640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 0x55d7b28aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cb37fe640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca40bc280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f228354692ef7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6c937fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40d4070 con 0x55d7b2896da0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6ca403b460 con 0x55d7b2896da0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca40c04b0 con 0x55d7b2896da0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ca4049070 con 0x55d7b2896da0 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cbd76f740 10 client.16455 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.065+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7b291c2f0 con 0x55d7b2896da0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ca40c04b0 con 0x55d7b2896da0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ca403bce0 con 0x55d7b2896da0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 10 client.16455 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 10 client.16455 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 20 client.16455 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b291ca70 0x55d7b291ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 4 client.16455 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7b2802f00 con 0x55d7b291ca70 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b291ca70 0x55d7b291ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6cb3fff640 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b291ca70 0x55d7b291ee90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc6e2022524ac68a server_cookie=938861e8ef88eb3a in_seq=0 out_seq=0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.066+0000 7f6c937fe640 10 client.16455 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6c937fe640 1 -- 172.21.15.45:0/1652257674 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7b2802f00 con 0x55d7b291ca70 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6c937fe640 10 client.16455 handle_command_reply: tid=0 2024-04-25T00:24:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 client.16455 shutdown 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 msgr2=0x55d7b28a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4000c00 tx=0x7f6ca40bc280 comp rx=0 tx=0).stop 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 shutdown_connections 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d7b291ca70 0x55d7b291ee90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7b28a5980 0x55d7b28aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d7b2896da0 0x55d7b28a9c10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 --2- 172.21.15.45:0/1652257674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7b2896480 0x55d7b28a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6c937fe640 0 client.16455 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 >> 172.21.15.45:0/1652257674 conn(0x55d7b28928a0 msgr2=0x55d7b28a1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 shutdown_connections 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 1 -- 172.21.15.45:0/1652257674 wait complete. 2024-04-25T00:24:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbd76f740 20 client.16455 trim_cache size 0 max 16384 2024-04-25T00:24:45.076 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 msgr2=0x7f6c9c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 0x7f6c9c0a51c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6cac022000 tx=0x7f6cac020000 comp rx=0 tx=0).stop 2024-04-25T00:24:45.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 msgr2=0x7f6cb4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402a940 tx=0x7f6ca4003b90 comp rx=0 tx=0).stop 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 shutdown_connections 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.067+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6c9c0a2d00 0x7f6c9c0a51c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cb4107720 0x7f6cb419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cb41020d0 0x7f6cb4198600 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 --2- 172.21.15.45:0/470044699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4101710 0x7f6cb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 >> 172.21.15.45:0/470044699 conn(0x7f6cb4078b90 msgr2=0x7f6cb40ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 shutdown_connections 2024-04-25T00:24:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.068+0000 7f6cbaa5d640 1 -- 172.21.15.45:0/470044699 wait complete. 2024-04-25T00:24:45.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/2075929604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb01002e0 msgr2=0x7f9eb01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/2075929604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb01002e0 0x7f9eb01006c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009900 tx=0x7f9ea002a410 comp rx=0 tx=0).stop 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/2075929604 shutdown_connections 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/2075929604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb0106aa0 0x7f9eb010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/2075929604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb0100c90 0x7f9eb01063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/2075929604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb01002e0 0x7f9eb01006c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/2075929604 >> 172.21.15.45:0/2075929604 conn(0x7f9eb00758d0 msgr2=0x7f9eb0075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/2075929604 shutdown_connections 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.182+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/2075929604 wait complete. 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 Processor -- start 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 -- start start 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 0x7f9eb0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb019c8c0 0x7f9eb019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111b90 con 0x7f9eb0106aa0 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111a10 con 0x7f9eb019c8c0 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eb5d1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0111d10 con 0x7f9eb01002e0 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37968/0 (socket says 172.21.15.45:37968) 2024-04-25T00:24:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eaeffd640 1 -- 172.21.15.45:0/697457222 learned_addr learned my addr 172.21.15.45:0/697457222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eaffff640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb019c8c0 0x7f9eb019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.183+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 0x7f9eb0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 -- 172.21.15.45:0/697457222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 msgr2=0x7f9eb0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 0x7f9eb0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 -- 172.21.15.45:0/697457222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb019c8c0 msgr2=0x7f9eb019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb019c8c0 0x7f9eb019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 -- 172.21.15.45:0/697457222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb019d3b0 con 0x7f9eb0106aa0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 0x7f9eb0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c970 tx=0x7f9ea400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eacff9640 1 -- 172.21.15.45:0/697457222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4014070 con 0x7f9eb0106aa0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eacff9640 1 -- 172.21.15.45:0/697457222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4004440 con 0x7f9eb0106aa0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb019d6a0 con 0x7f9eb0106aa0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eacff9640 1 -- 172.21.15.45:0/697457222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400f450 con 0x7f9eb0106aa0 2024-04-25T00:24:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.184+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a4e90 con 0x7f9eb0106aa0 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.186+0000 7f9eacff9640 1 -- 172.21.15.45:0/697457222 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ea4017070 con 0x7f9eb0106aa0 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.186+0000 7f9eb8a2f740 1 Processor -- start 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.186+0000 7f9eb8a2f740 1 -- start start 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eacff9640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 0x7f9e980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56429862a670 con 0x5642986c5ea0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5642985d2970 con 0x5642986c5580 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564298626630 con 0x5642986d0570 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaffff640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 0x7f9e980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37976/0 (socket says 172.21.15.45:37976) 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/3509008938 learned_addr learned my addr 172.21.15.45:0/3509008938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eacff9640 1 -- 172.21.15.45:0/697457222 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9ea40cf6c0 con 0x7f9eb0106aa0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaffff640 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/3509008938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 msgr2=0x5642986d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/3509008938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 msgr2=0x5642986c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/3509008938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642985f6400 con 0x5642986c5ea0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaffff640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 0x7f9e980a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005cc0 tx=0x7f9ea800a8a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.187+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea0032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a5e18f9060fe61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9e95ffb640 1 -- 172.21.15.45:0/3509008938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003f070 con 0x5642986c5ea0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9e95ffb640 1 -- 172.21.15.45:0/3509008938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00317b0 con 0x5642986c5ea0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9e95ffb640 1 -- 172.21.15.45:0/3509008938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0035070 con 0x5642986c5ea0 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3509008938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 msgr2=0x5642986d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea0032d70 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3509008938 shutdown_connections 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986d0030 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3509008938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3509008938 >> 172.21.15.45:0/3509008938 conn(0x5642986c18a0 msgr2=0x5642986c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3509008938 shutdown_connections 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.188+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3509008938 wait complete. 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 -- start start 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 Processor -- start 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5642986d3300 con 0x5642986c5ea0 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5642985e6050 con 0x5642986c5580 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5642986d3480 con 0x5642986d0570 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eb8a2f740 10 client.0 _ll_register_callbacks cb 0x56429833dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eaeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37984/0 (socket says 172.21.15.45:37984) 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eaeffd640 1 -- 172.21.15.45:0/247986060 learned_addr learned my addr 172.21.15.45:0/247986060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.189+0000 7f9eaeffd640 1 -- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 msgr2=0x5642986cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaffff640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaeffd640 1 -- 172.21.15.45:0/247986060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 msgr2=0x5642986cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaeffd640 1 -- 172.21.15.45:0/247986060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642985b7dc0 con 0x5642986c5ea0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ccb10 tx=0x7f9ea409b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc02386ffd3eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea409b880 con 0x5642986c5ea0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9ea40cf310 con 0x5642986c5ea0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00317b0 con 0x5642986c5ea0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ea40cf510 con 0x5642986c5ea0 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 10 client.16467 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56429833e3f0 con 0x5642986c5ea0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00317b0 con 0x5642986c5ea0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ea4097020 con 0x5642986c5ea0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 10 client.16467 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 10 client.16467 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 20 client.16467 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5642986e0540 0x5642986e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 4 client.16467 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.190+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564298637bb0 con 0x5642986e0540 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.191+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5642986e0540 0x5642986e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.191+0000 7f9e8effd640 10 client.16467 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.191+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5642986e0540 0x5642986e29d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2960e79f0c89feb4 server_cookie=70d3e4608b1f215f in_seq=0 out_seq=0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9e8effd640 1 -- 172.21.15.45:0/247986060 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564298637bb0 con 0x5642986e0540 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9e8effd640 10 client.16467 handle_command_reply: tid=0 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 client.16467 shutdown 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 msgr2=0x5642986cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ccb10 tx=0x7f9ea409b1e0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 shutdown_connections 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5642986e0540 0x5642986e29d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5ea0 0x5642986cd190 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9e8effd640 0 client.16467 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/247986060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 >> 172.21.15.45:0/247986060 conn(0x5642986c18a0 msgr2=0x5642986c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 shutdown_connections 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/247986060 wait complete. 2024-04-25T00:24:45.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.192+0000 7f9eb8a2f740 20 client.16467 trim_cache size 0 max 16384 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.205+0000 7f9eb8a2f740 1 Processor -- start 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.205+0000 7f9eb8a2f740 1 -- start start 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 0x5642986d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ae9d0 con 0x5642986c5580 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b0510 con 0x5642986c5ea0 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01af0a0 con 0x5642986d0570 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37990/0 (socket says 172.21.15.45:37990) 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/2695019389 learned_addr learned my addr 172.21.15.45:0/2695019389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 0x5642986d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaffff640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/2695019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 msgr2=0x5642986d8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/2695019389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 msgr2=0x5642986d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 0x5642986d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 -- 172.21.15.45:0/2695019389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642984087d0 con 0x5642986c5580 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 0x5642986d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00053e0 tx=0x7f9ea002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48196f29db713a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9e957fa640 1 -- 172.21.15.45:0/2695019389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0035050 con 0x5642986c5580 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9e957fa640 1 -- 172.21.15.45:0/2695019389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00317b0 con 0x5642986c5580 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.206+0000 7f9e957fa640 1 -- 172.21.15.45:0/2695019389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003f040 con 0x5642986c5580 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/2695019389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 msgr2=0x5642986cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00053e0 tx=0x7f9ea002a410 comp rx=0 tx=0).stop 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/2695019389 shutdown_connections 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986d0570 0x5642986d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5ea0 0x5642986d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/2695019389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986c5580 0x5642986cf9d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/2695019389 >> 172.21.15.45:0/2695019389 conn(0x5642986c18a0 msgr2=0x5642986c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/2695019389 shutdown_connections 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/2695019389 wait complete. 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.207+0000 7f9eb8a2f740 1 -- start start 2024-04-25T00:24:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 Processor -- start 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986c5ea0 0x5642986d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01afe40 con 0x5642986d0570 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01af770 con 0x5642986c5580 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01ac9b0 con 0x5642986c5ea0 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eb8a2f740 10 client.0 _ll_register_callbacks cb 0x7f9ea409b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38006/0 (socket says 172.21.15.45:38006) 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 -- 172.21.15.45:0/3179114655 learned_addr learned my addr 172.21.15.45:0/3179114655 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaeffd640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986c5ea0 0x5642986d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 -- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986c5ea0 msgr2=0x5642986d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986c5ea0 0x5642986d6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 -- 172.21.15.45:0/3179114655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 msgr2=0x5642986d6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986d6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 -- 172.21.15.45:0/3179114655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564298746060 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9eaffff640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8010cb0 tx=0x7f9ea8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff43d895819e7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.208+0000 7f9e8effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8020b20 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9ea8020cc0 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00317b0 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ea8004b20 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 10 client.16473 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564298747200 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00317b0 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ea8021410 con 0x5642986d0570 2024-04-25T00:24:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 10 client.16473 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 10 client.16473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 20 client.16473 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564298747980 0x564298749da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 4 client.16473 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.209+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564298631f00 con 0x564298747980 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564298747980 0x564298749da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9e8effd640 10 client.16473 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eaf7fe640 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564298747980 0x564298749da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc108d69952990dd server_cookie=b914ee7cb619b22c in_seq=0 out_seq=0 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9e8effd640 1 -- 172.21.15.45:0/3179114655 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564298631f00 con 0x564298747980 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9e8effd640 10 client.16473 handle_command_reply: tid=0 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 client.16473 shutdown 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 msgr2=0x564298745b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8010cb0 tx=0x7f9ea8005bd0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 shutdown_connections 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564298747980 0x564298749da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5642986d0570 0x564298745b20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9e8effd640 0 client.16473 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642986c5ea0 0x5642986d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 --2- 172.21.15.45:0/3179114655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642986c5580 0x5642986d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.210+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 >> 172.21.15.45:0/3179114655 conn(0x5642986c18a0 msgr2=0x5642986c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 shutdown_connections 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb8a2f740 1 -- 172.21.15.45:0/3179114655 wait complete. 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb8a2f740 20 client.16473 trim_cache size 0 max 16384 2024-04-25T00:24:45.217 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 msgr2=0x7f9e980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 0x7f9e980a51c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005cc0 tx=0x7f9ea800a8a0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 msgr2=0x7f9eb0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c970 tx=0x7f9ea400ce30 comp rx=0 tx=0).stop 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 shutdown_connections 2024-04-25T00:24:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9e980a2d00 0x7f9e980a51c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eb019c8c0 0x7f9eb019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb0106aa0 0x7f9eb0198590 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 --2- 172.21.15.45:0/697457222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9eb01002e0 0x7f9eb0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 >> 172.21.15.45:0/697457222 conn(0x7f9eb00758d0 msgr2=0x7f9eb00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 shutdown_connections 2024-04-25T00:24:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.211+0000 7f9eb5d1b640 1 -- 172.21.15.45:0/697457222 wait complete. 2024-04-25T00:24:45.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:24:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 -- 172.21.15.45:0/529744697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c107730 msgr2=0x7f709c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 --2- 172.21.15.45:0/529744697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c107730 0x7f709c107b10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f708c009930 tx=0x7f708c02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 -- 172.21.15.45:0/529744697 shutdown_connections 2024-04-25T00:24:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 --2- 172.21.15.45:0/529744697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c06a780 0x7f709c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 --2- 172.21.15.45:0/529744697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 --2- 172.21.15.45:0/529744697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c107730 0x7f709c107b10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 -- 172.21.15.45:0/529744697 >> 172.21.15.45:0/529744697 conn(0x7f709c0776a0 msgr2=0x7f709c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 -- 172.21.15.45:0/529744697 shutdown_connections 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.327+0000 7f70a146d640 1 -- 172.21.15.45:0/529744697 wait complete. 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 Processor -- start 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 -- start start 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c107730 0x7f709c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f709c111b40 con 0x7f709c06a780 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1119c0 con 0x7f709c069dc0 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f70a146d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f709c111cc0 con 0x7f709c107730 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38020/0 (socket says 172.21.15.45:38020) 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 -- 172.21.15.45:0/3655425050 learned_addr learned my addr 172.21.15.45:0/3655425050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709affd640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c107730 0x7f709c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 -- 172.21.15.45:0/3655425050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c107730 msgr2=0x7f709c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c107730 0x7f709c19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 -- 172.21.15.45:0/3655425050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 msgr2=0x7f709c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709a7fc640 1 -- 172.21.15.45:0/3655425050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c19cff0 con 0x7f709c06a780 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.328+0000 7f709affd640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f709000e9f0 tx=0x7f709000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f7083fff640 1 -- 172.21.15.45:0/3655425050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709000cd80 con 0x7f709c06a780 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c19d2e0 con 0x7f709c06a780 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f7083fff640 1 -- 172.21.15.45:0/3655425050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7090010910 con 0x7f709c06a780 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f7083fff640 1 -- 172.21.15.45:0/3655425050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70900188f0 con 0x7f709c06a780 2024-04-25T00:24:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.329+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c1a4b70 con 0x7f709c06a780 2024-04-25T00:24:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.331+0000 7f7083fff640 1 -- 172.21.15.45:0/3655425050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7090010ac0 con 0x7f709c06a780 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.331+0000 7f70a4177740 1 Processor -- start 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 -- start start 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dc2b0 0x560a6a8dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8edd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a6a841670 con 0x560a6a8dcc60 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a6a7e9970 con 0x560a6a8dc2b0 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f70a4177740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a6a83d630 con 0x560a6a8eb980 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38028/0 (socket says 172.21.15.45:38028) 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 -- 172.21.15.45:0/426442874 learned_addr learned my addr 172.21.15.45:0/426442874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709b7fe640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dc2b0 0x560a6a8dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709a7fc640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f7083fff640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 0x7f70840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 -- 172.21.15.45:0/426442874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 msgr2=0x560a6a8edd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8edd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 -- 172.21.15.45:0/426442874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dc2b0 msgr2=0x560a6a8dc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dc2b0 0x560a6a8dc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 -- 172.21.15.45:0/426442874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6a80d400 con 0x560a6a8dcc60 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709a7fc640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8edd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 0x7f70840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f7083fff640 1 -- 172.21.15.45:0/3655425050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f70900cfd50 con 0x7f709c06a780 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.332+0000 7f709affd640 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f708c006010 tx=0x7f708c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88835ba92de2516c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70817fa640 1 -- 172.21.15.45:0/426442874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c03f070 con 0x560a6a8dcc60 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70817fa640 1 -- 172.21.15.45:0/426442874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708c004030 con 0x560a6a8dcc60 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70817fa640 1 -- 172.21.15.45:0/426442874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c035070 con 0x560a6a8dcc60 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 -- 172.21.15.45:0/426442874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 msgr2=0x560a6a8eb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 0x7f70840a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7094005fd0 tx=0x7f7094004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f708c006010 tx=0x7f708c009680 comp rx=0 tx=0).stop 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 -- 172.21.15.45:0/426442874 shutdown_connections 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8edd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dcc60 0x560a6a8eb440 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 --2- 172.21.15.45:0/426442874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dc2b0 0x560a6a8dc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 -- 172.21.15.45:0/426442874 >> 172.21.15.45:0/426442874 conn(0x560a6a8d88a0 msgr2=0x560a6a8d8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 -- 172.21.15.45:0/426442874 shutdown_connections 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.333+0000 7f70a4177740 1 -- 172.21.15.45:0/426442874 wait complete. 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 -- start start 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 Processor -- start 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 0x560a6a8e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 0x560a6a8e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560a6a8ee6e0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560a6a7fd050 con 0x560a6a8eb980 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560a6a8ee860 con 0x560a6a8dcc60 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f70a4177740 10 client.0 _ll_register_callbacks cb 0x560a6a554be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38044/0 (socket says 172.21.15.45:38044) 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 -- 172.21.15.45:0/3115871291 learned_addr learned my addr 172.21.15.45:0/3115871291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 0x560a6a8e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 -- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 msgr2=0x560a6a8e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 0x560a6a8e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 0x560a6a8e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 -- 172.21.15.45:0/3115871291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 msgr2=0x560a6a8e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 0x560a6a8e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709affd640 1 -- 172.21.15.45:0/3115871291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6a7cedc0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 0x560a6a8e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.334+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 0x560a6a8e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f709affd640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f708c02a940 tx=0x7f708c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49f8f4a3c74ff71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c04b020 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f708c0315b0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708c004030 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f708c056050 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 10 client.16485 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a6a5553f0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708c004030 con 0x560a6a8dc2b0 2024-04-25T00:24:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f708c03b460 con 0x560a6a8dc2b0 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 10 client.16485 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 10 client.16485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 20 client.16485 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a8fb790 0x560a6a8fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 4 client.16485 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.335+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6a84ebb0 con 0x560a6a8fb790 2024-04-25T00:24:45.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.336+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a8fb790 0x560a6a8fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.336+0000 7f707eff5640 10 client.16485 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.336+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a8fb790 0x560a6a8fdb80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=866494dfd143bc80 server_cookie=c70b7f7c6b92ec26 in_seq=0 out_seq=0 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f707eff5640 1 -- 172.21.15.45:0/3115871291 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a6a84ebb0 con 0x560a6a8fb790 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f707eff5640 10 client.16485 handle_command_reply: tid=0 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 client.16485 shutdown 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 msgr2=0x560a6a8eb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f708c02a940 tx=0x7f708c009260 comp rx=0 tx=0).stop 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 shutdown_connections 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a8fb790 0x560a6a8fdb80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8eb980 0x560a6a8e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8dcc60 0x560a6a8e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 --2- 172.21.15.45:0/3115871291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8eb290 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f707eff5640 0 client.16485 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 >> 172.21.15.45:0/3115871291 conn(0x560a6a8d88a0 msgr2=0x560a6a8de6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 shutdown_connections 2024-04-25T00:24:45.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 1 -- 172.21.15.45:0/3115871291 wait complete. 2024-04-25T00:24:45.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.337+0000 7f70a4177740 20 client.16485 trim_cache size 0 max 16384 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.350+0000 7f70a4177740 1 Processor -- start 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.350+0000 7f70a4177740 1 -- start start 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1ac930 con 0x560a6a8dc2b0 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1ae470 con 0x560a6a8dcc60 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f70a4177740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1ad000 con 0x560a6a8eb980 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38050/0 (socket says 172.21.15.45:38050) 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 -- 172.21.15.45:0/514660564 learned_addr learned my addr 172.21.15.45:0/514660564 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709b7fe640 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709a7fc640 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 -- 172.21.15.45:0/514660564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 msgr2=0x560a6a8eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 -- 172.21.15.45:0/514660564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 msgr2=0x560a6a8e7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 -- 172.21.15.45:0/514660564 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6a61f7d0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f709affd640 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f708c0057c0 tx=0x7f708c032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849019208e85386a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f7080ff9640 1 -- 172.21.15.45:0/514660564 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c05f070 con 0x560a6a8dc2b0 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f7080ff9640 1 -- 172.21.15.45:0/514660564 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708c03be50 con 0x560a6a8dc2b0 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.351+0000 7f7080ff9640 1 -- 172.21.15.45:0/514660564 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c04b040 con 0x560a6a8dc2b0 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- 172.21.15.45:0/514660564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 msgr2=0x560a6a8f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f708c0057c0 tx=0x7f708c032fc0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- 172.21.15.45:0/514660564 shutdown_connections 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 --2- 172.21.15.45:0/514660564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f7640 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- 172.21.15.45:0/514660564 >> 172.21.15.45:0/514660564 conn(0x560a6a8d88a0 msgr2=0x560a6a8e6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- 172.21.15.45:0/514660564 shutdown_connections 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- 172.21.15.45:0/514660564 wait complete. 2024-04-25T00:24:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 -- start start 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.352+0000 7f70a4177740 1 Processor -- start 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1adda0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1ad6d0 con 0x560a6a8dcc60 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1aa910 con 0x560a6a8eb980 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 10 client.0 _ll_register_callbacks cb 0x7f7094009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38060/0 (socket says 172.21.15.45:38060) 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 -- 172.21.15.45:0/3707081402 learned_addr learned my addr 172.21.15.45:0/3707081402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 -- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 msgr2=0x560a6a8e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 -- 172.21.15.45:0/3707081402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 msgr2=0x560a6a8e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 -- 172.21.15.45:0/3707081402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6a8e5970 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709b7fe640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f709affd640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f708c03b8b0 tx=0x7f708c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870d3abecee4cca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f707eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c05f070 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708c03be50 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.353+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f708c0317b0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f708c06c050 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 10 client.16491 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a6a9618f0 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708c03be50 con 0x560a6a8dc2b0 2024-04-25T00:24:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f708c05a430 con 0x560a6a8dc2b0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 10 client.16491 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 20 client.16491 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a961fe0 0x560a6a964400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 4 client.16491 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a6a8e8e50 con 0x560a6a961fe0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.354+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a961fe0 0x560a6a964400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f709a7fc640 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a961fe0 0x560a6a964400 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=331c015c9acb71f6 server_cookie=3e62d59f1be321e6 in_seq=0 out_seq=0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f707eff5640 10 client.16491 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f707eff5640 1 -- 172.21.15.45:0/3707081402 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a6a8e8e50 con 0x560a6a961fe0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f707eff5640 10 client.16491 handle_command_reply: tid=0 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 client.16491 shutdown 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 msgr2=0x560a6a8f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f708c03b8b0 tx=0x7f708c009260 comp rx=0 tx=0).stop 2024-04-25T00:24:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 shutdown_connections 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x560a6a961fe0 0x560a6a964400 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560a6a8eb980 0x560a6a8e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f707eff5640 0 client.16491 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a6a8dcc60 0x560a6a8e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 --2- 172.21.15.45:0/3707081402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560a6a8dc2b0 0x560a6a8f3d40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 >> 172.21.15.45:0/3707081402 conn(0x560a6a8d88a0 msgr2=0x560a6a8dde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 shutdown_connections 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 1 -- 172.21.15.45:0/3707081402 wait complete. 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.355+0000 7f70a4177740 20 client.16491 trim_cache size 0 max 16384 2024-04-25T00:24:45.362 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 msgr2=0x7f70840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 0x7f70840a51c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7094005fd0 tx=0x7f7094004a40 comp rx=0 tx=0).stop 2024-04-25T00:24:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 msgr2=0x7f709c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f709000e9f0 tx=0x7f709000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 shutdown_connections 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f70840a2d00 0x7f70840a51c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f709c107730 0x7f709c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f709c06a780 0x7f709c198540 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 --2- 172.21.15.45:0/3655425050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f709c069dc0 0x7f709c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 >> 172.21.15.45:0/3655425050 conn(0x7f709c0776a0 msgr2=0x7f709c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 shutdown_connections 2024-04-25T00:24:45.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.356+0000 7f70a146d640 1 -- 172.21.15.45:0/3655425050 wait complete. 2024-04-25T00:24:45.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 -- 172.21.15.45:0/3212558053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934106630 msgr2=0x7fd934106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 --2- 172.21.15.45:0/3212558053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934106630 0x7fd934106a10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd924009930 tx=0x7fd92402a420 comp rx=0 tx=0).stop 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 -- 172.21.15.45:0/3212558053 shutdown_connections 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 --2- 172.21.15.45:0/3212558053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100fe0 0x7fd93410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 --2- 172.21.15.45:0/3212558053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934100620 0x7fd934100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 --2- 172.21.15.45:0/3212558053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934106630 0x7fd934106a10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.471+0000 7fd938ec1640 1 -- 172.21.15.45:0/3212558053 >> 172.21.15.45:0/3212558053 conn(0x7fd9340776a0 msgr2=0x7fd934077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.472+0000 7fd938ec1640 1 -- 172.21.15.45:0/3212558053 shutdown_connections 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.472+0000 7fd938ec1640 1 -- 172.21.15.45:0/3212558053 wait complete. 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.472+0000 7fd938ec1640 1 Processor -- start 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.472+0000 7fd938ec1640 1 -- start start 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100620 0x7fd934198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934106630 0x7fd93419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd934110a40 con 0x7fd934100fe0 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341108c0 con 0x7fd934100620 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd934110bc0 con 0x7fd934106630 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38072/0 (socket says 172.21.15.45:38072) 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 -- 172.21.15.45:0/801513482 learned_addr learned my addr 172.21.15.45:0/801513482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd93259c640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100620 0x7fd934198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd932d9d640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934106630 0x7fd93419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 -- 172.21.15.45:0/801513482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934106630 msgr2=0x7fd93419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934106630 0x7fd93419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 -- 172.21.15.45:0/801513482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100620 msgr2=0x7fd934198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100620 0x7fd934198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 -- 172.21.15.45:0/801513482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419cff0 con 0x7fd934100fe0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd929d9b640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00d770 tx=0x7fd91c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd92bfff640 1 -- 172.21.15.45:0/801513482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c017070 con 0x7fd934100fe0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419d2e0 con 0x7fd934100fe0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd92bfff640 1 -- 172.21.15.45:0/801513482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd91c005500 con 0x7fd934100fe0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.473+0000 7fd92bfff640 1 -- 172.21.15.45:0/801513482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004e00 con 0x7fd934100fe0 2024-04-25T00:24:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.474+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a4b70 con 0x7fd934100fe0 2024-04-25T00:24:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.476+0000 7fd92bfff640 1 -- 172.21.15.45:0/801513482 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd91c004fa0 con 0x7fd934100fe0 2024-04-25T00:24:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.476+0000 7fd93bbcd740 1 Processor -- start 2024-04-25T00:24:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.476+0000 7fd93bbcd740 1 -- start start 2024-04-25T00:24:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a39960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a44030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fed599e670 con 0x55fed5a44570 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fed5946970 con 0x55fed5a39580 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fed599a630 con 0x55fed5a39ea0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd92bfff640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 0x7fd9200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd932d9d640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 0x7fd9200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38078/0 (socket says 172.21.15.45:38078) 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 -- 172.21.15.45:0/364365047 learned_addr learned my addr 172.21.15.45:0/364365047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd932d9d640 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a39960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd93259c640 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a44030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd92bfff640 1 -- 172.21.15.45:0/801513482 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd91c0cf290 con 0x7fd934100fe0 2024-04-25T00:24:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 -- 172.21.15.45:0/364365047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 msgr2=0x55fed5a44030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a44030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 -- 172.21.15.45:0/364365047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 msgr2=0x55fed5a39960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a39960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 -- 172.21.15.45:0/364365047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed596a400 con 0x55fed5a44570 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd932d9d640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 0x7fd9200a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009270 tx=0x7fd92c009db0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.477+0000 7fd929d9b640 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0cc680 tx=0x7fd91c06e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e17a21acae1ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd928d99640 1 -- 172.21.15.45:0/364365047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c020040 con 0x55fed5a44570 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd928d99640 1 -- 172.21.15.45:0/364365047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd91c0046c0 con 0x55fed5a44570 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd928d99640 1 -- 172.21.15.45:0/364365047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0a4060 con 0x55fed5a44570 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 -- 172.21.15.45:0/364365047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 msgr2=0x55fed5a46960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0cc680 tx=0x7fd91c06e460 comp rx=0 tx=0).stop 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 -- 172.21.15.45:0/364365047 shutdown_connections 2024-04-25T00:24:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5a46960 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a44030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/364365047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a39960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 -- 172.21.15.45:0/364365047 >> 172.21.15.45:0/364365047 conn(0x55fed5a358a0 msgr2=0x55fed5a35c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 -- 172.21.15.45:0/364365047 shutdown_connections 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.478+0000 7fd93bbcd740 1 -- 172.21.15.45:0/364365047 wait complete. 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 -- start start 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 Processor -- start 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a43f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 0x55fed5a41770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fed5a47360 con 0x55fed5a39ea0 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fed595a050 con 0x55fed5a39580 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fed5a474e0 con 0x55fed5a44570 2024-04-25T00:24:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93bbcd740 10 client.0 _ll_register_callbacks cb 0x55fed56b1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38080/0 (socket says 172.21.15.45:38080) 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 -- 172.21.15.45:0/480677250 learned_addr learned my addr 172.21.15.45:0/480677250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd93259c640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a43f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd932d9d640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 0x55fed5a41770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 -- 172.21.15.45:0/480677250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 msgr2=0x55fed5a41770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 0x55fed5a41770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 -- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 msgr2=0x55fed5a43f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a43f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd929d9b640 1 -- 172.21.15.45:0/480677250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed592bdc0 con 0x55fed5a39ea0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.479+0000 7fd932d9d640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 0x55fed5a41770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd929d9b640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd91c09b0f0 tx=0x7fd91c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c70a4b5dcd91d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c020070 con 0x55fed5a39ea0 2024-04-25T00:24:45.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd91c009cc0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fed59abbb0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd91c09f7a0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 10 client.16503 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fed56b23f0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fed59abbb0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd91c0cc8a0 con 0x55fed5a39ea0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 10 client.16503 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 20 client.16503 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5a54470 0x55fed5a56860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 4 client.16503 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed59a5f00 con 0x55fed5a54470 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.480+0000 7fd93259c640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5a54470 0x55fed5a56860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.481+0000 7fd93259c640 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5a54470 0x55fed5a56860 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e94cd8e7c33b7b6 server_cookie=21d9942d9fb8c3ac in_seq=0 out_seq=0 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.481+0000 7fd919ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd919ffb640 1 -- 172.21.15.45:0/480677250 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fed59a5f00 con 0x55fed5a54470 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd919ffb640 10 client.16503 handle_command_reply: tid=0 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 client.16503 shutdown 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 msgr2=0x55fed5a411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd91c09b0f0 tx=0x7fd91c009c30 comp rx=0 tx=0).stop 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 shutdown_connections 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a44570 0x55fed5a41770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5a54470 0x55fed5a56860 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39ea0 0x55fed5a411f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/480677250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5a43f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 >> 172.21.15.45:0/480677250 conn(0x55fed5a358a0 msgr2=0x55fed5a34b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd919ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 shutdown_connections 2024-04-25T00:24:45.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 1 -- 172.21.15.45:0/480677250 wait complete. 2024-04-25T00:24:45.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.482+0000 7fd93bbcd740 20 client.16503 trim_cache size 0 max 16384 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 Processor -- start 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 -- start start 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a43a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 0x55fed5a5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341aeaf0 con 0x55fed5a39580 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341b0630 con 0x55fed5a44570 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341af1c0 con 0x55fed5a39ea0 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38086/0 (socket says 172.21.15.45:38086) 2024-04-25T00:24:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd93259c640 1 -- 172.21.15.45:0/3821657660 learned_addr learned my addr 172.21.15.45:0/3821657660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.495+0000 7fd932d9d640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 0x55fed5a5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd929d9b640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a43a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 -- 172.21.15.45:0/3821657660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 msgr2=0x55fed5a43a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a43a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 -- 172.21.15.45:0/3821657660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 msgr2=0x55fed5a5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 0x55fed5a5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 -- 172.21.15.45:0/3821657660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed577c7d0 con 0x55fed5a39580 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd929d9b640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a43a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd932d9d640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 0x55fed5a5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93259c640 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd924009900 tx=0x7fd924031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17588ded9039e8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd91bfff640 1 -- 172.21.15.45:0/3821657660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924035040 con 0x55fed5a39580 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd91bfff640 1 -- 172.21.15.45:0/3821657660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd924031a70 con 0x55fed5a39580 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd91bfff640 1 -- 172.21.15.45:0/3821657660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924005580 con 0x55fed5a39580 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93bbcd740 1 -- 172.21.15.45:0/3821657660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 msgr2=0x55fed5a50220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.496+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd924009900 tx=0x7fd924031a00 comp rx=0 tx=0).stop 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- 172.21.15.45:0/3821657660 shutdown_connections 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a44570 0x55fed5a5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5a43a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/3821657660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a39580 0x55fed5a50220 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- 172.21.15.45:0/3821657660 >> 172.21.15.45:0/3821657660 conn(0x55fed5a358a0 msgr2=0x55fed5a342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- 172.21.15.45:0/3821657660 shutdown_connections 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- 172.21.15.45:0/3821657660 wait complete. 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- start start 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 Processor -- start 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5ab6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5ab7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341aff60 con 0x55fed5a44570 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341af890 con 0x55fed5a39580 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341acad0 con 0x55fed5a39ea0 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93bbcd740 10 client.0 _ll_register_callbacks cb 0x7fd92c0126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd932d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd932d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38102/0 (socket says 172.21.15.45:38102) 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd932d9d640 1 -- 172.21.15.45:0/1032929891 learned_addr learned my addr 172.21.15.45:0/1032929891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd929d9b640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5ab7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.497+0000 7fd93259c640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5ab6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 -- 172.21.15.45:0/1032929891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 msgr2=0x55fed5ab7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5ab7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 -- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 msgr2=0x55fed5ab6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5ab6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 -- 172.21.15.45:0/1032929891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed5ab1ae0 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd932d9d640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd92c010d50 tx=0x7fd92c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27669d58e658c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c02f060 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd92c01b040 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd924031a70 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd92c036050 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd93bbcd740 10 client.16509 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fed5abe750 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.498+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd924031a70 con 0x55fed5a44570 2024-04-25T00:24:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd92c01f050 con 0x55fed5a44570 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 10 client.16509 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 20 client.16509 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5abef00 0x55fed5ac1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 4 client.16509 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fed5923820 con 0x55fed5abef00 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93259c640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5abef00 0x55fed5ac1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93259c640 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5abef00 0x55fed5ac1320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5032214f558aa857 server_cookie=8259b9a397f0a503 in_seq=0 out_seq=0 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd919ffb640 10 client.16509 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd919ffb640 1 -- 172.21.15.45:0/1032929891 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fed5923820 con 0x55fed5abef00 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd919ffb640 10 client.16509 handle_command_reply: tid=0 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 1 client.16509 shutdown 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 msgr2=0x55fed5ab13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.499+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd92c010d50 tx=0x7fd92c01c000 comp rx=0 tx=0).stop 2024-04-25T00:24:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 shutdown_connections 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fed5abef00 0x55fed5ac1320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fed5a44570 0x55fed5ab13f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fed5a39ea0 0x55fed5ab7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 --2- 172.21.15.45:0/1032929891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fed5a39580 0x55fed5ab6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd919ffb640 0 client.16509 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 >> 172.21.15.45:0/1032929891 conn(0x55fed5a358a0 msgr2=0x55fed5a34b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 shutdown_connections 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 1 -- 172.21.15.45:0/1032929891 wait complete. 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd93bbcd740 20 client.16509 trim_cache size 0 max 16384 2024-04-25T00:24:45.510 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 msgr2=0x7fd9200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 0x7fd9200a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009270 tx=0x7fd92c009db0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 msgr2=0x7fd934198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00d770 tx=0x7fd91c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 shutdown_connections 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9200a2d00 0x7fd9200a51c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd934106630 0x7fd93419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd934100fe0 0x7fd934198590 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 --2- 172.21.15.45:0/801513482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd934100620 0x7fd934198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 >> 172.21.15.45:0/801513482 conn(0x7fd9340776a0 msgr2=0x7fd9340ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 shutdown_connections 2024-04-25T00:24:45.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:45.500+0000 7fd938ec1640 1 -- 172.21.15.45:0/801513482 wait complete. 2024-04-25T00:24:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:45 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2243674811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:45.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:45 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2243674811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:45.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:45 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2243674811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:46.513 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:24:46.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:24:46.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:24:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- 172.21.15.45:0/4261331611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 msgr2=0x7f6dfc10b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 --2- 172.21.15.45:0/4261331611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc10b7c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009900 tx=0x7f6df002a510 comp rx=0 tx=0).stop 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- 172.21.15.45:0/4261331611 shutdown_connections 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 --2- 172.21.15.45:0/4261331611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 --2- 172.21.15.45:0/4261331611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc10b7c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 --2- 172.21.15.45:0/4261331611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 0x7f6dfc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- 172.21.15.45:0/4261331611 >> 172.21.15.45:0/4261331611 conn(0x7f6dfc077740 msgr2=0x7f6dfc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- 172.21.15.45:0/4261331611 shutdown_connections 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- 172.21.15.45:0/4261331611 wait complete. 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 Processor -- start 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.634+0000 7f6e02510640 1 -- start start 2024-04-25T00:24:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 0x7f6dfc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111b00 con 0x7f6dfc1032e0 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111980 con 0x7f6dfc10bd00 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e02510640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111c80 con 0x7f6dfc1029c0 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38106/0 (socket says 172.21.15.45:38106) 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 -- 172.21.15.45:0/1380228612 learned_addr learned my addr 172.21.15.45:0/1380228612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 0x7f6dfc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 -- 172.21.15.45:0/1380228612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 msgr2=0x7f6dfc198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 0x7f6dfc198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 -- 172.21.15.45:0/1380228612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 msgr2=0x7f6dfc19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 -- 172.21.15.45:0/1380228612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc19d030 con 0x7f6dfc1032e0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.635+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006010 tx=0x7f6df00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.636+0000 7f6df97fa640 1 -- 172.21.15.45:0/1380228612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df003f030 con 0x7f6dfc1032e0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.636+0000 7f6df97fa640 1 -- 172.21.15.45:0/1380228612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0003ce0 con 0x7f6dfc1032e0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.636+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc19d2c0 con 0x7f6dfc1032e0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.636+0000 7f6df97fa640 1 -- 172.21.15.45:0/1380228612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0035070 con 0x7f6dfc1032e0 2024-04-25T00:24:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.636+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc1a4bb0 con 0x7f6dfc1032e0 2024-04-25T00:24:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.638+0000 7f6df97fa640 1 -- 172.21.15.45:0/1380228612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6df0042030 con 0x7f6dfc1032e0 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.638+0000 7f6e0521e740 1 Processor -- start 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.638+0000 7f6e0521e740 1 -- start start 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cbb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564148c1b2c0 con 0x564148cbb6a0 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564148bc3910 con 0x564148cbcd90 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e0521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564148c17280 con 0x564148cc6420 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6df97fa640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 0x7f6de40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38108/0 (socket says 172.21.15.45:38108) 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2006348361 learned_addr learned my addr 172.21.15.45:0/2006348361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 0x7f6de40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e00aad640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cbb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2006348361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 msgr2=0x564148cc8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6df97fa640 1 -- 172.21.15.45:0/1380228612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6df00efe20 con 0x7f6dfc1032e0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2006348361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 msgr2=0x564148cbb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cbb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2006348361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564148bd4fb0 con 0x564148cbb6a0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009c90 tx=0x7f6dec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d87d152c97896c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.639+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 0x7f6de40a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005c40 tx=0x7f6df4009530 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6de27fc640 1 -- 172.21.15.45:0/2006348361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec012450 con 0x564148cbb6a0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6de27fc640 1 -- 172.21.15.45:0/2006348361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec012ad0 con 0x564148cbb6a0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6de27fc640 1 -- 172.21.15.45:0/2006348361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00fbf0 con 0x564148cbb6a0 2024-04-25T00:24:46.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 -- 172.21.15.45:0/2006348361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 msgr2=0x564148cbbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009c90 tx=0x7f6dec0117b0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 -- 172.21.15.45:0/2006348361 shutdown_connections 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cbbb20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2006348361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cbb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 -- 172.21.15.45:0/2006348361 >> 172.21.15.45:0/2006348361 conn(0x564148c7f950 msgr2=0x564148cb7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 -- 172.21.15.45:0/2006348361 shutdown_connections 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.640+0000 7f6e0521e740 1 -- 172.21.15.45:0/2006348361 wait complete. 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 -- start start 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 Processor -- start 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564148cc92b0 con 0x564148cbb6a0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564148bd6e90 con 0x564148cbcd90 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564148cc9430 con 0x564148cc6420 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6e0521e740 10 client.0 _ll_register_callbacks cb 0x564148c12970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38110/0 (socket says 172.21.15.45:38110) 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.641+0000 7f6dfbfff640 1 -- 172.21.15.45:0/510395003 learned_addr learned my addr 172.21.15.45:0/510395003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6e00aad640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 -- 172.21.15.45:0/510395003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 msgr2=0x564148cc3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 -- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 msgr2=0x564148cc37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 -- 172.21.15.45:0/510395003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641489f77e0 con 0x564148cbb6a0 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6e00aad640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00fb90 tx=0x7f6dec00fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbbd2ac691becd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec029030 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6dec020040 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dec012ad0 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dec033050 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6e0521e740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564148a57f70 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec012ad0 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.642+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dec0046c0 con 0x564148cbb6a0 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 20 client.16521 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148cd63f0 0x564148cd87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 4 client.16521 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564148bef5a0 con 0x564148cd63f0 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148cd63f0 0x564148cd87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6ddb7fe640 10 client.16521 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.643+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148cd63f0 0x564148cd87e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a92cb296c9fdbb server_cookie=ffaff3a1728be997 in_seq=0 out_seq=0 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/510395003 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564148bef5a0 con 0x564148cd63f0 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6ddb7fe640 10 client.16521 handle_command_reply: tid=0 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 client.16521 shutdown 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 msgr2=0x564148cc3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00fb90 tx=0x7f6dec00fbc0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 shutdown_connections 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148cd63f0 0x564148cd87e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 --2- 172.21.15.45:0/510395003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cc3280 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6ddb7fe640 0 client.16521 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.644+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 >> 172.21.15.45:0/510395003 conn(0x564148c7f950 msgr2=0x564148cbda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.645+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 shutdown_connections 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.645+0000 7f6e0521e740 1 -- 172.21.15.45:0/510395003 wait complete. 2024-04-25T00:24:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.645+0000 7f6e0521e740 20 client.16521 trim_cache size 0 max 16384 2024-04-25T00:24:46.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 Processor -- start 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 -- start start 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cb27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1ac980 con 0x564148cbb6a0 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1ae4c0 con 0x564148cbcd90 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.662+0000 7f6e0521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1ad050 con 0x564148cc6420 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38120/0 (socket says 172.21.15.45:38120) 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2018285167 learned_addr learned my addr 172.21.15.45:0/2018285167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6e00aad640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cb27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2018285167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 msgr2=0x564148cb27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cb27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2018285167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 msgr2=0x564148cc2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 -- 172.21.15.45:0/2018285167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641488ef170 con 0x564148cbb6a0 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6e00aad640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cb27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6dec004cd0 tx=0x7f6dec009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883bc065f4bf7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6de1ffb640 1 -- 172.21.15.45:0/2018285167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec03c070 con 0x564148cbb6a0 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6de1ffb640 1 -- 172.21.15.45:0/2018285167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec012dd0 con 0x564148cbb6a0 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.663+0000 7f6de1ffb640 1 -- 172.21.15.45:0/2018285167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec029070 con 0x564148cbb6a0 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- 172.21.15.45:0/2018285167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 msgr2=0x564148cd21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6dec004cd0 tx=0x7f6dec009280 comp rx=0 tx=0).stop 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- 172.21.15.45:0/2018285167 shutdown_connections 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cc6420 0x564148cb27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 --2- 172.21.15.45:0/2018285167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cbb6a0 0x564148cd21b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- 172.21.15.45:0/2018285167 >> 172.21.15.45:0/2018285167 conn(0x564148c7f950 msgr2=0x564148cb20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- 172.21.15.45:0/2018285167 shutdown_connections 2024-04-25T00:24:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- 172.21.15.45:0/2018285167 wait complete. 2024-04-25T00:24:46.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 -- start start 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.664+0000 7f6e0521e740 1 Processor -- start 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cbb6a0 0x564148cc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1addf0 con 0x564148cc6420 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1ad720 con 0x564148cbcd90 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1aa960 con 0x564148cbb6a0 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e0521e740 10 client.0 _ll_register_callbacks cb 0x7f6dec009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38130/0 (socket says 172.21.15.45:38130) 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 -- 172.21.15.45:0/1199179803 learned_addr learned my addr 172.21.15.45:0/1199179803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6dfb7fe640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cbb6a0 0x564148cc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 -- 172.21.15.45:0/1199179803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cbb6a0 msgr2=0x564148cc0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cbb6a0 0x564148cc0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 -- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 msgr2=0x564148cc0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 -- 172.21.15.45:0/1199179803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564148d3bc40 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6e00aad640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6df4012fd0 tx=0x7f6df4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec241ce6bd56b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6ddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.665+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df402d060 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6df401db50 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dec012dd0 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6df4034050 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 10 client.16527 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564148d3cdf0 con 0x564148cc6420 2024-04-25T00:24:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dec012dd0 con 0x564148cc6420 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6df4027070 con 0x564148cc6420 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 10 client.16527 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 20 client.16527 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148d3d570 0x564148d3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 4 client.16527 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564148c28800 con 0x564148d3d570 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.666+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148d3d570 0x564148d3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.667+0000 7f6dfbfff640 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148d3d570 0x564148d3f990 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665b8c9bef6c2a55 server_cookie=416d9a1f7f8df551 in_seq=0 out_seq=0 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.667+0000 7f6ddb7fe640 10 client.16527 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/1199179803 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564148c28800 con 0x564148d3d570 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6ddb7fe640 10 client.16527 handle_command_reply: tid=0 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 client.16527 shutdown 2024-04-25T00:24:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 msgr2=0x564148d3b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6df4012fd0 tx=0x7f6df4004a40 comp rx=0 tx=0).stop 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 shutdown_connections 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564148d3d570 0x564148d3f990 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6ddb7fe640 0 client.16527 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564148cc6420 0x564148d3b700 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564148cbcd90 0x564148cc0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 --2- 172.21.15.45:0/1199179803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564148cbb6a0 0x564148cc0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 >> 172.21.15.45:0/1199179803 conn(0x564148c7f950 msgr2=0x564148cc3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 shutdown_connections 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 1 -- 172.21.15.45:0/1199179803 wait complete. 2024-04-25T00:24:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.668+0000 7f6e0521e740 20 client.16527 trim_cache size 0 max 16384 2024-04-25T00:24:46.675 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:46.675 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:24:46.675 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "6df21c77-fb0c-4808-8afd-fed306d89abd", 2024-04-25T00:24:46.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:24:46.675 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 msgr2=0x7f6de40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 0x7f6de40a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005c40 tx=0x7f6df4009530 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 msgr2=0x7f6dfc198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006010 tx=0x7f6df00042f0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 shutdown_connections 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6de40a2d00 0x7f6de40a51c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dfc10bd00 0x7f6dfc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dfc1032e0 0x7f6dfc198580 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 --2- 172.21.15.45:0/1380228612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dfc1029c0 0x7f6dfc198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 >> 172.21.15.45:0/1380228612 conn(0x7f6dfc077740 msgr2=0x7f6dfc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 shutdown_connections 2024-04-25T00:24:46.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.669+0000 7f6e02510640 1 -- 172.21.15.45:0/1380228612 wait complete. 2024-04-25T00:24:46.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:46.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 -- 172.21.15.45:0/3581207343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b801020d0 msgr2=0x7f5b8010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 --2- 172.21.15.45:0/3581207343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b801020d0 0x7f5b8010e5e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009cd0 tx=0x7f5b7802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 -- 172.21.15.45:0/3581207343 shutdown_connections 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 --2- 172.21.15.45:0/3581207343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b801020d0 0x7f5b8010e5e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 --2- 172.21.15.45:0/3581207343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b80101710 0x7f5b80101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 --2- 172.21.15.45:0/3581207343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 0x7f5b80107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 -- 172.21.15.45:0/3581207343 >> 172.21.15.45:0/3581207343 conn(0x7f5b800fd180 msgr2=0x7f5b800ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 -- 172.21.15.45:0/3581207343 shutdown_connections 2024-04-25T00:24:46.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.785+0000 7f5b8869e640 1 -- 172.21.15.45:0/3581207343 wait complete. 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 Processor -- start 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 -- start start 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b801020d0 0x7f5b80198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 0x7f5b8019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80111b30 con 0x7f5b80101710 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801119b0 con 0x7f5b80107720 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8869e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80111cb0 con 0x7f5b801020d0 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38148/0 (socket says 172.21.15.45:38148) 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 -- 172.21.15.45:0/2550072419 learned_addr learned my addr 172.21.15.45:0/2550072419 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 0x7f5b8019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b85c39640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b801020d0 0x7f5b80198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 -- 172.21.15.45:0/2550072419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b801020d0 msgr2=0x7f5b80198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b801020d0 0x7f5b80198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 -- 172.21.15.45:0/2550072419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 msgr2=0x7f5b8019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 0x7f5b8019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.786+0000 7f5b8643a640 1 -- 172.21.15.45:0/2550072419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8019d000 con 0x7f5b80101710 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00e9f0 tx=0x7f5b7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b777fe640 1 -- 172.21.15.45:0/2550072419 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00cd80 con 0x7f5b80101710 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b777fe640 1 -- 172.21.15.45:0/2550072419 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c010910 con 0x7f5b80101710 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8019d2f0 con 0x7f5b80101710 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b777fe640 1 -- 172.21.15.45:0/2550072419 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c018990 con 0x7f5b80101710 2024-04-25T00:24:46.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.787+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801a4b80 con 0x7f5b80101710 2024-04-25T00:24:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.789+0000 7f5b777fe640 1 -- 172.21.15.45:0/2550072419 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b7c010ac0 con 0x7f5b80101710 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.789+0000 7f5b8b3ab740 1 Processor -- start 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.789+0000 7f5b8b3ab740 1 -- start start 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.789+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 0x55e74ff32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.789+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff3fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e74fe976b0 con 0x55e74ff32f50 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e74fe3f880 con 0x55e74ff3d620 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e74fe93670 con 0x55e74ff32630 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b777fe640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 0x7f5b6c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38158/0 (socket says 172.21.15.45:38158) 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 -- 172.21.15.45:0/2014440903 learned_addr learned my addr 172.21.15.45:0/2014440903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 0x55e74ff32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 0x7f5b6c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b85c39640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff3fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b777fe640 1 -- 172.21.15.45:0/2550072419 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5b7c0cfaf0 con 0x7f5b80101710 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 -- 172.21.15.45:0/2014440903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 msgr2=0x55e74ff32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 0x55e74ff32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 -- 172.21.15.45:0/2014440903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 msgr2=0x55e74ff3fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff3fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 -- 172.21.15.45:0/2014440903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74fc757d0 con 0x55e74ff32f50 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b85c39640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 0x55e74ff32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c6a0 tx=0x7f5b7c09b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9778d09c9c4dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 0x7f5b6c0a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b78004ab0 tx=0x7f5b78004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b74ff9640 1 -- 172.21.15.45:0/2014440903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c09bcb0 con 0x55e74ff32f50 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b74ff9640 1 -- 172.21.15.45:0/2014440903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c0a0060 con 0x55e74ff32f50 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.790+0000 7f5b74ff9640 1 -- 172.21.15.45:0/2014440903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c09bf10 con 0x55e74ff32f50 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2014440903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 msgr2=0x55e74ff3d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c6a0 tx=0x7f5b7c09b8b0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2014440903 shutdown_connections 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff3fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32f50 0x55e74ff3d0e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2014440903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32630 0x55e74ff32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2014440903 >> 172.21.15.45:0/2014440903 conn(0x55e74ff2e950 msgr2=0x55e74ff2ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2014440903 shutdown_connections 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.791+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2014440903 wait complete. 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 -- start start 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 Processor -- start 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff3aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e74ff40350 con 0x55e74ff32630 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e74fe52f50 con 0x55e74ff32f50 2024-04-25T00:24:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e74ff404d0 con 0x55e74ff3d620 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8b3ab740 10 client.0 _ll_register_callbacks cb 0x55e74fe8e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38170/0 (socket says 172.21.15.45:38170) 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 -- 172.21.15.45:0/1570453705 learned_addr learned my addr 172.21.15.45:0/1570453705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b85c39640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff3aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 -- 172.21.15.45:0/1570453705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 msgr2=0x55e74ff3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 -- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 msgr2=0x55e74ff3aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff3aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 -- 172.21.15.45:0/1570453705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74fbab3f0 con 0x55e74ff32630 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b85c39640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff3aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b8643a640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0cd4e0 tx=0x7f5b7c0a49f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe74adb0c2c71d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.792+0000 7f5b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c098390 con 0x55e74ff32630 2024-04-25T00:24:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5b7c09bf10 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c0a0060 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c09c150 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 10 client.16539 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e74fb6b0f0 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c0a0060 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b7c018b30 con 0x55e74ff32630 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 10 client.16539 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 20 client.16539 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ff4d6c0 0x55e74ff4fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 4 client.16539 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.793+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e74fea4bf0 con 0x55e74ff4d6c0 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.794+0000 7f5b85c39640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ff4d6c0 0x55e74ff4fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.794+0000 7f5b85c39640 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ff4d6c0 0x55e74ff4fab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163b2398b04fedaf server_cookie=88d8bf06f86a24f0 in_seq=0 out_seq=0 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.794+0000 7f5b69ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.794+0000 7f5b69ffb640 1 -- 172.21.15.45:0/1570453705 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e74fea4bf0 con 0x55e74ff4d6c0 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b69ffb640 10 client.16539 handle_command_reply: tid=0 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 client.16539 shutdown 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 msgr2=0x55e74ff3a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0cd4e0 tx=0x7f5b7c0a49f0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 shutdown_connections 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff3aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ff4d6c0 0x55e74ff4fab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/1570453705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3a570 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b69ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 >> 172.21.15.45:0/1570453705 conn(0x55e74ff2e950 msgr2=0x55e74ff2daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 shutdown_connections 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/1570453705 wait complete. 2024-04-25T00:24:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.795+0000 7f5b8b3ab740 20 client.16539 trim_cache size 0 max 16384 2024-04-25T00:24:46.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:46 smithi122 ceph-mon[32959]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 Processor -- start 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 -- start start 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff2ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801aebd0 con 0x55e74ff32630 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801b0710 con 0x55e74ff32f50 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.812+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801af2a0 con 0x55e74ff3d620 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38172/0 (socket says 172.21.15.45:38172) 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 -- 172.21.15.45:0/2534963047 learned_addr learned my addr 172.21.15.45:0/2534963047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b85c39640 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff2ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 -- 172.21.15.45:0/2534963047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 msgr2=0x55e74ff34490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff34490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 -- 172.21.15.45:0/2534963047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 msgr2=0x55e74ff2ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff2ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 -- 172.21.15.45:0/2534963047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74ff20090 con 0x55e74ff32630 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b8643a640 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c098680 tx=0x7f5b7c09a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52314068f7940e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b6bfff640 1 -- 172.21.15.45:0/2534963047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c09bf10 con 0x55e74ff32630 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b6bfff640 1 -- 172.21.15.45:0/2534963047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c09b250 con 0x55e74ff32630 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.813+0000 7f5b6bfff640 1 -- 172.21.15.45:0/2534963047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0a4490 con 0x55e74ff32630 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2534963047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 msgr2=0x55e74ff3cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c098680 tx=0x7f5b7c09a1c0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2534963047 shutdown_connections 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff3d620 0x55e74ff34490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff32f50 0x55e74ff2ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/2534963047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff3cca0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2534963047 >> 172.21.15.45:0/2534963047 conn(0x55e74ff2e950 msgr2=0x55e74ff33d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2534963047 shutdown_connections 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/2534963047 wait complete. 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- start start 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 Processor -- start 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32f50 0x55e74ff50110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801b0040 con 0x55e74ff32630 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801af970 con 0x55e74ff3d620 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b801acbb0 con 0x55e74ff32f50 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.814+0000 7f5b8b3ab740 10 client.0 _ll_register_callbacks cb 0x7f5b78009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38184/0 (socket says 172.21.15.45:38184) 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 -- 172.21.15.45:0/745834440 learned_addr learned my addr 172.21.15.45:0/745834440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b85c39640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32f50 0x55e74ff50110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b86c3b640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 -- 172.21.15.45:0/745834440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32f50 msgr2=0x55e74ff50110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32f50 0x55e74ff50110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 -- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 msgr2=0x55e74ff50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 -- 172.21.15.45:0/745834440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74ff50d40 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8643a640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c099f60 tx=0x7f5b7c0ccfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a63bbca89f62e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c098320 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5b7c0a4490 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c09b250 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c0af050 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8b3ab740 10 client.16545 fetch_fsmap learned FSMap version 28 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e74ff50d40 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.815+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c09b250 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b7c097b90 con 0x55e74ff32630 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 10 client.16545 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 20 client.16545 populate_metadata read hostname 'smithi045' 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ffb3a60 0x55e74ffb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 4 client.16545 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e74fe9ef40 con 0x55e74ffb3a60 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b85c39640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ffb3a60 0x55e74ffb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b69ffb640 10 client.16545 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.816+0000 7f5b85c39640 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ffb3a60 0x55e74ffb5e50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=defba773bb7238bf server_cookie=87fd6bd6fd72882f in_seq=0 out_seq=0 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b69ffb640 1 -- 172.21.15.45:0/745834440 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e74fe9ef40 con 0x55e74ffb3a60 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b69ffb640 10 client.16545 handle_command_reply: tid=0 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 client.16545 shutdown 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 msgr2=0x55e74ff52dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c099f60 tx=0x7f5b7c0ccfc0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 shutdown_connections 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e74ff3d620 0x55e74ff50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e74ff32f50 0x55e74ff50110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e74ffb3a60 0x55e74ffb5e50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 --2- 172.21.15.45:0/745834440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e74ff32630 0x55e74ff52dd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 >> 172.21.15.45:0/745834440 conn(0x55e74ff2e950 msgr2=0x55e74ff3ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b69ffb640 0 client.16545 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 shutdown_connections 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 1 -- 172.21.15.45:0/745834440 wait complete. 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8b3ab740 20 client.16545 trim_cache size 0 max 16384 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout: "6df21c77-fb0c-4808-8afd-fed306d89abd": { 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:24:46.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "6df21c77-fb0c-4808-8afd-fed306d89abd", 2024-04-25T00:24:46.826 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:24:46.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:24:46.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:24:46.826 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:24:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 msgr2=0x7f5b6c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 0x7f5b6c0a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b78004ab0 tx=0x7f5b78004a40 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 msgr2=0x7f5b80198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00e9f0 tx=0x7f5b7c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 shutdown_connections 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5b6c0a2d00 0x7f5b6c0a51c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80107720 0x7f5b8019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b801020d0 0x7f5b80198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 --2- 172.21.15.45:0/2550072419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b80101710 0x7f5b80198010 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.817+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 >> 172.21.15.45:0/2550072419 conn(0x7f5b800fd180 msgr2=0x7f5b800fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.818+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 shutdown_connections 2024-04-25T00:24:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:24:46.818+0000 7f5b8869e640 1 -- 172.21.15.45:0/2550072419 wait complete. 2024-04-25T00:24:46.828 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:46 smithi045 ceph-mon[29194]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-25T00:24:46.829 INFO:tasks.cephfs.filesystem:scrub status for tag:6df21c77-fb0c-4808-8afd-fed306d89abd - {'path': '/', 'tag': '6df21c77-fb0c-4808-8afd-fed306d89abd', 'options': 'recursive,force'} 2024-04-25T00:24:46.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:46 smithi071 ceph-mon[32667]: pgmap v87: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-25T00:24:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:24:48.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:24:48.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.3 MiB/s wr, 15 op/s 2024-04-25T00:24:48.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:48 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.3 MiB/s wr, 15 op/s 2024-04-25T00:24:48.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:48 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:24:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:24:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:24:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: pgmap v88: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.3 MiB/s wr, 15 op/s 2024-04-25T00:24:48.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:48 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:24:50.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:50 smithi122 ceph-mon[32959]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 20 op/s 2024-04-25T00:24:50.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:50 smithi071 ceph-mon[32667]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 20 op/s 2024-04-25T00:24:50.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:50 smithi045 ceph-mon[29194]: pgmap v89: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 MiB/s wr, 20 op/s 2024-04-25T00:24:52.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:52 smithi122 ceph-mon[32959]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:24:52.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:52 smithi071 ceph-mon[32667]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:24:52.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:52 smithi045 ceph-mon[29194]: pgmap v90: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-25T00:24:54.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:54 smithi122 ceph-mon[32959]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:24:54.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:54 smithi071 ceph-mon[32667]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:24:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:54 smithi045 ceph-mon[29194]: pgmap v91: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T00:24:56.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:56 smithi122 ceph-mon[32959]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T00:24:56.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:56 smithi071 ceph-mon[32667]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T00:24:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:56 smithi045 ceph-mon[29194]: pgmap v92: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T00:24:58.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:24:58 smithi122 ceph-mon[32959]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T00:24:58.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:24:58 smithi071 ceph-mon[32667]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T00:24:58.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:24:58 smithi045 ceph-mon[29194]: pgmap v93: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T00:25:00.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:00 smithi122 ceph-mon[32959]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:25:00.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:00 smithi071 ceph-mon[32667]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:25:00.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:00 smithi045 ceph-mon[29194]: pgmap v94: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:25:02.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:02 smithi122 ceph-mon[32959]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:02.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:02 smithi071 ceph-mon[32667]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:02 smithi045 ceph-mon[29194]: pgmap v95: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:04.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:04 smithi122 ceph-mon[32959]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:25:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:04 smithi071 ceph-mon[32667]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:25:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:04 smithi045 ceph-mon[29194]: pgmap v96: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T00:25:05.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:05 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:05 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:05 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:06.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:06 smithi071 ceph-mon[32667]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:06.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:06.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:06.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:06 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:25:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:06 smithi045 ceph-mon[29194]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:06 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:25:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:06 smithi122 ceph-mon[32959]: pgmap v97: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:06 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:25:08.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:08 smithi071 ceph-mon[32667]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:08.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:08 smithi045 ceph-mon[29194]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:08 smithi122 ceph-mon[32959]: pgmap v98: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:10.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:10 smithi071 ceph-mon[32667]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:25:10.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:10 smithi045 ceph-mon[29194]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:25:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:10 smithi122 ceph-mon[32959]: pgmap v99: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:25:12.858 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:12 smithi122 ceph-mon[32959]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:12 smithi071 ceph-mon[32667]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:12.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:12 smithi045 ceph-mon[29194]: pgmap v100: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:25:14.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:14 smithi071 ceph-mon[32667]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:25:14.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:14 smithi045 ceph-mon[29194]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:25:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:14 smithi122 ceph-mon[32959]: pgmap v101: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:25:16.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:16.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:16.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:16 smithi045 ceph-mon[29194]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:16.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:16 smithi071 ceph-mon[32667]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- 172.21.15.45:0/3852908069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3806a780 msgr2=0x7f1f3810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 --2- 172.21.15.45:0/3852908069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3806a780 0x7f1f3810e5f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009cd0 tx=0x7f1f3002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- 172.21.15.45:0/3852908069 shutdown_connections 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 --2- 172.21.15.45:0/3852908069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3806a780 0x7f1f3810e5f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 --2- 172.21.15.45:0/3852908069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38069dc0 0x7f1f3806a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 --2- 172.21.15.45:0/3852908069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f38107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- 172.21.15.45:0/3852908069 >> 172.21.15.45:0/3852908069 conn(0x7f1f380776a0 msgr2=0x7f1f38077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- 172.21.15.45:0/3852908069 shutdown_connections 2024-04-25T00:25:16.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- 172.21.15.45:0/3852908069 wait complete. 2024-04-25T00:25:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 Processor -- start 2024-04-25T00:25:16.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.950+0000 7f1f3e638640 1 -- start start 2024-04-25T00:25:16.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 0x7f1f381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f3819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38111b40 con 0x7f1f38069dc0 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381119c0 con 0x7f1f3806a780 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3e638640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38111cc0 con 0x7f1f38107730 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59896/0 (socket says 172.21.15.45:59896) 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 -- 172.21.15.45:0/2858207063 learned_addr learned my addr 172.21.15.45:0/2858207063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f377fe640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 0x7f1f381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f3819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 -- 172.21.15.45:0/2858207063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 msgr2=0x7f1f3819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f3819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 -- 172.21.15.45:0/2858207063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 msgr2=0x7f1f381984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 0x7f1f381984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 -- 172.21.15.45:0/2858207063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3819cf50 con 0x7f1f38069dc0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f377fe640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 0x7f1f381984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f3819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.951+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e9f0 tx=0x7f1f2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.952+0000 7f1f357fa640 1 -- 172.21.15.45:0/2858207063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800cd80 con 0x7f1f38069dc0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.952+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3819d240 con 0x7f1f38069dc0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.952+0000 7f1f357fa640 1 -- 172.21.15.45:0/2858207063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f28005500 con 0x7f1f38069dc0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.952+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f381a4ad0 con 0x7f1f38069dc0 2024-04-25T00:25:16.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.952+0000 7f1f357fa640 1 -- 172.21.15.45:0/2858207063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28018630 con 0x7f1f38069dc0 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.954+0000 7f1f357fa640 1 -- 172.21.15.45:0/2858207063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1f280046c0 con 0x7f1f38069dc0 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.954+0000 7f1f41423740 1 Processor -- start 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.954+0000 7f1f41423740 1 -- start start 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1da8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559da1d0d6b0 con 0x559da1da8d10 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559da1cb5880 con 0x559da1db7a30 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f41423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559da1d09670 con 0x559da1da8360 2024-04-25T00:25:16.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f357fa640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 0x7f1f240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59898/0 (socket says 172.21.15.45:59898) 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 -- 172.21.15.45:0/860359972 learned_addr learned my addr 172.21.15.45:0/860359972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 0x7f1f240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f377fe640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1da8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f357fa640 1 -- 172.21.15.45:0/2858207063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1f280cfdd0 con 0x7f1f38069dc0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 -- 172.21.15.45:0/860359972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 msgr2=0x559da1da8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1da8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 -- 172.21.15.45:0/860359972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 msgr2=0x559da1db9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 -- 172.21.15.45:0/860359972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da1aeb7d0 con 0x559da1da8d10 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1da8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f37fff640 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1f280cd1c0 tx=0x7f1f28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89ddc66a56e3bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.955+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 0x7f1f240a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f30033040 tx=0x7f1f30005da0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f2e7fc640 1 -- 172.21.15.45:0/860359972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f280ade60 con 0x559da1da8d10 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f2e7fc640 1 -- 172.21.15.45:0/860359972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2809c630 con 0x559da1da8d10 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f2e7fc640 1 -- 172.21.15.45:0/860359972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2801fe30 con 0x559da1da8d10 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 -- 172.21.15.45:0/860359972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 msgr2=0x559da1db74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1f280cd1c0 tx=0x7f1f28004a40 comp rx=0 tx=0).stop 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 -- 172.21.15.45:0/860359972 shutdown_connections 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db74f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 --2- 172.21.15.45:0/860359972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1da8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 -- 172.21.15.45:0/860359972 >> 172.21.15.45:0/860359972 conn(0x559da1da4950 msgr2=0x559da1da4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 -- 172.21.15.45:0/860359972 shutdown_connections 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.956+0000 7f1f41423740 1 -- 172.21.15.45:0/860359972 wait complete. 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 -- start start 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 Processor -- start 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1db4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559da1dba7f0 con 0x559da1da8d10 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559da1cc8f50 con 0x559da1db7a30 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559da1dba970 con 0x559da1da8360 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f41423740 10 client.0 _ll_register_callbacks cb 0x559da1d04000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59914/0 (socket says 172.21.15.45:59914) 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.957+0000 7f1f377fe640 1 -- 172.21.15.45:0/2723643322 learned_addr learned my addr 172.21.15.45:0/2723643322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1db4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 -- 172.21.15.45:0/2723643322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 msgr2=0x559da1db4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1db4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 -- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 msgr2=0x559da1db53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 -- 172.21.15.45:0/2723643322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da1a213f0 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1db4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f377fe640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009ed0 tx=0x7f1f200117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6091066823b2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20004220 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1f200043c0 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f2809c630 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f20012720 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f41423740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559da19e10f0 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.958+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2809c630 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f2000f040 con 0x559da1da8d10 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 20 client.16557 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1dc7d40 0x559da1dca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 4 client.16557 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da1d1abf0 con 0x559da1dc7d40 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1dc7d40 0x559da1dca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f37fff640 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1dc7d40 0x559da1dca200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff46c8d5035326d server_cookie=7a39ef0afa137fcc in_seq=0 out_seq=0 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.959+0000 7f1f177fe640 10 client.16557 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.960+0000 7f1f177fe640 1 -- 172.21.15.45:0/2723643322 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559da1d1abf0 con 0x559da1dc7d40 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.960+0000 7f1f177fe640 10 client.16557 handle_command_reply: tid=0 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.960+0000 7f1f41423740 1 client.16557 shutdown 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.960+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 msgr2=0x559da1db4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.960+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009ed0 tx=0x7f1f200117b0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 shutdown_connections 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1dc7d40 0x559da1dca200 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8d10 0x559da1db4e90 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 --2- 172.21.15.45:0/2723643322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8360 0x559da1db4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f177fe640 0 client.16557 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:16.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 >> 172.21.15.45:0/2723643322 conn(0x559da1da4950 msgr2=0x559da1daa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 shutdown_connections 2024-04-25T00:25:16.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 1 -- 172.21.15.45:0/2723643322 wait complete. 2024-04-25T00:25:16.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.961+0000 7f1f41423740 20 client.16557 trim_cache size 0 max 16384 2024-04-25T00:25:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.978+0000 7f1f41423740 1 Processor -- start 2024-04-25T00:25:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.978+0000 7f1f41423740 1 -- start start 2024-04-25T00:25:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.978+0000 7f1f41423740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.978+0000 7f1f41423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8d10 0x559da1dc3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f41423740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f41423740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ac910 con 0x559da1da8360 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f41423740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ae450 con 0x559da1db7a30 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f41423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381acfe0 con 0x559da1da8d10 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59928/0 (socket says 172.21.15.45:59928) 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 -- 172.21.15.45:0/757954330 learned_addr learned my addr 172.21.15.45:0/757954330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f377fe640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8d10 0x559da1dc3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 -- 172.21.15.45:0/757954330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8d10 msgr2=0x559da1dc3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8d10 0x559da1dc3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 -- 172.21.15.45:0/757954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 msgr2=0x559da1db5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 -- 172.21.15.45:0/757954330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da1da4780 con 0x559da1da8360 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f37fff640 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1f2809bc30 tx=0x7f1f280b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed62d99033eda82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f2dffb640 1 -- 172.21.15.45:0/757954330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2809bf30 con 0x559da1da8360 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f2dffb640 1 -- 172.21.15.45:0/757954330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f28099fd0 con 0x559da1da8360 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.979+0000 7f1f2dffb640 1 -- 172.21.15.45:0/757954330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f280cd3e0 con 0x559da1da8360 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- 172.21.15.45:0/757954330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 msgr2=0x559da1db7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1f2809bc30 tx=0x7f1f280b5040 comp rx=0 tx=0).stop 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- 172.21.15.45:0/757954330 shutdown_connections 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1db7a30 0x559da1db5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1da8d10 0x559da1dc3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 --2- 172.21.15.45:0/757954330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1db7080 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- 172.21.15.45:0/757954330 >> 172.21.15.45:0/757954330 conn(0x559da1da4950 msgr2=0x559da1db4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- 172.21.15.45:0/757954330 shutdown_connections 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- 172.21.15.45:0/757954330 wait complete. 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 -- start start 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.980+0000 7f1f41423740 1 Processor -- start 2024-04-25T00:25:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1da8d10 0x559da1dbba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1db7a30 0x559da1dbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381add80 con 0x559da1da8360 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ad6b0 con 0x559da1da8d10 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381aa8f0 con 0x559da1db7a30 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f41423740 10 client.0 _ll_register_callbacks cb 0x7f1f20009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59934/0 (socket says 172.21.15.45:59934) 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 -- 172.21.15.45:0/419056979 learned_addr learned my addr 172.21.15.45:0/419056979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f3cbd5640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1db7a30 0x559da1dbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f377fe640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1da8d10 0x559da1dbba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 -- 172.21.15.45:0/419056979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1db7a30 msgr2=0x559da1dbbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1db7a30 0x559da1dbbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 -- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1da8d10 msgr2=0x559da1dbba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1da8d10 0x559da1dbba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 -- 172.21.15.45:0/419056979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da1e2d380 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f37fff640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1f280a4670 tx=0x7f1f280a4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81d7a1aee1a051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f177fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.981+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2809bf30 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1f280b0140 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f28099fd0 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f2809a230 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 10 client.16563 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559da1e2e500 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f28099fd0 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f280a4710 con 0x559da1da8360 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 20 client.16563 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1e2e6d0 0x559da1e30ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 4 client.16563 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559da1d14f40 con 0x559da1e2e6d0 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.982+0000 7f1f377fe640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1e2e6d0 0x559da1e30ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f177fe640 10 client.16563 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f377fe640 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1e2e6d0 0x559da1e30ac0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b194078614afbd7 server_cookie=d6ba0b9645b8c9c0 in_seq=0 out_seq=0 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f177fe640 1 -- 172.21.15.45:0/419056979 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559da1d14f40 con 0x559da1e2e6d0 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f177fe640 10 client.16563 handle_command_reply: tid=0 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 client.16563 shutdown 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 msgr2=0x559da1dbb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1f280a4670 tx=0x7f1f280a4d20 comp rx=0 tx=0).stop 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 shutdown_connections 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559da1db7a30 0x559da1dbbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1da8d10 0x559da1dbba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559da1e2e6d0 0x559da1e30ac0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 --2- 172.21.15.45:0/419056979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559da1da8360 0x559da1dbb550 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f177fe640 0 client.16563 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 >> 172.21.15.45:0/419056979 conn(0x559da1da4950 msgr2=0x559da1da3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 shutdown_connections 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 1 -- 172.21.15.45:0/419056979 wait complete. 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.983+0000 7f1f41423740 20 client.16563 trim_cache size 0 max 16384 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:25:16.990 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 msgr2=0x7f1f240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 0x7f1f240a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f30033040 tx=0x7f1f30005da0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 msgr2=0x7f1f38197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e9f0 tx=0x7f1f2800edf0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 shutdown_connections 2024-04-25T00:25:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1f240a2d00 0x7f1f240a51c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f38107730 0x7f1f3819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f3806a780 0x7f1f381984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 --2- 172.21.15.45:0/2858207063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f38069dc0 0x7f1f38197f60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 >> 172.21.15.45:0/2858207063 conn(0x7f1f380776a0 msgr2=0x7f1f380fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 shutdown_connections 2024-04-25T00:25:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:16.984+0000 7f1f3e638640 1 -- 172.21.15.45:0/2858207063 wait complete. 2024-04-25T00:25:16.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:25:16.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:25:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:16 smithi122 ceph-mon[32959]: pgmap v102: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:25:17.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 -- 172.21.15.45:0/811010217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4100690 msgr2=0x7fd3b4100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/811010217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4100690 0x7fd3b4100b10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009930 tx=0x7fd39c02a420 comp rx=0 tx=0).stop 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 -- 172.21.15.45:0/811010217 shutdown_connections 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/811010217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4101050 0x7fd3b410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/811010217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4100690 0x7fd3b4100b10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/811010217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 0x7fd3b4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 -- 172.21.15.45:0/811010217 >> 172.21.15.45:0/811010217 conn(0x7fd3b40777a0 msgr2=0x7fd3b4077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 -- 172.21.15.45:0/811010217 shutdown_connections 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.139+0000 7fd3b90d4640 1 -- 172.21.15.45:0/811010217 wait complete. 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 Processor -- start 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 -- start start 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4100690 0x7fd3b4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 0x7fd3b41a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4110a60 con 0x7fd3b4101050 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41108e0 con 0x7fd3b4106650 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b90d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4110be0 con 0x7fd3b4100690 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59958/0 (socket says 172.21.15.45:59958) 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 -- 172.21.15.45:0/1591874451 learned_addr learned my addr 172.21.15.45:0/1591874451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b2d9d640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4100690 0x7fd3b4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3b359e640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 0x7fd3b41a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 -- 172.21.15.45:0/1591874451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4100690 msgr2=0x7fd3b4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4100690 0x7fd3b4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 -- 172.21.15.45:0/1591874451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 msgr2=0x7fd3b41a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 0x7fd3b41a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.140+0000 7fd3abfff640 1 -- 172.21.15.45:0/1591874451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b41a4460 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3abfff640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009230 tx=0x7fd39c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c03f030 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b41a4630 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c003ad0 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c035070 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.141+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a4aa0 con 0x7fd3b4101050 2024-04-25T00:25:17.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.142+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b4101d80 con 0x7fd3b4101050 2024-04-25T00:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.143+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd39c042030 con 0x7fd3b4101050 2024-04-25T00:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.144+0000 7fd3b0d99640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 0x7fd3a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.144+0000 7fd3b2d9d640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 0x7fd3a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.144+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd39c0efbd0 con 0x7fd3b4101050 2024-04-25T00:25:17.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.144+0000 7fd3b2d9d640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 0x7fd3a00a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b50 tx=0x7fd3a4005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.148+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd39c0bc490 con 0x7fd3b4101050 2024-04-25T00:25:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.374+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3b4108bb0 con 0x7fd3b4101050 2024-04-25T00:25:17.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.375+0000 7fd3b0d99640 1 -- 172.21.15.45:0/1591874451 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fd39c0bbd90 con 0x7fd3b4101050 2024-04-25T00:25:17.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:25:17.376 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:25:17.376 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:25:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 msgr2=0x7fd3a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 0x7fd3a00a51c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b50 tx=0x7fd3a4005f90 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 msgr2=0x7fd3b41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009230 tx=0x7fd39c004270 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 shutdown_connections 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3a00a2d00 0x7fd3a00a51c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3b4106650 0x7fd3b41a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3b4101050 0x7fd3b41993e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 --2- 172.21.15.45:0/1591874451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b4100690 0x7fd3b4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 >> 172.21.15.45:0/1591874451 conn(0x7fd3b40777a0 msgr2=0x7fd3b4103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 shutdown_connections 2024-04-25T00:25:17.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.377+0000 7fd3b90d4640 1 -- 172.21.15.45:0/1591874451 wait complete. 2024-04-25T00:25:17.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 -- 172.21.15.45:0/748038792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368100620 msgr2=0x7f7368100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 --2- 172.21.15.45:0/748038792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368100620 0x7f7368100aa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7350009930 tx=0x7f735002a420 comp rx=0 tx=0).stop 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 -- 172.21.15.45:0/748038792 shutdown_connections 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 --2- 172.21.15.45:0/748038792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f736810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 --2- 172.21.15.45:0/748038792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368100620 0x7f7368100aa0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 --2- 172.21.15.45:0/748038792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368106630 0x7f7368106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 -- 172.21.15.45:0/748038792 >> 172.21.15.45:0/748038792 conn(0x7f73680776a0 msgr2=0x7f7368077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 -- 172.21.15.45:0/748038792 shutdown_connections 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.493+0000 7f736cf14640 1 -- 172.21.15.45:0/748038792 wait complete. 2024-04-25T00:25:17.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 Processor -- start 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 -- start start 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368100620 0x7f7368198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f7368199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7368110a40 con 0x7f7368106630 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73681108c0 con 0x7f7368100620 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736cf14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7368110bc0 con 0x7f7368100fe0 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59988/0 (socket says 172.21.15.45:59988) 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 -- 172.21.15.45:0/1905870752 learned_addr learned my addr 172.21.15.45:0/1905870752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f736659c640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368100620 0x7f7368198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f7368199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 -- 172.21.15.45:0/1905870752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 msgr2=0x7f7368199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f7368199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 -- 172.21.15.45:0/1905870752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368100620 msgr2=0x7f7368198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368100620 0x7f7368198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f7366d9d640 1 -- 172.21.15.45:0/1905870752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681a4400 con 0x7f7368106630 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.494+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f7368199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f7366d9d640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f736000eaa0 tx=0x7f736000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f735ffff640 1 -- 172.21.15.45:0/1905870752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000cdb0 con 0x7f7368106630 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681a45d0 con 0x7f7368106630 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f735ffff640 1 -- 172.21.15.45:0/1905870752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360005020 con 0x7f7368106630 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f735ffff640 1 -- 172.21.15.45:0/1905870752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600116d0 con 0x7f7368106630 2024-04-25T00:25:17.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.495+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681a4b40 con 0x7f7368106630 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.497+0000 7f735ffff640 1 -- 172.21.15.45:0/1905870752 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f73600054d0 con 0x7f7368106630 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.497+0000 7f736fc23740 1 Processor -- start 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 -- start start 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3decd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562f3d40670 con 0x5562f3ddb480 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5562f3ce8970 con 0x5562f3ddbda0 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736fc23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5562f3d3c630 con 0x5562f3dea980 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735ffff640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 0x7f73540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60002/0 (socket says 172.21.15.45:60002) 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 -- 172.21.15.45:0/1226037866 learned_addr learned my addr 172.21.15.45:0/1226037866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f7366d9d640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 0x7f73540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736659c640 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735ffff640 1 -- 172.21.15.45:0/1905870752 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f73600cf9d0 con 0x7f7368106630 2024-04-25T00:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 -- 172.21.15.45:0/1226037866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 msgr2=0x5562f3decd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3decd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 -- 172.21.15.45:0/1226037866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 msgr2=0x5562f3dea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f735dd9b640 1 -- 172.21.15.45:0/1226037866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3d0c400 con 0x5562f3ddb480 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.498+0000 7f736659c640 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7350005c40 tx=0x7f7350032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96439001023a88f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f7366d9d640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 0x7f73540a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f736009bb20 tx=0x7f736006db50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f735cd99640 1 -- 172.21.15.45:0/1226037866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035050 con 0x5562f3ddb480 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f735cd99640 1 -- 172.21.15.45:0/1226037866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73500317b0 con 0x5562f3ddb480 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f735cd99640 1 -- 172.21.15.45:0/1226037866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500046c0 con 0x5562f3ddb480 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f736fc23740 1 -- 172.21.15.45:0/1226037866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 msgr2=0x5562f3ddb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.499+0000 7f736fc23740 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7350005c40 tx=0x7f7350032f00 comp rx=0 tx=0).stop 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- 172.21.15.45:0/1226037866 shutdown_connections 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3decd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- 172.21.15.45:0/1226037866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3ddb860 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- 172.21.15.45:0/1226037866 >> 172.21.15.45:0/1226037866 conn(0x5562f3dd78a0 msgr2=0x5562f3dd7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- 172.21.15.45:0/1226037866 shutdown_connections 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- 172.21.15.45:0/1226037866 wait complete. 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- start start 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 Processor -- start 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3dea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3de7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562f3ded6e0 con 0x5562f3dea980 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5562f3cfc050 con 0x5562f3ddb480 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5562f3ded860 con 0x5562f3ddbda0 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f736fc23740 10 client.0 _ll_register_callbacks cb 0x5562f3a53be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60010/0 (socket says 172.21.15.45:60010) 2024-04-25T00:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f735dd9b640 1 -- 172.21.15.45:0/2590268474 learned_addr learned my addr 172.21.15.45:0/2590268474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.500+0000 7f735dd9b640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3dea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f736659c640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3de7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 -- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 msgr2=0x5562f3de7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3de7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 -- 172.21.15.45:0/2590268474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 msgr2=0x5562f3dea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3dea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 -- 172.21.15.45:0/2590268474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f3ccddc0 con 0x5562f3dea980 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f735dd9b640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7350032d90 tx=0x7f73500054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2392873646d1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035040 con 0x5562f3dea980 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7350004ca0 con 0x5562f3dea980 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73500317b0 con 0x5562f3dea980 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7350004020 con 0x5562f3dea980 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f736fc23740 10 client.16581 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562f3a543f0 con 0x5562f3dea980 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73500317b0 con 0x5562f3dea980 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.501+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7350003b60 con 0x5562f3dea980 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 10 client.16581 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 10 client.16581 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 20 client.16581 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3dfa6f0 0x5562f3dfcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 4 client.16581 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562f3d4dbb0 con 0x5562f3dfa6f0 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736659c640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3dfa6f0 0x5562f3dfcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f736659c640 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3dfa6f0 0x5562f3dfcae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8960807817ce2c9 server_cookie=e4e10d915303a0d in_seq=0 out_seq=0 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.502+0000 7f734dffb640 10 client.16581 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f734dffb640 1 -- 172.21.15.45:0/2590268474 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562f3d4dbb0 con 0x5562f3dfa6f0 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f734dffb640 10 client.16581 handle_command_reply: tid=0 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 client.16581 shutdown 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 msgr2=0x5562f3de7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7350032d90 tx=0x7f73500054c0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 shutdown_connections 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3dfa6f0 0x5562f3dfcae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3de7a60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3de7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 --2- 172.21.15.45:0/2590268474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3dea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 >> 172.21.15.45:0/2590268474 conn(0x5562f3dd78a0 msgr2=0x5562f3dd6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f734dffb640 0 client.16581 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 shutdown_connections 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 1 -- 172.21.15.45:0/2590268474 wait complete. 2024-04-25T00:25:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.503+0000 7f736fc23740 20 client.16581 trim_cache size 0 max 16384 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 Processor -- start 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 -- start start 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3de9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3dffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ac880 con 0x5562f3dea980 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ae3c0 con 0x5562f3ddb480 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736fc23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73681acf50 con 0x5562f3ddbda0 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60026/0 (socket says 172.21.15.45:60026) 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 -- 172.21.15.45:0/4259063095 learned_addr learned my addr 172.21.15.45:0/4259063095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 -- 172.21.15.45:0/4259063095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 msgr2=0x5562f3dffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736659c640 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3dffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3dffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 -- 172.21.15.45:0/4259063095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 msgr2=0x5562f3de9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3de9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 -- 172.21.15.45:0/4259063095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3b1e7d0 con 0x5562f3dea980 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f736659c640 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3dffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.517+0000 7f735dd9b640 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7350005350 tx=0x7f73500058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32906c040505fc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f734ffff640 1 -- 172.21.15.45:0/4259063095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035040 con 0x5562f3dea980 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f734ffff640 1 -- 172.21.15.45:0/4259063095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350004ca0 con 0x5562f3dea980 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f734ffff640 1 -- 172.21.15.45:0/4259063095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350031e30 con 0x5562f3dea980 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- 172.21.15.45:0/4259063095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 msgr2=0x5562f3e00040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7350005350 tx=0x7f73500058b0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- 172.21.15.45:0/4259063095 shutdown_connections 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3dea980 0x5562f3e00040 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3ddbda0 0x5562f3dffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 --2- 172.21.15.45:0/4259063095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddb480 0x5562f3de9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- 172.21.15.45:0/4259063095 >> 172.21.15.45:0/4259063095 conn(0x5562f3dd78a0 msgr2=0x5562f3df63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- 172.21.15.45:0/4259063095 shutdown_connections 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- 172.21.15.45:0/4259063095 wait complete. 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.518+0000 7f736fc23740 1 -- start start 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 Processor -- start 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dfd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3dfd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73681adcf0 con 0x5562f3ddb480 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ad620 con 0x5562f3ddbda0 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73681aa860 con 0x5562f3dea980 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 10 client.0 _ll_register_callbacks cb 0x7f7350032e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60040/0 (socket says 172.21.15.45:60040) 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 -- 172.21.15.45:0/1840851234 learned_addr learned my addr 172.21.15.45:0/1840851234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3dfd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736659c640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dfd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 -- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 msgr2=0x5562f3dfd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3dfd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 -- 172.21.15.45:0/1840851234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 msgr2=0x5562f3dfd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dfd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 -- 172.21.15.45:0/1840851234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f3dfe1c0 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f735dd9b640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f735002a940 tx=0x7f735003d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe3dcdbf1e32004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f734dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035040 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f735003dcc0 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.519+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7350004ca0 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7350004e50 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 10 client.16587 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562f3e60970 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350004ca0 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73500316d0 con 0x5562f3ddb480 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 10 client.16587 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 10 client.16587 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 20 client.16587 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3e60d90 0x5562f3e63180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 4 client.16587 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562f3d47f00 con 0x5562f3e60d90 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.520+0000 7f736659c640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3e60d90 0x5562f3e63180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f734dffb640 10 client.16587 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736659c640 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3e60d90 0x5562f3e63180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df4fdbde6a9e2e4 server_cookie=79c7fdea8ac02a72 in_seq=0 out_seq=0 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f734dffb640 1 -- 172.21.15.45:0/1840851234 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562f3d47f00 con 0x5562f3e60d90 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f734dffb640 10 client.16587 handle_command_reply: tid=0 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 client.16587 shutdown 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 msgr2=0x5562f3dffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f735002a940 tx=0x7f735003d7c0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 shutdown_connections 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5562f3e60d90 0x5562f3e63180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5562f3dea980 0x5562f3dfd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5562f3ddbda0 0x5562f3dfd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 --2- 172.21.15.45:0/1840851234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562f3ddb480 0x5562f3dffff0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f734dffb640 0 client.16587 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 >> 172.21.15.45:0/1840851234 conn(0x5562f3dd78a0 msgr2=0x5562f3de5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 shutdown_connections 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 1 -- 172.21.15.45:0/1840851234 wait complete. 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736fc23740 20 client.16587 trim_cache size 0 max 16384 2024-04-25T00:25:17.529 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 msgr2=0x7f73540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 0x7f73540a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f736009bb20 tx=0x7f736006db50 comp rx=0 tx=0).stop 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 msgr2=0x7f73681a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.521+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f736000eaa0 tx=0x7f736000ef60 comp rx=0 tx=0).stop 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 shutdown_connections 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f73540a2d00 0x7f73540a51c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7368106630 0x7f73681a3ec0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7368100fe0 0x7f7368199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 --2- 172.21.15.45:0/1905870752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7368100620 0x7f7368198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 >> 172.21.15.45:0/1905870752 conn(0x7f73680776a0 msgr2=0x7f73681031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 shutdown_connections 2024-04-25T00:25:17.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.522+0000 7f736cf14640 1 -- 172.21.15.45:0/1905870752 wait complete. 2024-04-25T00:25:17.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 -- 172.21.15.45:0/3562226881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688100c90 msgr2=0x7f86881063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 --2- 172.21.15.45:0/3562226881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688100c90 0x7f86881063c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f867802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 -- 172.21.15.45:0/3562226881 shutdown_connections 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 --2- 172.21.15.45:0/3562226881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8688106aa0 0x7f868810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 --2- 172.21.15.45:0/3562226881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688100c90 0x7f86881063c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 --2- 172.21.15.45:0/3562226881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86881002e0 0x7f86881006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 -- 172.21.15.45:0/3562226881 >> 172.21.15.45:0/3562226881 conn(0x7f86880758d0 msgr2=0x7f8688075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 -- 172.21.15.45:0/3562226881 shutdown_connections 2024-04-25T00:25:17.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.638+0000 7f8690093640 1 -- 172.21.15.45:0/3562226881 wait complete. 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 Processor -- start 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 -- start start 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881002e0 0x7f8688101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 0x7f8688101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8688111b90 con 0x7f8688101cd0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8688111a10 con 0x7f8688106aa0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f8690093640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8688111d10 con 0x7f86881002e0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60064/0 (socket says 172.21.15.45:60064) 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 -- 172.21.15.45:0/1799923610 learned_addr learned my addr 172.21.15.45:0/1799923610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868de2f640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881002e0 0x7f8688101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868d62e640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 0x7f8688101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 -- 172.21.15.45:0/1799923610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881002e0 msgr2=0x7f8688101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881002e0 0x7f8688101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 -- 172.21.15.45:0/1799923610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 msgr2=0x7f8688101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 0x7f8688101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868e630640 1 -- 172.21.15.45:0/1799923610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86881a6920 con 0x7f8688101cd0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.639+0000 7f868d62e640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 0x7f8688101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f868e630640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f868000eaa0 tx=0x7f868000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f867effd640 1 -- 172.21.15.45:0/1799923610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680004d20 con 0x7f8688101cd0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f867effd640 1 -- 172.21.15.45:0/1799923610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8680005020 con 0x7f8688101cd0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86881a6c10 con 0x7f8688101cd0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f867effd640 1 -- 172.21.15.45:0/1799923610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86800116d0 con 0x7f8688101cd0 2024-04-25T00:25:17.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.640+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86881a7130 con 0x7f8688101cd0 2024-04-25T00:25:17.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.642+0000 7f867effd640 1 -- 172.21.15.45:0/1799923610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86800051d0 con 0x7f8688101cd0 2024-04-25T00:25:17.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.642+0000 7f8692d9d740 1 Processor -- start 2024-04-25T00:25:17.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.642+0000 7f8692d9d740 1 -- start start 2024-04-25T00:25:17.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc06960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc11030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557cffb6b670 con 0x557cffc11570 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557cffb13970 con 0x557cffc06580 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f8692d9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557cffb67630 con 0x557cffc06ea0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f867effd640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 0x7f86740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60080/0 (socket says 172.21.15.45:60080) 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 -- 172.21.15.45:0/198177606 learned_addr learned my addr 172.21.15.45:0/198177606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868e630640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 0x7f86740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868de2f640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc11030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc06960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f867effd640 1 -- 172.21.15.45:0/1799923610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f86800cf940 con 0x7f8688101cd0 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 -- 172.21.15.45:0/198177606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 msgr2=0x557cffc11030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc11030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 -- 172.21.15.45:0/198177606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 msgr2=0x557cffc06960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc06960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 -- 172.21.15.45:0/198177606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cffb37400 con 0x557cffc11570 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868de2f640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868d62e640 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f86780056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23622f46f893a71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.643+0000 7f868e630640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 0x7f86740a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86800ccde0 tx=0x7f8680086040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8673fff640 1 -- 172.21.15.45:0/198177606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678035040 con 0x557cffc11570 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8673fff640 1 -- 172.21.15.45:0/198177606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86780035f0 con 0x557cffc11570 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8673fff640 1 -- 172.21.15.45:0/198177606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867803d6e0 con 0x557cffc11570 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 -- 172.21.15.45:0/198177606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 msgr2=0x557cffc13960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f86780056d0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 -- 172.21.15.45:0/198177606 shutdown_connections 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc13960 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 --2- 172.21.15.45:0/198177606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc06960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 -- 172.21.15.45:0/198177606 >> 172.21.15.45:0/198177606 conn(0x557cffc028a0 msgr2=0x557cffc02c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 -- 172.21.15.45:0/198177606 shutdown_connections 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.644+0000 7f8692d9d740 1 -- 172.21.15.45:0/198177606 wait complete. 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 -- start start 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 Processor -- start 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc0e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557cffc14300 con 0x557cffc11570 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557cffb27050 con 0x557cffc06580 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557cffc14480 con 0x557cffc06ea0 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f8692d9d740 10 client.0 _ll_register_callbacks cb 0x557cff87ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60082/0 (socket says 172.21.15.45:60082) 2024-04-25T00:25:17.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f868d62e640 1 -- 172.21.15.45:0/2695083044 learned_addr learned my addr 172.21.15.45:0/2695083044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.645+0000 7f868d62e640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc10e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868de2f640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc0e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 -- 172.21.15.45:0/2695083044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 msgr2=0x557cffc0e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc0e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 -- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 msgr2=0x557cffc10e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc10e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 -- 172.21.15.45:0/2695083044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cffaf8dc0 con 0x557cffc11570 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f868d62e640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8678033000 tx=0x7f8678005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c1c90d55e8caf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678035070 con 0x557cffc11570 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f867804e030 con 0x557cffc11570 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86780035f0 con 0x557cffc11570 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f867805b050 con 0x557cffc11570 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f8692d9d740 10 client.16599 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cff87f3f0 con 0x557cffc11570 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86780035f0 con 0x557cffc11570 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f867803a040 con 0x557cffc11570 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.646+0000 7f8692d9d740 10 client.16599 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f8692d9d740 10 client.16599 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f8692d9d740 20 client.16599 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc213d0 0x557cffc237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f8692d9d740 4 client.16599 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cffb78bb0 con 0x557cffc213d0 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f868de2f640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc213d0 0x557cffc237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f86717fa640 10 client.16599 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.647+0000 7f868de2f640 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc213d0 0x557cffc237c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4a2ad1a959df359 server_cookie=fef651f9bf2ab9a in_seq=0 out_seq=0 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f86717fa640 1 -- 172.21.15.45:0/2695083044 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cffb78bb0 con 0x557cffc213d0 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f86717fa640 10 client.16599 handle_command_reply: tid=0 2024-04-25T00:25:17.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 client.16599 shutdown 2024-04-25T00:25:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 msgr2=0x557cffc0e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8678033000 tx=0x7f8678005a20 comp rx=0 tx=0).stop 2024-04-25T00:25:17.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:17 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1591874451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 shutdown_connections 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc213d0 0x557cffc237c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc0e6b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06ea0 0x557cffc0e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f86717fa640 0 client.16599 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2695083044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc10e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 >> 172.21.15.45:0/2695083044 conn(0x557cffc028a0 msgr2=0x557cffc086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 shutdown_connections 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.648+0000 7f8692d9d740 1 -- 172.21.15.45:0/2695083044 wait complete. 2024-04-25T00:25:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.649+0000 7f8692d9d740 20 client.16599 trim_cache size 0 max 16384 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 Processor -- start 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 -- start start 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06580 0x557cffc1d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 0x557cffc28d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b0ce0 con 0x557cffc11570 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b2820 con 0x557cffc06ea0 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f8692d9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b13b0 con 0x557cffc06580 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60096/0 (socket says 172.21.15.45:60096) 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 -- 172.21.15.45:0/2776224498 learned_addr learned my addr 172.21.15.45:0/2776224498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06580 0x557cffc1d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868de2f640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 0x557cffc28d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 -- 172.21.15.45:0/2776224498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06580 msgr2=0x557cffc1d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06580 0x557cffc1d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 -- 172.21.15.45:0/2776224498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 msgr2=0x557cffc28d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 0x557cffc28d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868d62e640 1 -- 172.21.15.45:0/2776224498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cff9497d0 con 0x557cffc11570 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.662+0000 7f868de2f640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 0x557cffc28d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f868d62e640 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8678031550 tx=0x7f867804b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358ac9497f59bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f86737fe640 1 -- 172.21.15.45:0/2776224498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678035030 con 0x557cffc11570 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f86737fe640 1 -- 172.21.15.45:0/2776224498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867804d020 con 0x557cffc11570 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f86737fe640 1 -- 172.21.15.45:0/2776224498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867803d480 con 0x557cffc11570 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 -- 172.21.15.45:0/2776224498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 msgr2=0x557cffc292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8678031550 tx=0x7f867804b040 comp rx=0 tx=0).stop 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 -- 172.21.15.45:0/2776224498 shutdown_connections 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc11570 0x557cffc292a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06ea0 0x557cffc28d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2776224498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc06580 0x557cffc1d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 -- 172.21.15.45:0/2776224498 >> 172.21.15.45:0/2776224498 conn(0x557cffc028a0 msgr2=0x557cffc01b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 -- 172.21.15.45:0/2776224498 shutdown_connections 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.663+0000 7f8692d9d740 1 -- 172.21.15.45:0/2776224498 wait complete. 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 -- start start 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 Processor -- start 2024-04-25T00:25:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc19750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 0x557cffc16ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b2150 con 0x557cffc06ea0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86881b1a80 con 0x557cffc06580 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86881aecc0 con 0x557cffc11570 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f8692d9d740 10 client.0 _ll_register_callbacks cb 0x7f8678032de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f868de2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f868de2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60106/0 (socket says 172.21.15.45:60106) 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f868de2f640 1 -- 172.21.15.45:0/2067929155 learned_addr learned my addr 172.21.15.45:0/2067929155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f868d62e640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc19750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.664+0000 7f868d62e640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 0x557cffc16ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 -- 172.21.15.45:0/2067929155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 msgr2=0x557cffc16ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 0x557cffc16ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 -- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 msgr2=0x557cffc19750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc19750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 -- 172.21.15.45:0/2067929155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cffc174e0 con 0x557cffc06ea0 2024-04-25T00:25:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868d62e640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 0x557cffc16ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f868de2f640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8684005e10 tx=0x7f8684010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524f3130d111a631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400fd60 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8684005020 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867804d020 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f868402e050 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f8692d9d740 10 client.16605 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cffc87270 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867804d020 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f868401f0f0 con 0x557cffc06ea0 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.665+0000 7f8692d9d740 10 client.16605 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 10 client.16605 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 20 client.16605 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc87a20 0x557cffc89e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 4 client.16605 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:17.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cffb72f00 con 0x557cffc87a20 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f868d62e640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc87a20 0x557cffc89e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f86717fa640 10 client.16605 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f868d62e640 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc87a20 0x557cffc89e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35a58eceb9efef2a server_cookie=2fc8a1bdfac3af3c in_seq=0 out_seq=0 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f86717fa640 1 -- 172.21.15.45:0/2067929155 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cffb72f00 con 0x557cffc87a20 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f86717fa640 10 client.16605 handle_command_reply: tid=0 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 1 client.16605 shutdown 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 msgr2=0x557cffc19c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.666+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8684005e10 tx=0x7f8684010b60 comp rx=0 tx=0).stop 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 shutdown_connections 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x557cffc87a20 0x557cffc89e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557cffc11570 0x557cffc16ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557cffc06ea0 0x557cffc19c90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f86717fa640 0 client.16605 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 --2- 172.21.15.45:0/2067929155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cffc06580 0x557cffc19750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 >> 172.21.15.45:0/2067929155 conn(0x557cffc028a0 msgr2=0x557cffc1c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 shutdown_connections 2024-04-25T00:25:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 1 -- 172.21.15.45:0/2067929155 wait complete. 2024-04-25T00:25:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8692d9d740 20 client.16605 trim_cache size 0 max 16384 2024-04-25T00:25:17.674 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 msgr2=0x7f86740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 0x7f86740a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86800ccde0 tx=0x7f8680086040 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 msgr2=0x7f86881a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f868000eaa0 tx=0x7f868000eea0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 shutdown_connections 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86740a2d00 0x7f86740a51c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8688101cd0 0x7f86881a6280 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8688106aa0 0x7f8688101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 --2- 172.21.15.45:0/1799923610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881002e0 0x7f8688101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 >> 172.21.15.45:0/1799923610 conn(0x7f86880758d0 msgr2=0x7f86880fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 shutdown_connections 2024-04-25T00:25:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.667+0000 7f8690093640 1 -- 172.21.15.45:0/1799923610 wait complete. 2024-04-25T00:25:17.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:25:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- 172.21.15.45:0/438552140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4101760 msgr2=0x7fe8f4101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/438552140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4101760 0x7fe8f4101be0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009930 tx=0x7fe8e802a420 comp rx=0 tx=0).stop 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- 172.21.15.45:0/438552140 shutdown_connections 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/438552140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4102120 0x7fe8f410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/438552140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4101760 0x7fe8f4101be0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/438552140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4107770 0x7fe8f4107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- 172.21.15.45:0/438552140 >> 172.21.15.45:0/438552140 conn(0x7fe8f40fd250 msgr2=0x7fe8f40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- 172.21.15.45:0/438552140 shutdown_connections 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- 172.21.15.45:0/438552140 wait complete. 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 Processor -- start 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.783+0000 7fe8f95b7640 1 -- start start 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4101760 0x7fe8f41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4102120 0x7fe8f41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4111b80 con 0x7fe8f4107770 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4111a00 con 0x7fe8f4101760 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f95b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4111d00 con 0x7fe8f4102120 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60126/0 (socket says 172.21.15.45:60126) 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 -- 172.21.15.45:0/2718702937 learned_addr learned my addr 172.21.15.45:0/2718702937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4101760 0x7fe8f41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4102120 0x7fe8f41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 -- 172.21.15.45:0/2718702937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4102120 msgr2=0x7fe8f41985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4102120 0x7fe8f41985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 -- 172.21.15.45:0/2718702937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4101760 msgr2=0x7fe8f41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4101760 0x7fe8f41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 -- 172.21.15.45:0/2718702937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f419d090 con 0x7fe8f4107770 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8f37fe640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00eb10 tx=0x7fe8ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.784+0000 7fe8dbfff640 1 -- 172.21.15.45:0/2718702937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec004b20 con 0x7fe8f4107770 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.785+0000 7fe8dbfff640 1 -- 172.21.15.45:0/2718702937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec004cc0 con 0x7fe8f4107770 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.785+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f419d380 con 0x7fe8f4107770 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.785+0000 7fe8dbfff640 1 -- 172.21.15.45:0/2718702937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec010760 con 0x7fe8f4107770 2024-04-25T00:25:17.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.785+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41a4c10 con 0x7fe8f4107770 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8dbfff640 1 -- 172.21.15.45:0/2718702937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe8ec005020 con 0x7fe8f4107770 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 Processor -- start 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 -- start start 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262580 0x558b3d262960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262ea0 0x558b3d26d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b3d1c7670 con 0x558b3d26d570 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558b3d16f970 con 0x558b3d262ea0 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b3d1c3630 con 0x558b3d262580 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60142/0 (socket says 172.21.15.45:60142) 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3626456953 learned_addr learned my addr 172.21.15.45:0/3626456953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262ea0 0x558b3d26d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8dbfff640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 0x7fe8dc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.787+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262580 0x558b3d262960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f37fe640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 0x7fe8dc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3626456953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262580 msgr2=0x558b3d262960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262580 0x558b3d262960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3626456953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262ea0 msgr2=0x558b3d26d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262ea0 0x558b3d26d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3626456953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3d193400 con 0x558b3d26d570 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8dbfff640 1 -- 172.21.15.45:0/2718702937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8ec0cfd60 con 0x7fe8f4107770 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005c40 tx=0x7fe8e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e30ad48e2e52a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8d97fa640 1 -- 172.21.15.45:0/3626456953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035040 con 0x558b3d26d570 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8d97fa640 1 -- 172.21.15.45:0/3626456953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e80319c0 con 0x558b3d26d570 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8d97fa640 1 -- 172.21.15.45:0/3626456953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004ac0 con 0x558b3d26d570 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.788+0000 7fe8f37fe640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 0x7fe8dc0a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09beb0 tx=0x7fe8ec06ef90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3626456953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 msgr2=0x558b3d26f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005c40 tx=0x7fe8e8005fd0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3626456953 shutdown_connections 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d26f960 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262ea0 0x558b3d26d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3626456953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262580 0x558b3d262960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3626456953 >> 172.21.15.45:0/3626456953 conn(0x558b3d25e8a0 msgr2=0x558b3d25ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3626456953 shutdown_connections 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3626456953 wait complete. 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- start start 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 Processor -- start 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d26a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d26d570 0x558b3d26a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b3d270300 con 0x558b3d262580 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558b3d183050 con 0x558b3d26d570 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b3d270480 con 0x558b3d262ea0 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8fc3a7740 10 client.0 _ll_register_callbacks cb 0x558b3cedabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60146/0 (socket says 172.21.15.45:60146) 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.789+0000 7fe8f27fc640 1 -- 172.21.15.45:0/137232015 learned_addr learned my addr 172.21.15.45:0/137232015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d26a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d26d570 0x558b3d26a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 -- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 msgr2=0x558b3d26a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d26a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 -- 172.21.15.45:0/137232015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d26d570 msgr2=0x558b3d26a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d26d570 0x558b3d26a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 -- 172.21.15.45:0/137232015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3d154dc0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80057c0 tx=0x7fe8e8005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5658ed9b774ca09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035040 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8e80048a0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e80319c0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8004aa0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8fc3a7740 10 client.16617 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b3cedb3f0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.790+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e80319c0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8e803a3f0 con 0x558b3d262580 2024-04-25T00:25:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 10 client.16617 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 10 client.16617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 20 client.16617 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d27d3d0 0x558b3d27f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 4 client.16617 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3d1d4bb0 con 0x558b3d27d3d0 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d27d3d0 0x558b3d27f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d27d3d0 0x558b3d27f7c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c177d1c03cfcc26 server_cookie=1d0d20149d366486 in_seq=0 out_seq=0 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.791+0000 7fe8d6ff5640 10 client.16617 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/137232015 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b3d1d4bb0 con 0x558b3d27d3d0 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8d6ff5640 10 client.16617 handle_command_reply: tid=0 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 client.16617 shutdown 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 msgr2=0x558b3d26ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80057c0 tx=0x7fe8e8005700 comp rx=0 tx=0).stop 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 shutdown_connections 2024-04-25T00:25:17.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d27d3d0 0x558b3d27f7c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d26d570 0x558b3d26a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d26a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8d6ff5640 0 client.16617 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/137232015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d262580 0x558b3d26ce80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 >> 172.21.15.45:0/137232015 conn(0x558b3d25e8a0 msgr2=0x558b3d2646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 shutdown_connections 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/137232015 wait complete. 2024-04-25T00:25:17.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.792+0000 7fe8fc3a7740 20 client.16617 trim_cache size 0 max 16384 2024-04-25T00:25:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 Processor -- start 2024-04-25T00:25:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 -- start start 2024-04-25T00:25:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d269020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41aeab0 con 0x558b3d26d570 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b05f0 con 0x558b3d262580 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41af180 con 0x558b3d262ea0 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60162/0 (socket says 172.21.15.45:60162) 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3002907971 learned_addr learned my addr 172.21.15.45:0/3002907971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d269020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3002907971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 msgr2=0x558b3d2827f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2827f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3002907971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 msgr2=0x558b3d269020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d269020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.806+0000 7fe8f27fc640 1 -- 172.21.15.45:0/3002907971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3cfa57d0 con 0x558b3d26d570 2024-04-25T00:25:17.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80096c0 tx=0x7fe8e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e180d4041bc9923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8d8ff9640 1 -- 172.21.15.45:0/3002907971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035070 con 0x558b3d26d570 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8d8ff9640 1 -- 172.21.15.45:0/3002907971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e804f070 con 0x558b3d26d570 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8d8ff9640 1 -- 172.21.15.45:0/3002907971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e805c070 con 0x558b3d26d570 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3002907971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 msgr2=0x558b3d282d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80096c0 tx=0x7fe8e8004770 comp rx=0 tx=0).stop 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3002907971 shutdown_connections 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d282d30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2827f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/3002907971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d269020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3002907971 >> 172.21.15.45:0/3002907971 conn(0x558b3d25e8a0 msgr2=0x558b3d268910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3002907971 shutdown_connections 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.807+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/3002907971 wait complete. 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 -- start start 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 Processor -- start 2024-04-25T00:25:17.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d2da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41aff20 con 0x558b3d26d570 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41af850 con 0x558b3d262580 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41aca90 con 0x558b3d262ea0 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8fc3a7740 10 client.0 _ll_register_callbacks cb 0x7fe8e400b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60174/0 (socket says 172.21.15.45:60174) 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 -- 172.21.15.45:0/1942712324 learned_addr learned my addr 172.21.15.45:0/1942712324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d2da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 -- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 msgr2=0x558b3d2daf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2daf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 -- 172.21.15.45:0/1942712324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 msgr2=0x558b3d2da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d2da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f27fc640 1 -- 172.21.15.45:0/1942712324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3d282450 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.808+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8f27fc640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005900 tx=0x7fe8e803a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535b8547edaccec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e805c070 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8e803a590 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e804f070 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8035030 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 10 client.16623 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b3d2e3280 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e804f070 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8e8062070 con 0x558b3d26d570 2024-04-25T00:25:17.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 10 client.16623 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 10 client.16623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 20 client.16623 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d2e3700 0x558b3d2e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 4 client.16623 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.809+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b3d1cef00 con 0x558b3d2e3700 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d2e3700 0x558b3d2e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8f2ffd640 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d2e3700 0x558b3d2e5af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707aaae348e98be2 server_cookie=700ac485e5aeda5 in_seq=0 out_seq=0 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8d6ff5640 10 client.16623 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8d6ff5640 1 -- 172.21.15.45:0/1942712324 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b3d1cef00 con 0x558b3d2e3700 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8d6ff5640 10 client.16623 handle_command_reply: tid=0 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 client.16623 shutdown 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 msgr2=0x558b3d281c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005900 tx=0x7fe8e803a3f0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 shutdown_connections 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558b3d2e3700 0x558b3d2e5af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b3d26d570 0x558b3d281c20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3d262ea0 0x558b3d2daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 --2- 172.21.15.45:0/1942712324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b3d262580 0x558b3d2da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8d6ff5640 0 client.16623 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 >> 172.21.15.45:0/1942712324 conn(0x558b3d25e8a0 msgr2=0x558b3d25db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 shutdown_connections 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 1 -- 172.21.15.45:0/1942712324 wait complete. 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.810+0000 7fe8fc3a7740 20 client.16623 trim_cache size 0 max 16384 2024-04-25T00:25:17.817 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 msgr2=0x7fe8dc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 0x7fe8dc0a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec09beb0 tx=0x7fe8ec06ef90 comp rx=0 tx=0).stop 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 msgr2=0x7fe8f419c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00eb10 tx=0x7fe8ec00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 shutdown_connections 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8dc0a2d00 0x7fe8dc0a51c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8f4107770 0x7fe8f419c960 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f4102120 0x7fe8f41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 --2- 172.21.15.45:0/2718702937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f4101760 0x7fe8f41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 >> 172.21.15.45:0/2718702937 conn(0x7fe8f40fd250 msgr2=0x7fe8f40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 shutdown_connections 2024-04-25T00:25:17.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.811+0000 7fe8f95b7640 1 -- 172.21.15.45:0/2718702937 wait complete. 2024-04-25T00:25:17.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:25:17.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:17 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1591874451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.928+0000 7f7e9afc1640 1 -- 172.21.15.45:0/1933967296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9410bd00 msgr2=0x7f7e9410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.928+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/1933967296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9410bd00 0x7f7e9410e1a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ca0 tx=0x7f7e8c02a9e0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- 172.21.15.45:0/1933967296 shutdown_connections 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/1933967296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9410bd00 0x7f7e9410e1a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/1933967296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941032e0 0x7f7e9410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/1933967296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e941029c0 0x7f7e94102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- 172.21.15.45:0/1933967296 >> 172.21.15.45:0/1933967296 conn(0x7f7e94077740 msgr2=0x7f7e94077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- 172.21.15.45:0/1933967296 shutdown_connections 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- 172.21.15.45:0/1933967296 wait complete. 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 Processor -- start 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- start start 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 0x7f7e94198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 0x7f7e9419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111b00 con 0x7f7e941032e0 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111980 con 0x7f7e941029c0 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e9afc1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94111c80 con 0x7f7e9410bd00 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60198/0 (socket says 172.21.15.45:60198) 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.929+0000 7f7e93fff640 1 -- 172.21.15.45:0/542634701 learned_addr learned my addr 172.21.15.45:0/542634701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e9955e640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 0x7f7e9419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 0x7f7e94198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 -- 172.21.15.45:0/542634701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 msgr2=0x7f7e9419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 0x7f7e9419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 -- 172.21.15.45:0/542634701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 msgr2=0x7f7e94198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 0x7f7e94198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 -- 172.21.15.45:0/542634701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9419cfc0 con 0x7f7e941032e0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 0x7f7e94198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e9955e640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 0x7f7e9419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e93fff640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e9f0 tx=0x7f7e8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e91ffb640 1 -- 172.21.15.45:0/542634701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800cd80 con 0x7f7e941032e0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e91ffb640 1 -- 172.21.15.45:0/542634701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e88010910 con 0x7f7e941032e0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9419d2b0 con 0x7f7e941032e0 2024-04-25T00:25:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e91ffb640 1 -- 172.21.15.45:0/542634701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88018990 con 0x7f7e941032e0 2024-04-25T00:25:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.930+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e941a4b40 con 0x7f7e941032e0 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e91ffb640 1 -- 172.21.15.45:0/542634701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7e88010ac0 con 0x7f7e941032e0 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 Processor -- start 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 -- start start 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650b870a670 con 0x5650b87b4980 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5650b86b2970 con 0x5650b87a5da0 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5650b8706630 con 0x5650b87a5480 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e91ffb640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 0x7f7e7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60200/0 (socket says 172.21.15.45:60200) 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e93fff640 1 -- 172.21.15.45:0/3005620017 learned_addr learned my addr 172.21.15.45:0/3005620017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.933+0000 7f7e9955e640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 0x7f7e7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e9955e640 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e91ffb640 1 -- 172.21.15.45:0/542634701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7e880cfba0 con 0x7f7e941032e0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 -- 172.21.15.45:0/3005620017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 msgr2=0x5650b87a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 -- 172.21.15.45:0/3005620017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 msgr2=0x5650b87b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 -- 172.21.15.45:0/3005620017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b86d6400 con 0x5650b87b4980 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e93fff640 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7e880981c0 tx=0x7f7e88018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea71bd4d78c32385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e9955e640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 0x7f7e7c0a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009430 tx=0x7f7e8c03a040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e7affd640 1 -- 172.21.15.45:0/3005620017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880a49d0 con 0x5650b87b4980 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e7affd640 1 -- 172.21.15.45:0/3005620017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8809bd60 con 0x5650b87b4980 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.934+0000 7f7e7affd640 1 -- 172.21.15.45:0/3005620017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88018d90 con 0x5650b87b4980 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3005620017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 msgr2=0x5650b87b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7e880981c0 tx=0x7f7e88018420 comp rx=0 tx=0).stop 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3005620017 shutdown_connections 2024-04-25T00:25:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87b6d70 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3005620017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3005620017 >> 172.21.15.45:0/3005620017 conn(0x5650b87a18a0 msgr2=0x5650b87a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3005620017 shutdown_connections 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3005620017 wait complete. 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 -- start start 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.935+0000 7f7e9ddae740 1 Processor -- start 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87b4980 0x5650b87b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650b87b76e0 con 0x5650b87a5da0 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5650b86c6050 con 0x5650b87b4980 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5650b87b7860 con 0x5650b87a5480 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 10 client.0 _ll_register_callbacks cb 0x5650b841dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60216/0 (socket says 172.21.15.45:60216) 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 -- 172.21.15.45:0/2258914420 learned_addr learned my addr 172.21.15.45:0/2258914420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9955e640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87b4980 0x5650b87b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 -- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 msgr2=0x5650b87b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 -- 172.21.15.45:0/2258914420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87b4980 msgr2=0x5650b87b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87b4980 0x5650b87b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 -- 172.21.15.45:0/2258914420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b8697dc0 con 0x5650b87a5da0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e880a44a0 tx=0x7f7e8809b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5c0649bfd47cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880a49d0 con 0x5650b87a5da0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7e88018d40 con 0x5650b87a5da0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.936+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8809bd60 con 0x5650b87a5da0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e88010430 con 0x5650b87a5da0 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 10 client.16635 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650b841e3f0 con 0x5650b87a5da0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8809bd60 con 0x5650b87a5da0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7e880a12a0 con 0x5650b87a5da0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 10 client.16635 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 10 client.16635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 20 client.16635 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b87c47c0 0x5650b87c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 4 client.16635 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b8717bb0 con 0x5650b87c47c0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.937+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b87c47c0 0x5650b87c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.938+0000 7f7e73fff640 10 client.16635 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.938+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b87c47c0 0x5650b87c6bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b4c7c0c07986a4f server_cookie=1e2e10d04bebee61 in_seq=0 out_seq=0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.938+0000 7f7e73fff640 1 -- 172.21.15.45:0/2258914420 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650b8717bb0 con 0x5650b87c47c0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.938+0000 7f7e73fff640 10 client.16635 handle_command_reply: tid=0 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 client.16635 shutdown 2024-04-25T00:25:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 msgr2=0x5650b87b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e880a44a0 tx=0x7f7e8809b460 comp rx=0 tx=0).stop 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 shutdown_connections 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b87c47c0 0x5650b87c6bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87b4980 0x5650b87b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5da0 0x5650b87b1590 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e73fff640 0 client.16635 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2258914420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 >> 172.21.15.45:0/2258914420 conn(0x5650b87a18a0 msgr2=0x5650b87a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 shutdown_connections 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2258914420 wait complete. 2024-04-25T00:25:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.939+0000 7f7e9ddae740 20 client.16635 trim_cache size 0 max 16384 2024-04-25T00:25:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 Processor -- start 2024-04-25T00:25:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 -- start start 2024-04-25T00:25:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ac9a0 con 0x5650b87b4980 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ae4e0 con 0x5650b87a5da0 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ad070 con 0x5650b87a5480 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.952+0000 7f7e9955e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60222/0 (socket says 172.21.15.45:60222) 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 -- 172.21.15.45:0/3867675562 learned_addr learned my addr 172.21.15.45:0/3867675562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e93fff640 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 -- 172.21.15.45:0/3867675562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 msgr2=0x5650b87c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 -- 172.21.15.45:0/3867675562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 msgr2=0x5650b87b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 -- 172.21.15.45:0/3867675562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b84e87d0 con 0x5650b87b4980 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e9955e640 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02ebc0 tx=0x7f7e8c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8d64b9db6c13c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e7a7fc640 1 -- 172.21.15.45:0/3867675562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c033070 con 0x5650b87b4980 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e7a7fc640 1 -- 172.21.15.45:0/3867675562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c02e910 con 0x5650b87b4980 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.953+0000 7f7e7a7fc640 1 -- 172.21.15.45:0/3867675562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c02fd40 con 0x5650b87b4980 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3867675562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 msgr2=0x5650b87cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02ebc0 tx=0x7f7e8c03e000 comp rx=0 tx=0).stop 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3867675562 shutdown_connections 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87b4980 0x5650b87cc1a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/3867675562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87a5480 0x5650b87c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3867675562 >> 172.21.15.45:0/3867675562 conn(0x5650b87a18a0 msgr2=0x5650b87acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3867675562 shutdown_connections 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- 172.21.15.45:0/3867675562 wait complete. 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 -- start start 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.954+0000 7f7e9ddae740 1 Processor -- start 2024-04-25T00:25:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87b4980 0x5650b87af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ade10 con 0x5650b87a5480 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ad740 con 0x5650b87a5da0 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941aa980 con 0x5650b87b4980 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 10 client.0 _ll_register_callbacks cb 0x7f7e8809b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60226/0 (socket says 172.21.15.45:60226) 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 -- 172.21.15.45:0/2104731904 learned_addr learned my addr 172.21.15.45:0/2104731904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 -- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87b4980 msgr2=0x5650b87af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87b4980 0x5650b87af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 -- 172.21.15.45:0/2104731904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 msgr2=0x5650b87aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 -- 172.21.15.45:0/2104731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b8829d00 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e98d5d640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009cb0 tx=0x7f7e84014a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb768ef35b6fc0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401e640 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7e8401e7e0 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c02e910 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e8402e050 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 10 client.16641 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.955+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5650b882b060 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c02e910 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7e84026530 con 0x5650b87a5480 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 10 client.16641 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 10 client.16641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 20 client.16641 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b882b810 0x5650b882dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 4 client.16641 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650b87b1e60 con 0x5650b882b810 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b882b810 0x5650b882dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e73fff640 10 client.16641 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.956+0000 7f7e93fff640 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b882b810 0x5650b882dc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b87e14df5923318a server_cookie=cd7f22f175a51196 in_seq=0 out_seq=0 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e73fff640 1 -- 172.21.15.45:0/2104731904 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650b87b1e60 con 0x5650b882b810 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e73fff640 10 client.16641 handle_command_reply: tid=0 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 client.16641 shutdown 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 msgr2=0x5650b87ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009cb0 tx=0x7f7e84014a60 comp rx=0 tx=0).stop 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 shutdown_connections 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5650b882b810 0x5650b882dc50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650b87b4980 0x5650b87af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650b87a5da0 0x5650b87aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e73fff640 0 client.16641 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 --2- 172.21.15.45:0/2104731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b87a5480 0x5650b87ae560 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 >> 172.21.15.45:0/2104731904 conn(0x5650b87a18a0 msgr2=0x5650b87a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 shutdown_connections 2024-04-25T00:25:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 1 -- 172.21.15.45:0/2104731904 wait complete. 2024-04-25T00:25:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.957+0000 7f7e9ddae740 20 client.16641 trim_cache size 0 max 16384 2024-04-25T00:25:17.964 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 msgr2=0x7f7e7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 0x7f7e7c0a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009430 tx=0x7f7e8c03a040 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 msgr2=0x7f7e94198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e9f0 tx=0x7f7e8800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 shutdown_connections 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7e7c0a2d00 0x7f7e7c0a51c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9410bd00 0x7f7e9419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e941032e0 0x7f7e94198560 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 --2- 172.21.15.45:0/542634701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e941029c0 0x7f7e94198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 >> 172.21.15.45:0/542634701 conn(0x7f7e94077740 msgr2=0x7f7e940fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 shutdown_connections 2024-04-25T00:25:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:17.958+0000 7f7e9afc1640 1 -- 172.21.15.45:0/542634701 wait complete. 2024-04-25T00:25:17.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:25:18.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:17 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1591874451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- 172.21.15.45:0/580135993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c107730 msgr2=0x7fbc9c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- 172.21.15.45:0/580135993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c107730 0x7fbc9c107b10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009930 tx=0x7fbc9802a420 comp rx=0 tx=0).stop 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- 172.21.15.45:0/580135993 shutdown_connections 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- 172.21.15.45:0/580135993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 0x7fbc9c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- 172.21.15.45:0/580135993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- 172.21.15.45:0/580135993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c107730 0x7fbc9c107b10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- 172.21.15.45:0/580135993 >> 172.21.15.45:0/580135993 conn(0x7fbc9c0776a0 msgr2=0x7fbc9c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- 172.21.15.45:0/580135993 shutdown_connections 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- 172.21.15.45:0/580135993 wait complete. 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 Processor -- start 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 -- start start 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.073+0000 7fbca34ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 0x7fbc9c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c107730 0x7fbc9c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111b00 con 0x7fbc9c069dc0 2024-04-25T00:25:18.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111980 con 0x7fbc9c06a780 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111c80 con 0x7fbc9c107730 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60244/0 (socket says 172.21.15.45:60244) 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 -- 172.21.15.45:0/2329657605 learned_addr learned my addr 172.21.15.45:0/2329657605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca0a55640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 0x7fbc9c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1a57640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c107730 0x7fbc9c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 -- 172.21.15.45:0/2329657605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c107730 msgr2=0x7fbc9c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c107730 0x7fbc9c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 -- 172.21.15.45:0/2329657605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 msgr2=0x7fbc9c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 0x7fbc9c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 -- 172.21.15.45:0/2329657605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c19d090 con 0x7fbc9c069dc0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca1256640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009230 tx=0x7fbc98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbc927fc640 1 -- 172.21.15.45:0/2329657605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9803f030 con 0x7fbc9c069dc0 2024-04-25T00:25:18.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbc927fc640 1 -- 172.21.15.45:0/2329657605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98003ad0 con 0x7fbc9c069dc0 2024-04-25T00:25:18.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c19d320 con 0x7fbc9c069dc0 2024-04-25T00:25:18.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbc927fc640 1 -- 172.21.15.45:0/2329657605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98035070 con 0x7fbc9c069dc0 2024-04-25T00:25:18.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.074+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1a4c10 con 0x7fbc9c069dc0 2024-04-25T00:25:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.076+0000 7fbc927fc640 1 -- 172.21.15.45:0/2329657605 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc98042030 con 0x7fbc9c069dc0 2024-04-25T00:25:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 Processor -- start 2024-04-25T00:25:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 -- start start 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbc927fc640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 0x7fbc880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5af0980 0x55fea5af2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fea5a46670 con 0x55fea5af0980 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fea59ee970 con 0x55fea5ae1da0 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca61ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fea5a42630 con 0x55fea5ae1480 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.077+0000 7fbca1a57640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 0x7fbc880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbc927fc640 1 -- 172.21.15.45:0/2329657605 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc980efd60 con 0x7fbc9c069dc0 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1a57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5af0980 0x55fea5af2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35738/0 (socket says 172.21.15.45:35738) 2024-04-25T00:25:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 -- 172.21.15.45:0/1580019050 learned_addr learned my addr 172.21.15.45:0/1580019050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca0a55640 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 -- 172.21.15.45:0/1580019050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 msgr2=0x55fea5af0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 -- 172.21.15.45:0/1580019050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5af0980 msgr2=0x55fea5af2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5af0980 0x55fea5af2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 -- 172.21.15.45:0/1580019050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea5a12400 con 0x55fea5ae1480 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1a57640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 0x7fbc880a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc94005fd0 tx=0x7fbc94009e90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbca1256640 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009900 tx=0x7fbc980bb930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a347d33af3dcf4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbc877fe640 1 -- 172.21.15.45:0/1580019050 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98049040 con 0x55fea5ae1480 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbc877fe640 1 -- 172.21.15.45:0/1580019050 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.078+0000 7fbc877fe640 1 -- 172.21.15.45:0/1580019050 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98003c80 con 0x55fea5ae1480 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- 172.21.15.45:0/1580019050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 msgr2=0x55fea5ae1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009900 tx=0x7fbc980bb930 comp rx=0 tx=0).stop 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- 172.21.15.45:0/1580019050 shutdown_connections 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5af0980 0x55fea5af2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1580019050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1480 0x55fea5ae1860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- 172.21.15.45:0/1580019050 >> 172.21.15.45:0/1580019050 conn(0x55fea5add8a0 msgr2=0x55fea5addc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- 172.21.15.45:0/1580019050 shutdown_connections 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- 172.21.15.45:0/1580019050 wait complete. 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.079+0000 7fbca61ca740 1 -- start start 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 Processor -- start 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1da0 0x55fea5aed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 0x55fea5aedad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fea5af3500 con 0x55fea5ae1480 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fea5a02050 con 0x55fea5af0980 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fea5af3680 con 0x55fea5ae1da0 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca61ca740 10 client.0 _ll_register_callbacks cb 0x55fea5759be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60270/0 (socket says 172.21.15.45:60270) 2024-04-25T00:25:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 -- 172.21.15.45:0/1864786218 learned_addr learned my addr 172.21.15.45:0/1864786218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca0a55640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1da0 0x55fea5aed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1a57640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 0x55fea5aedad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 -- 172.21.15.45:0/1864786218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1da0 msgr2=0x55fea5aed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1da0 0x55fea5aed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 -- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 msgr2=0x55fea5aedad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 0x55fea5aedad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1256640 1 -- 172.21.15.45:0/1864786218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea59d3dc0 con 0x55fea5ae1480 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.080+0000 7fbca1a57640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 0x55fea5aedad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca1256640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbc980bbf60 tx=0x7fbc980bba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353b23e02674b8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98049070 con 0x55fea5ae1480 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc980ed3d0 con 0x55fea5ae1480 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc9809f050 con 0x55fea5ae1480 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 10 client.16653 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fea575a3f0 con 0x55fea5ae1480 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc980d0440 con 0x55fea5ae1480 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 10 client.16653 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 20 client.16653 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b00510 0x55fea5b02900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 4 client.16653 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.081+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea5a53bb0 con 0x55fea5b00510 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.082+0000 7fbca0a55640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b00510 0x55fea5b02900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.082+0000 7fbca0a55640 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b00510 0x55fea5b02900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebe7755ba219d2d server_cookie=41e87975835b86ab in_seq=0 out_seq=0 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.082+0000 7fbc84ff9640 10 client.16653 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbc84ff9640 1 -- 172.21.15.45:0/1864786218 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fea5a53bb0 con 0x55fea5b00510 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbc84ff9640 10 client.16653 handle_command_reply: tid=0 2024-04-25T00:25:18.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 client.16653 shutdown 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 msgr2=0x55fea5af02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbc980bbf60 tx=0x7fbc980bba60 comp rx=0 tx=0).stop 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 shutdown_connections 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5af0980 0x55fea5aedad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5ae1da0 0x55fea5aed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b00510 0x55fea5b02900 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 --2- 172.21.15.45:0/1864786218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af02a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 >> 172.21.15.45:0/1864786218 conn(0x55fea5add8a0 msgr2=0x55fea5ae36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbc84ff9640 0 client.16653 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 shutdown_connections 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 1 -- 172.21.15.45:0/1864786218 wait complete. 2024-04-25T00:25:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.083+0000 7fbca61ca740 20 client.16653 trim_cache size 0 max 16384 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.096+0000 7fbca61ca740 1 Processor -- start 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.096+0000 7fbca61ca740 1 -- start start 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5ae8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5aec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ac8b0 con 0x55fea5ae1480 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ae3f0 con 0x55fea5ae1da0 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca61ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1acf80 con 0x55fea5af0980 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60276/0 (socket says 172.21.15.45:60276) 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 -- 172.21.15.45:0/860013037 learned_addr learned my addr 172.21.15.45:0/860013037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca0a55640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5ae8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1a57640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5aec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 -- 172.21.15.45:0/860013037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 msgr2=0x55fea5aec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5aec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 -- 172.21.15.45:0/860013037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 msgr2=0x55fea5ae8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5ae8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 -- 172.21.15.45:0/860013037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea58247d0 con 0x55fea5ae1480 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca0a55640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5ae8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbca1256640 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbc980c4660 tx=0x7fbc980ed200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71af6b5fa0a8a4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbc86ffd640 1 -- 172.21.15.45:0/860013037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98049040 con 0x55fea5ae1480 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbc86ffd640 1 -- 172.21.15.45:0/860013037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.097+0000 7fbc86ffd640 1 -- 172.21.15.45:0/860013037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980ed3d0 con 0x55fea5ae1480 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- 172.21.15.45:0/860013037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 msgr2=0x55fea5afc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbc980c4660 tx=0x7fbc980ed200 comp rx=0 tx=0).stop 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- 172.21.15.45:0/860013037 shutdown_connections 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5aec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5ae8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 --2- 172.21.15.45:0/860013037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5afc2b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- 172.21.15.45:0/860013037 >> 172.21.15.45:0/860013037 conn(0x55fea5add8a0 msgr2=0x55fea5aebe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- 172.21.15.45:0/860013037 shutdown_connections 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- 172.21.15.45:0/860013037 wait complete. 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 -- start start 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.098+0000 7fbca61ca740 1 Processor -- start 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5ae9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1add20 con 0x55fea5ae1480 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ad650 con 0x55fea5ae1da0 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1aa890 con 0x55fea5af0980 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca61ca740 10 client.0 _ll_register_callbacks cb 0x7fbc94011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60286/0 (socket says 172.21.15.45:60286) 2024-04-25T00:25:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 -- 172.21.15.45:0/2392009314 learned_addr learned my addr 172.21.15.45:0/2392009314 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1a57640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5ae9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca0a55640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 -- 172.21.15.45:0/2392009314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 msgr2=0x55fea5ae9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5ae9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 -- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 msgr2=0x55fea5af8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 -- 172.21.15.45:0/2392009314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea5ae9dc0 con 0x55fea5ae1480 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbca1256640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbc980b8430 tx=0x7fbc980d73e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27b96ddd5effa71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.099+0000 7fbc84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98049050 con 0x55fea5ae1480 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fbc980ed3d0 con 0x55fea5ae1480 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc980cd050 con 0x55fea5ae1480 2024-04-25T00:25:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 10 client.16659 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fea5ae9dc0 con 0x55fea5ae1480 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc980bbfd0 con 0x55fea5ae1480 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc98031c40 con 0x55fea5ae1480 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 10 client.16659 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 20 client.16659 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b663c0 0x55fea5b687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 4 client.16659 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fea5a4df00 con 0x55fea5b663c0 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.100+0000 7fbca0a55640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b663c0 0x55fea5b687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbc84ff9640 10 client.16659 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca0a55640 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b663c0 0x55fea5b687b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41da4fe7c035722 server_cookie=25e4aec1f988743f in_seq=0 out_seq=0 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbc84ff9640 1 -- 172.21.15.45:0/2392009314 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fea5a4df00 con 0x55fea5b663c0 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbc84ff9640 10 client.16659 handle_command_reply: tid=0 2024-04-25T00:25:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 client.16659 shutdown 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 msgr2=0x55fea5af87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbc980b8430 tx=0x7fbc980d73e0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 shutdown_connections 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fea5af0980 0x55fea5ae9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fea5ae1da0 0x55fea5af8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55fea5b663c0 0x55fea5b687b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 --2- 172.21.15.45:0/2392009314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fea5ae1480 0x55fea5af87c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbc84ff9640 0 client.16659 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 >> 172.21.15.45:0/2392009314 conn(0x55fea5add8a0 msgr2=0x55fea5ae8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 shutdown_connections 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 1 -- 172.21.15.45:0/2392009314 wait complete. 2024-04-25T00:25:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.101+0000 7fbca61ca740 20 client.16659 trim_cache size 0 max 16384 2024-04-25T00:25:18.109 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 msgr2=0x7fbc880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 0x7fbc880a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc94005fd0 tx=0x7fbc94009e90 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 msgr2=0x7fbc9c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009230 tx=0x7fbc98004270 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 shutdown_connections 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbc880a2d00 0x7fbc880a51c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c107730 0x7fbc9c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9c06a780 0x7fbc9c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 --2- 172.21.15.45:0/2329657605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc9c069dc0 0x7fbc9c1980a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 >> 172.21.15.45:0/2329657605 conn(0x7fbc9c0776a0 msgr2=0x7fbc9c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 shutdown_connections 2024-04-25T00:25:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:18.102+0000 7fbca34ba640 1 -- 172.21.15.45:0/2329657605 wait complete. 2024-04-25T00:25:18.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:18 smithi071 ceph-mon[32667]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:25:18.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:18 smithi045 ceph-mon[29194]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:25:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:18 smithi122 ceph-mon[32959]: pgmap v103: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:25:19.114 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:25:19.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:25:19.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:25:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.233+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2132935430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4100990 msgr2=0x7fe8c4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.233+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2132935430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4100990 0x7fe8c4100df0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009900 tx=0x7fe8b402a410 comp rx=0 tx=0).stop 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.233+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2132935430 shutdown_connections 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.233+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2132935430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4101330 0x7fe8c410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.233+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2132935430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4100990 0x7fe8c4100df0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2132935430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4106690 0x7fe8c4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2132935430 >> 172.21.15.45:0/2132935430 conn(0x7fe8c4076440 msgr2=0x7fe8c4076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2132935430 shutdown_connections 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2132935430 wait complete. 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 Processor -- start 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 -- start start 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4100990 0x7fe8c419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4106690 0x7fe8c419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.234+0000 7fe8cb5b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41109e0 con 0x7fe8c4101330 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8cb5b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4110860 con 0x7fe8c4100990 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8cb5b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4110b60 con 0x7fe8c4106690 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60302/0 (socket says 172.21.15.45:60302) 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/2684995638 learned_addr learned my addr 172.21.15.45:0/2684995638 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4106690 0x7fe8c419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c934c640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4100990 0x7fe8c419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/2684995638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4106690 msgr2=0x7fe8c419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4106690 0x7fe8c419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/2684995638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4100990 msgr2=0x7fe8c419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4100990 0x7fe8c419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/2684995638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c419f180 con 0x7fe8c4101330 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4006010 tx=0x7fe8b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8ba7fc640 1 -- 172.21.15.45:0/2684995638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b403f030 con 0x7fe8c4101330 2024-04-25T00:25:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8ba7fc640 1 -- 172.21.15.45:0/2684995638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b4003ce0 con 0x7fe8c4101330 2024-04-25T00:25:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c419f410 con 0x7fe8c4101330 2024-04-25T00:25:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.235+0000 7fe8ba7fc640 1 -- 172.21.15.45:0/2684995638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4035070 con 0x7fe8c4101330 2024-04-25T00:25:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.236+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c41a6d00 con 0x7fe8c4101330 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.238+0000 7fe8ba7fc640 1 -- 172.21.15.45:0/2684995638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe8b4042030 con 0x7fe8c4101330 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.238+0000 7fe8ce39e740 1 Processor -- start 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.238+0000 7fe8ce39e740 1 -- start start 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.238+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f2084402c0 con 0x55f2084dd980 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f2083e8910 con 0x55f2084dc290 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f20843c280 con 0x55f2084e7010 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60308/0 (socket says 172.21.15.45:60308) 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/813466217 learned_addr learned my addr 172.21.15.45:0/813466217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ba7fc640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 0x7fe8b00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c934c640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/813466217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 msgr2=0x55f2084e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/813466217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 msgr2=0x55f2084dc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084dc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/813466217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2083f9fb0 con 0x55f2084dd980 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c934c640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8ba7fc640 1 -- 172.21.15.45:0/2684995638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8b40ef880 con 0x7fe8c4101330 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 0x7fe8b00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc010510 tx=0x7fe8bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10df904147a6e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.239+0000 7fe8af7fe640 1 -- 172.21.15.45:0/813466217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc01d400 con 0x55f2084dd980 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8af7fe640 1 -- 172.21.15.45:0/813466217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc0134c0 con 0x55f2084dd980 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8af7fe640 1 -- 172.21.15.45:0/813466217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0118a0 con 0x55f2084dd980 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 0x7fe8b00a5240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc021000 tx=0x7fe8bc01e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 -- 172.21.15.45:0/813466217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 msgr2=0x55f2084dbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc010510 tx=0x7fe8bc009f90 comp rx=0 tx=0).stop 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 -- 172.21.15.45:0/813466217 shutdown_connections 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/813466217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084dbcc0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 -- 172.21.15.45:0/813466217 >> 172.21.15.45:0/813466217 conn(0x55f2084a4950 msgr2=0x55f2084d87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 -- 172.21.15.45:0/813466217 shutdown_connections 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.240+0000 7fe8ce39e740 1 -- 172.21.15.45:0/813466217 wait complete. 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 -- start start 2024-04-25T00:25:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 Processor -- start 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f2084e9ea0 con 0x55f2084dd980 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f2083fbe90 con 0x55f2084dc290 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f2084ea020 con 0x55f2084e7010 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8ce39e740 10 client.0 _ll_register_callbacks cb 0x55f208437970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8c8b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8c8b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60324/0 (socket says 172.21.15.45:60324) 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.241+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/1459003130 learned_addr learned my addr 172.21.15.45:0/1459003130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c934c640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/1459003130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 msgr2=0x55f2084e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 msgr2=0x55f2084e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 -- 172.21.15.45:0/1459003130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20821c7e0 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c934c640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40b8010 tx=0x7fe8b40bb7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6972a58c25dc32e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4049070 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8b40b8220 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8bc0134c0 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8b403b580 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8ce39e740 10 client.16671 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f20827cf70 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc0134c0 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.242+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8b40da270 con 0x55f2084dd980 2024-04-25T00:25:19.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 10 client.16671 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 20 client.16671 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f2084f6f30 0x55f2084f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 4 client.16671 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2084145a0 con 0x55f2084f6f30 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8c934c640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f2084f6f30 0x55f2084f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8acff9640 10 client.16671 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.243+0000 7fe8c934c640 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f2084f6f30 0x55f2084f9320 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20b757ed472b788 server_cookie=2e4a0a79c4c3e666 in_seq=0 out_seq=0 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8acff9640 1 -- 172.21.15.45:0/1459003130 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2084145a0 con 0x55f2084f6f30 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8acff9640 10 client.16671 handle_command_reply: tid=0 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 client.16671 shutdown 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 msgr2=0x55f2084e4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40b8010 tx=0x7fe8b40bb7b0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 shutdown_connections 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f2084f6f30 0x55f2084f9320 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dd980 0x55f2084e4420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1459003130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dc290 0x55f2084e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8acff9640 0 client.16671 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 >> 172.21.15.45:0/1459003130 conn(0x55f2084a4950 msgr2=0x55f2084de660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 shutdown_connections 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.244+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1459003130 wait complete. 2024-04-25T00:25:19.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.245+0000 7fe8ce39e740 20 client.16671 trim_cache size 0 max 16384 2024-04-25T00:25:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.262+0000 7fe8ce39e740 1 Processor -- start 2024-04-25T00:25:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.262+0000 7fe8ce39e740 1 -- start start 2024-04-25T00:25:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084dc290 0x55f2084f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084fea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b0c50 con 0x55f2084e7010 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b2790 con 0x55f2084dd980 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b1320 con 0x55f2084dc290 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60334/0 (socket says 172.21.15.45:60334) 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/1795592359 learned_addr learned my addr 172.21.15.45:0/1795592359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c934c640 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084dc290 0x55f2084f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/1795592359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084dc290 msgr2=0x55f2084f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084dc290 0x55f2084f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/1795592359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 msgr2=0x55f2084fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084fea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 -- 172.21.15.45:0/1795592359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f208114170 con 0x55f2084e7010 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009a00 tx=0x7fe8bc009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f879769dfdcc53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.263+0000 7fe8aeffd640 1 -- 172.21.15.45:0/1795592359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc013580 con 0x55f2084e7010 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8aeffd640 1 -- 172.21.15.45:0/1795592359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc011c00 con 0x55f2084e7010 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8aeffd640 1 -- 172.21.15.45:0/1795592359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc027480 con 0x55f2084e7010 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1795592359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 msgr2=0x55f2084fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009a00 tx=0x7fe8bc009a30 comp rx=0 tx=0).stop 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1795592359 shutdown_connections 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084e7010 0x55f2084fef50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/1795592359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084dc290 0x55f2084f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1795592359 >> 172.21.15.45:0/1795592359 conn(0x55f2084a4950 msgr2=0x55f2084d7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1795592359 shutdown_connections 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- 172.21.15.45:0/1795592359 wait complete. 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 -- start start 2024-04-25T00:25:19.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.264+0000 7fe8ce39e740 1 Processor -- start 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b20c0 con 0x55f2084dc290 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b19f0 con 0x55f2084dd980 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41aec30 con 0x55f2084e7010 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8ce39e740 10 client.0 _ll_register_callbacks cb 0x7fe8b403b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60338/0 (socket says 172.21.15.45:60338) 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 -- 172.21.15.45:0/915314574 learned_addr learned my addr 172.21.15.45:0/915314574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 -- 172.21.15.45:0/915314574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 msgr2=0x55f2084ebd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c9b4d640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084ebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 -- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 msgr2=0x55f2084eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084eb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c934c640 1 -- 172.21.15.45:0/915314574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2084ec2f0 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.265+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8c934c640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55f2084f9950 tx=0x7fe8c001b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416859face367b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c001b790 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8bc011c00 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe8c00035f0 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c001b930 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 10 client.16677 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f20855e090 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc011c00 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe8c0022070 con 0x55f2084dc290 2024-04-25T00:25:19.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 10 client.16677 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 20 client.16677 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f20855e810 0x55f208560c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 4 client.16677 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.266+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f20844d800 con 0x55f20855e810 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.267+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f20855e810 0x55f208560c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.267+0000 7fe8acff9640 10 client.16677 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.267+0000 7fe8c8b4b640 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f20855e810 0x55f208560c30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f4e0415f8b600a server_cookie=c58a40eed9367a7 in_seq=0 out_seq=0 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8acff9640 1 -- 172.21.15.45:0/915314574 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f20844d800 con 0x55f20855e810 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8acff9640 10 client.16677 handle_command_reply: tid=0 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 client.16677 shutdown 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 msgr2=0x55f2084ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55f2084f9950 tx=0x7fe8c001b660 comp rx=0 tx=0).stop 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 shutdown_connections 2024-04-25T00:25:19.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f2084e7010 0x55f2084ebd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2084dd980 0x55f2084eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f20855e810 0x55f208560c30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 --2- 172.21.15.45:0/915314574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2084dc290 0x55f2084ef470 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8acff9640 0 client.16677 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.268+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 >> 172.21.15.45:0/915314574 conn(0x55f2084a4950 msgr2=0x55f2084e3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 shutdown_connections 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8ce39e740 1 -- 172.21.15.45:0/915314574 wait complete. 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8ce39e740 20 client.16677 trim_cache size 0 max 16384 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:19.275 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:25:19.276 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "b9c51c98-8a7c-4d80-b886-1db496af515a", 2024-04-25T00:25:19.276 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:25:19.276 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 msgr2=0x7fe8b00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 0x7fe8b00a5240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc021000 tx=0x7fe8bc01e000 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 msgr2=0x7fe8c419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4006010 tx=0x7fe8b40042f0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 shutdown_connections 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe8b00a2d80 0x7fe8b00a5240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c4106690 0x7fe8c419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c4101330 0x7fe8c419a6d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 --2- 172.21.15.45:0/2684995638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8c4100990 0x7fe8c419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 >> 172.21.15.45:0/2684995638 conn(0x7fe8c4076440 msgr2=0x7fe8c40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 shutdown_connections 2024-04-25T00:25:19.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.269+0000 7fe8cb5b0640 1 -- 172.21.15.45:0/2684995638 wait complete. 2024-04-25T00:25:19.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:19.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1099693494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 msgr2=0x7fa19c103e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1099693494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c103e00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa190009930 tx=0x7fa19002a420 comp rx=0 tx=0).stop 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1099693494 shutdown_connections 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1099693494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 0x7fa19c110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1099693494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c103e00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1099693494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1099693494 >> 172.21.15.45:0/1099693494 conn(0x7fa19c0ff4d0 msgr2=0x7fa19c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1099693494 shutdown_connections 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1099693494 wait complete. 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 Processor -- start 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.385+0000 7fa1a12bd640 1 -- start start 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 0x7fa19c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c113da0 con 0x7fa19c103980 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c113c20 con 0x7fa19c109990 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c113f20 con 0x7fa19c104340 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60352/0 (socket says 172.21.15.45:60352) 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 -- 172.21.15.45:0/1203481160 learned_addr learned my addr 172.21.15.45:0/1203481160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 0x7fa19c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 -- 172.21.15.45:0/1203481160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 msgr2=0x7fa19c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 0x7fa19c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 -- 172.21.15.45:0/1203481160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 msgr2=0x7fa19c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 -- 172.21.15.45:0/1203481160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c19f2c0 con 0x7fa19c103980 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa19affd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00d950 tx=0x7fa18c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa183fff640 1 -- 172.21.15.45:0/1203481160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c017070 con 0x7fa19c103980 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c19f5b0 con 0x7fa19c103980 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa183fff640 1 -- 172.21.15.45:0/1203481160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa18c009c30 con 0x7fa19c103980 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.386+0000 7fa183fff640 1 -- 172.21.15.45:0/1203481160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00f6b0 con 0x7fa19c103980 2024-04-25T00:25:19.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.387+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c1a6e40 con 0x7fa19c103980 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa183fff640 1 -- 172.21.15.45:0/1203481160 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa18c0109d0 con 0x7fa19c103980 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 Processor -- start 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 -- start start 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbef50 0x561f37bc90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bcba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.389+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f37b236b0 con 0x561f37bbe630 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f37acb880 con 0x561f37bc9620 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f37b1f670 con 0x561f37bbef50 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60368/0 (socket says 172.21.15.45:60368) 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 -- 172.21.15.45:0/3431243127 learned_addr learned my addr 172.21.15.45:0/3431243127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa183fff640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 0x7fa1840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19affd640 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbef50 0x561f37bc90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bcba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 0x7fa1840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 -- 172.21.15.45:0/3431243127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbef50 msgr2=0x561f37bc90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbef50 0x561f37bc90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 -- 172.21.15.45:0/3431243127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 msgr2=0x561f37bcba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bcba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 -- 172.21.15.45:0/3431243127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f379017d0 con 0x561f37bbe630 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa183fff640 1 -- 172.21.15.45:0/1203481160 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa18c0cf450 con 0x7fa19c103980 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa194005c40 tx=0x7fa19400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c15eac3158b1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa1817fa640 1 -- 172.21.15.45:0/3431243127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194023020 con 0x561f37bbe630 2024-04-25T00:25:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa1817fa640 1 -- 172.21.15.45:0/3431243127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19400eda0 con 0x561f37bbe630 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa1817fa640 1 -- 172.21.15.45:0/3431243127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400fb30 con 0x561f37bbe630 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.390+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 0x7fa1840a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa194024000 tx=0x7fa194022000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3431243127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 msgr2=0x561f37bbea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa194005c40 tx=0x7fa19400e350 comp rx=0 tx=0).stop 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3431243127 shutdown_connections 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bcba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbef50 0x561f37bc90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3431243127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbe630 0x561f37bbea10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3431243127 >> 172.21.15.45:0/3431243127 conn(0x561f37bba950 msgr2=0x561f37bbad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3431243127 shutdown_connections 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3431243127 wait complete. 2024-04-25T00:25:19.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.391+0000 7fa1a3fd2740 1 -- start start 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 Processor -- start 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f37bcc410 con 0x561f37bbef50 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f37adef50 con 0x561f37bc9620 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f37bcc590 con 0x561f37bbe630 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa1a3fd2740 10 client.0 _ll_register_callbacks cb 0x561f37b1a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60378/0 (socket says 172.21.15.45:60378) 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 -- 172.21.15.45:0/3138516723 learned_addr learned my addr 172.21.15.45:0/3138516723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19affd640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 -- 172.21.15.45:0/3138516723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 msgr2=0x561f37bc6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 -- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 msgr2=0x561f37bc6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 -- 172.21.15.45:0/3138516723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f378373f0 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19affd640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a940 tx=0x7fa1900317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b792210ad728f7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.392+0000 7fa17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035040 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa190004240 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19400eda0 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa190031a10 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 10 client.16689 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f377f70f0 con 0x561f37bbef50 2024-04-25T00:25:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19400eda0 con 0x561f37bbef50 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa190004ac0 con 0x561f37bbef50 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 10 client.16689 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 20 client.16689 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37bd98c0 0x561f37bdbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 4 client.16689 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f37b30bf0 con 0x561f37bd98c0 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.393+0000 7fa19affd640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37bd98c0 0x561f37bdbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.394+0000 7fa19affd640 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37bd98c0 0x561f37bdbcd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2658fb261e3db59e server_cookie=65c62d92d49cd47c in_seq=0 out_seq=0 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.394+0000 7fa17eff5640 10 client.16689 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.394+0000 7fa17eff5640 1 -- 172.21.15.45:0/3138516723 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f37b30bf0 con 0x561f37bd98c0 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.394+0000 7fa17eff5640 10 client.16689 handle_command_reply: tid=0 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.394+0000 7fa1a3fd2740 1 client.16689 shutdown 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 msgr2=0x561f37bc6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a940 tx=0x7fa1900317e0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 shutdown_connections 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37bd98c0 0x561f37bdbcd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bc6ab0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/3138516723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa17eff5640 0 client.16689 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 >> 172.21.15.45:0/3138516723 conn(0x561f37bba950 msgr2=0x561f37bc0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 shutdown_connections 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/3138516723 wait complete. 2024-04-25T00:25:19.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.395+0000 7fa1a3fd2740 20 client.16689 trim_cache size 0 max 16384 2024-04-25T00:25:19.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.412+0000 7fa1a3fd2740 1 Processor -- start 2024-04-25T00:25:19.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.412+0000 7fa1a3fd2740 1 -- start start 2024-04-25T00:25:19.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.412+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1b0da0 con 0x561f37bbef50 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1b28e0 con 0x561f37bc9620 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1b1470 con 0x561f37bbe630 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60388/0 (socket says 172.21.15.45:60388) 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 -- 172.21.15.45:0/923630331 learned_addr learned my addr 172.21.15.45:0/923630331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19affd640 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 -- 172.21.15.45:0/923630331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 msgr2=0x561f37bc8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 -- 172.21.15.45:0/923630331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 msgr2=0x561f37bc5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 -- 172.21.15.45:0/923630331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f37bbdc40 con 0x561f37bbef50 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa190004ec0 tx=0x7fa19002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdc658bbfae7b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa180ff9640 1 -- 172.21.15.45:0/923630331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035070 con 0x561f37bbef50 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa180ff9640 1 -- 172.21.15.45:0/923630331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa190009520 con 0x561f37bbef50 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.413+0000 7fa180ff9640 1 -- 172.21.15.45:0/923630331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900055c0 con 0x561f37bbef50 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/923630331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 msgr2=0x561f37bd52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa190004ec0 tx=0x7fa19002a420 comp rx=0 tx=0).stop 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/923630331 shutdown_connections 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bc9620 0x561f37bc5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd52d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/923630331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bbe630 0x561f37bc8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/923630331 >> 172.21.15.45:0/923630331 conn(0x561f37bba950 msgr2=0x561f37bc4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/923630331 shutdown_connections 2024-04-25T00:25:19.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/923630331 wait complete. 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 -- start start 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 Processor -- start 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bbe630 0x561f37bd17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.414+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bc9620 0x561f37bc2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1b2210 con 0x561f37bbef50 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1b1b40 con 0x561f37bbe630 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c1aed80 con 0x561f37bc9620 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 10 client.0 _ll_register_callbacks cb 0x7fa190009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60400/0 (socket says 172.21.15.45:60400) 2024-04-25T00:25:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 -- 172.21.15.45:0/619134506 learned_addr learned my addr 172.21.15.45:0/619134506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19affd640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bbe630 0x561f37bd17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19b7fe640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bc9620 0x561f37bc2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 -- 172.21.15.45:0/619134506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bc9620 msgr2=0x561f37bc2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bc9620 0x561f37bc2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 -- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bbe630 msgr2=0x561f37bd17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bbe630 0x561f37bd17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 -- 172.21.15.45:0/619134506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f37bc2fc0 con 0x561f37bbef50 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa19a7fc640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa190009850 tx=0x7fa190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e130c68f0e04f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19005e070 con 0x561f37bbef50 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa190009520 con 0x561f37bbef50 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa190031db0 con 0x561f37bbef50 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa190067050 con 0x561f37bbef50 2024-04-25T00:25:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 10 client.16695 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.415+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f37c3f7e0 con 0x561f37bbef50 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa190009520 con 0x561f37bbef50 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa190061020 con 0x561f37bbef50 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 10 client.16695 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 20 client.16695 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37c3fe40 0x561f37c42230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 4 client.16695 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f37b2af40 con 0x561f37c3fe40 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa19affd640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37c3fe40 0x561f37c42230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa19affd640 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37c3fe40 0x561f37c42230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397788074729e6ea server_cookie=ee066c3a1740bd0e in_seq=0 out_seq=0 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.416+0000 7fa17eff5640 10 client.16695 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa17eff5640 1 -- 172.21.15.45:0/619134506 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561f37b2af40 con 0x561f37c3fe40 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa17eff5640 10 client.16695 handle_command_reply: tid=0 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 client.16695 shutdown 2024-04-25T00:25:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 msgr2=0x561f37bd1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa190009850 tx=0x7fa190000bc0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 shutdown_connections 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f37bc9620 0x561f37bc2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561f37c3fe40 0x561f37c42230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f37bbef50 0x561f37bd1d20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 --2- 172.21.15.45:0/619134506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37bbe630 0x561f37bd17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa17eff5640 0 client.16695 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 >> 172.21.15.45:0/619134506 conn(0x561f37bba950 msgr2=0x561f37bc56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 shutdown_connections 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 1 -- 172.21.15.45:0/619134506 wait complete. 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a3fd2740 20 client.16695 trim_cache size 0 max 16384 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "b9c51c98-8a7c-4d80-b886-1db496af515a": { 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "b9c51c98-8a7c-4d80-b886-1db496af515a", 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:25:19.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:25:19.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:25:19.426 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 msgr2=0x7fa1840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 0x7fa1840a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa194024000 tx=0x7fa194022000 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.417+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 msgr2=0x7fa19c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00d950 tx=0x7fa18c00de10 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 shutdown_connections 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1840a2d00 0x7fa1840a51c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa19c109990 0x7fa19c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa19c104340 0x7fa19c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 --2- 172.21.15.45:0/1203481160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa19c103980 0x7fa19c19a320 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 >> 172.21.15.45:0/1203481160 conn(0x7fa19c0ff4d0 msgr2=0x7fa19c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 shutdown_connections 2024-04-25T00:25:19.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:19.418+0000 7fa1a12bd640 1 -- 172.21.15.45:0/1203481160 wait complete. 2024-04-25T00:25:19.429 INFO:tasks.cephfs.filesystem:scrub status for tag:b9c51c98-8a7c-4d80-b886-1db496af515a - {'path': '/', 'tag': 'b9c51c98-8a7c-4d80-b886-1db496af515a', 'options': 'recursive,force'} 2024-04-25T00:25:20.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:20 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:25:20.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:20 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:20.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:20 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:25:20.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:20 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:20.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:20 smithi071 ceph-mon[32667]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:25:20.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:20 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:25:20.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:20 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:20.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:20 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:25:20.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:20 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:20.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:20 smithi045 ceph-mon[29194]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:25:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:20 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:25:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:20 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:20 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:25:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:20 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:20 smithi122 ceph-mon[32959]: pgmap v104: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:25:22.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:22 smithi071 ceph-mon[32667]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:22.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:22 smithi045 ceph-mon[29194]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:23.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:22 smithi122 ceph-mon[32959]: pgmap v105: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:23.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:23 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:25:23.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:23 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:25:24.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:23 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:25:24.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:24 smithi071 ceph-mon[32667]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:24.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:24 smithi045 ceph-mon[29194]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:24 smithi122 ceph-mon[32959]: pgmap v106: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:26.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:26 smithi071 ceph-mon[32667]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:26.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:26 smithi045 ceph-mon[29194]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:27.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:26 smithi122 ceph-mon[32959]: pgmap v107: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:28.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:28 smithi071 ceph-mon[32667]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:28.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:28 smithi045 ceph-mon[29194]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:28 smithi122 ceph-mon[32959]: pgmap v108: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:30.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:30 smithi071 ceph-mon[32667]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:25:30.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:30 smithi045 ceph-mon[29194]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:25:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:30 smithi122 ceph-mon[32959]: pgmap v109: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:25:32.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:32 smithi071 ceph-mon[32667]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:32.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:32 smithi045 ceph-mon[29194]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:32 smithi122 ceph-mon[32959]: pgmap v110: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:34 smithi071 ceph-mon[32667]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:25:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:34 smithi045 ceph-mon[29194]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:25:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:34 smithi122 ceph-mon[32959]: pgmap v111: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:25:36.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:36 smithi071 ceph-mon[32667]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:36 smithi045 ceph-mon[29194]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:36 smithi122 ceph-mon[32959]: pgmap v112: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:38.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:38 smithi071 ceph-mon[32667]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:38.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:38 smithi045 ceph-mon[29194]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:38 smithi122 ceph-mon[32959]: pgmap v113: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:40.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:40 smithi071 ceph-mon[32667]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:40.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:40 smithi045 ceph-mon[29194]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:40 smithi122 ceph-mon[32959]: pgmap v114: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:42 smithi045 ceph-mon[29194]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:43.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:42 smithi122 ceph-mon[32959]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:43.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:42 smithi071 ceph-mon[32667]: pgmap v115: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:44.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:44 smithi045 ceph-mon[29194]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:25:45.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:44 smithi122 ceph-mon[32959]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:25:45.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:44 smithi071 ceph-mon[32667]: pgmap v116: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:25:46.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:46 smithi045 ceph-mon[29194]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:25:47.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:46 smithi122 ceph-mon[32959]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:25:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:46 smithi071 ceph-mon[32667]: pgmap v117: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:25:48.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:48 smithi045 ceph-mon[29194]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:49.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:48 smithi122 ceph-mon[32959]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:49.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:48 smithi071 ceph-mon[32667]: pgmap v118: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:49.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:49.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 -- 172.21.15.45:0/2288543426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824107720 msgr2=0x7f2824107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 --2- 172.21.15.45:0/2288543426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824107720 0x7f2824107b00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2814009930 tx=0x7f281402a420 comp rx=0 tx=0).stop 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 -- 172.21.15.45:0/2288543426 shutdown_connections 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 --2- 172.21.15.45:0/2288543426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28241020d0 0x7f282410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 --2- 172.21.15.45:0/2288543426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2824101710 0x7f2824101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 --2- 172.21.15.45:0/2288543426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824107720 0x7f2824107b00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 -- 172.21.15.45:0/2288543426 >> 172.21.15.45:0/2288543426 conn(0x7f28240fd180 msgr2=0x7f28240ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.547+0000 7f282977e640 1 -- 172.21.15.45:0/2288543426 shutdown_connections 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 -- 172.21.15.45:0/2288543426 wait complete. 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 Processor -- start 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 -- start start 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28241020d0 0x7f2824073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2824107720 0x7f28240744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2824111b30 con 0x7f2824101710 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28241119b0 con 0x7f28241020d0 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.548+0000 7f282977e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2824111cb0 con 0x7f2824107720 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51672/0 (socket says 172.21.15.45:51672) 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 -- 172.21.15.45:0/3158557453 learned_addr learned my addr 172.21.15.45:0/3158557453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f281bfff640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28241020d0 0x7f2824073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f28237fe640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2824107720 0x7f28240744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 -- 172.21.15.45:0/3158557453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2824107720 msgr2=0x7f28240744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2824107720 0x7f28240744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 -- 172.21.15.45:0/3158557453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28241020d0 msgr2=0x7f2824073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28241020d0 0x7f2824073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 -- 172.21.15.45:0/3158557453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28241a6530 con 0x7f2824101710 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2822ffd640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2814009230 tx=0x7f2814004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2820ff9640 1 -- 172.21.15.45:0/3158557453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281403f030 con 0x7f2824101710 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2820ff9640 1 -- 172.21.15.45:0/3158557453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2814003ad0 con 0x7f2824101710 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f2820ff9640 1 -- 172.21.15.45:0/3158557453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814035070 con 0x7f2824101710 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28241a67c0 con 0x7f2824101710 2024-04-25T00:25:49.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.549+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28241a6d00 con 0x7f2824101710 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.551+0000 7f2820ff9640 1 -- 172.21.15.45:0/3158557453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2814042030 con 0x7f2824101710 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 Processor -- start 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 -- start start 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 0x55f208a48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f2089ad6b0 con 0x55f208a53620 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f208955880 con 0x55f208a48630 2024-04-25T00:25:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.552+0000 7f282c56a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f2089a9670 con 0x55f208a48f50 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51680/0 (socket says 172.21.15.45:51680) 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f2820ff9640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 0x7f28100a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 -- 172.21.15.45:0/2317549691 learned_addr learned my addr 172.21.15.45:0/2317549691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f28237fe640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 0x55f208a48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f28237fe640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 0x7f28100a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f2820ff9640 1 -- 172.21.15.45:0/3158557453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f28140efd30 con 0x7f2824101710 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 -- 172.21.15.45:0/2317549691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 msgr2=0x55f208a530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 -- 172.21.15.45:0/2317549691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 msgr2=0x55f208a48a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 0x55f208a48a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 -- 172.21.15.45:0/2317549691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20878b7d0 con 0x55f208a53620 2024-04-25T00:25:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f28237fe640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 0x55f208a48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f281bfff640 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f280c00bc80 tx=0x7f280c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322456ef16fe24e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f28237fe640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 0x7f28100a5270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f281c005fd0 tx=0x7f281c004ab0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.553+0000 7f28197fa640 1 -- 172.21.15.45:0/2317549691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c015070 con 0x55f208a53620 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f28197fa640 1 -- 172.21.15.45:0/2317549691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c011920 con 0x55f208a53620 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f28197fa640 1 -- 172.21.15.45:0/2317549691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0107f0 con 0x55f208a53620 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 -- 172.21.15.45:0/2317549691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 msgr2=0x55f208a55a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f280c00bc80 tx=0x7f280c00f730 comp rx=0 tx=0).stop 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 -- 172.21.15.45:0/2317549691 shutdown_connections 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a53620 0x55f208a55a10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 --2- 172.21.15.45:0/2317549691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48630 0x55f208a48a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 -- 172.21.15.45:0/2317549691 >> 172.21.15.45:0/2317549691 conn(0x55f208a44950 msgr2=0x55f208a44d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 -- 172.21.15.45:0/2317549691 shutdown_connections 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.554+0000 7f282c56a740 1 -- 172.21.15.45:0/2317549691 wait complete. 2024-04-25T00:25:49.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 -- start start 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 Processor -- start 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a50f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f208a56460 con 0x55f208a48630 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f208968f50 con 0x55f208a48f50 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f208a565e0 con 0x55f208a53620 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f282c56a740 10 client.0 _ll_register_callbacks cb 0x55f2089a4000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51690/0 (socket says 172.21.15.45:51690) 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f2822ffd640 1 -- 172.21.15.45:0/2287986353 learned_addr learned my addr 172.21.15.45:0/2287986353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f28237fe640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a50f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f281bfff640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f2822ffd640 1 -- 172.21.15.45:0/2287986353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 msgr2=0x55f208a50f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.555+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a50f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f2822ffd640 1 -- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 msgr2=0x55f208a509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f2822ffd640 1 -- 172.21.15.45:0/2287986353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2086c13f0 con 0x55f208a48630 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f28140b8860 tx=0x7f281408bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f51629592fa57ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814049040 con 0x55f208a48630 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f28140b8500 con 0x55f208a48630 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c011920 con 0x55f208a48630 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28140bbe90 con 0x55f208a48630 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f282c56a740 10 client.16707 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:49.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2086810f0 con 0x55f208a48630 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c011920 con 0x55f208a48630 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2814031bd0 con 0x55f208a48630 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.556+0000 7f282c56a740 10 client.16707 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f282c56a740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f282c56a740 20 client.16707 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208a63790 0x55f208a65c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f282c56a740 4 client.16707 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2089babf0 con 0x55f208a63790 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f281bfff640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208a63790 0x55f208a65c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f281bfff640 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208a63790 0x55f208a65c30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7306ed99cbc100e server_cookie=b96b4b9a00b673e6 in_seq=0 out_seq=0 2024-04-25T00:25:49.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.557+0000 7f280a7fc640 10 client.16707 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f280a7fc640 1 -- 172.21.15.45:0/2287986353 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2089babf0 con 0x55f208a63790 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f280a7fc640 10 client.16707 handle_command_reply: tid=0 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 client.16707 shutdown 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 msgr2=0x55f208a504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f28140b8860 tx=0x7f281408bcf0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 shutdown_connections 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a50f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208a63790 0x55f208a65c30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 --2- 172.21.15.45:0/2287986353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a504a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 >> 172.21.15.45:0/2287986353 conn(0x55f208a44950 msgr2=0x55f208a4a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f280a7fc640 0 client.16707 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:49.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 shutdown_connections 2024-04-25T00:25:49.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 1 -- 172.21.15.45:0/2287986353 wait complete. 2024-04-25T00:25:49.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.558+0000 7f282c56a740 20 client.16707 trim_cache size 0 max 16384 2024-04-25T00:25:49.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.576+0000 7f282c56a740 1 Processor -- start 2024-04-25T00:25:49.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.576+0000 7f282c56a740 1 -- start start 2024-04-25T00:25:49.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.576+0000 7f282c56a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f282c56a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f282c56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 0x55f208a43970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f282c56a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b0d00 con 0x55f208a48630 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f282c56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b2840 con 0x55f208a53620 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f282c56a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b13d0 con 0x55f208a48f50 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51700/0 (socket says 172.21.15.45:51700) 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 -- 172.21.15.45:0/2463521892 learned_addr learned my addr 172.21.15.45:0/2463521892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f28237fe640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 0x55f208a43970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f281bfff640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 -- 172.21.15.45:0/2463521892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 msgr2=0x55f208a5f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a5f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 -- 172.21.15.45:0/2463521892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 msgr2=0x55f208a43970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 0x55f208a43970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 -- 172.21.15.45:0/2463521892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f208a44780 con 0x55f208a48630 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f28237fe640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 0x55f208a43970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2822ffd640 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f28140bbb90 tx=0x7f28140045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9541d7d9d7f4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2818ff9640 1 -- 172.21.15.45:0/2463521892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814049070 con 0x55f208a48630 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2818ff9640 1 -- 172.21.15.45:0/2463521892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28140d8050 con 0x55f208a48630 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.577+0000 7f2818ff9640 1 -- 172.21.15.45:0/2463521892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140b8620 con 0x55f208a48630 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- 172.21.15.45:0/2463521892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 msgr2=0x55f208a52bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f28140bbb90 tx=0x7f28140045f0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- 172.21.15.45:0/2463521892 shutdown_connections 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a53620 0x55f208a43970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a48f50 0x55f208a5f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 --2- 172.21.15.45:0/2463521892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a52bd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- 172.21.15.45:0/2463521892 >> 172.21.15.45:0/2463521892 conn(0x55f208a44950 msgr2=0x55f208a43260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- 172.21.15.45:0/2463521892 shutdown_connections 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- 172.21.15.45:0/2463521892 wait complete. 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 -- start start 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.578+0000 7f282c56a740 1 Processor -- start 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a56f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b2170 con 0x55f208a48630 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28241b1aa0 con 0x55f208a48f50 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28241aece0 con 0x55f208a53620 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 10 client.0 _ll_register_callbacks cb 0x7f2814009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51708/0 (socket says 172.21.15.45:51708) 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 -- 172.21.15.45:0/1395736810 learned_addr learned my addr 172.21.15.45:0/1395736810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f281bfff640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a56f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f28237fe640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 -- 172.21.15.45:0/1395736810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 msgr2=0x55f208a574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 -- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 msgr2=0x55f208a56f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a56f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 -- 172.21.15.45:0/1395736810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f208a57cb0 con 0x55f208a48630 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f2822ffd640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f28140c45d0 tx=0x7f28140ed550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd8a4397cac592b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f280a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:49.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28140ea070 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f28140b8600 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.579+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28140d8050 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2814049070 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 10 client.16713 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f208a57cb0 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28140d8050 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28140b7be0 con 0x55f208a48630 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 10 client.16713 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 20 client.16713 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208aca0a0 0x55f208acc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 4 client.16713 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2089b4f40 con 0x55f208aca0a0 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f281bfff640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208aca0a0 0x55f208acc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f281bfff640 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208aca0a0 0x55f208acc490 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c96fd738758c375 server_cookie=1ccdcb6d2296355a in_seq=0 out_seq=0 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.580+0000 7f280a7fc640 10 client.16713 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f280a7fc640 1 -- 172.21.15.45:0/1395736810 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f2089b4f40 con 0x55f208aca0a0 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f280a7fc640 10 client.16713 handle_command_reply: tid=0 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 client.16713 shutdown 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 msgr2=0x55f208a5bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f28140c45d0 tx=0x7f28140ed550 comp rx=0 tx=0).stop 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 shutdown_connections 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f208a53620 0x55f208a574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f208a48f50 0x55f208a56f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f208aca0a0 0x55f208acc490 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 --2- 172.21.15.45:0/1395736810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f208a48630 0x55f208a5bbb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f280a7fc640 0 client.16713 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 >> 172.21.15.45:0/1395736810 conn(0x55f208a44950 msgr2=0x55f208a50cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 shutdown_connections 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 1 -- 172.21.15.45:0/1395736810 wait complete. 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282c56a740 20 client.16713 trim_cache size 0 max 16384 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:25:49.591 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:25:49.592 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 msgr2=0x7f28100a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 0x7f28100a5270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f281c005fd0 tx=0x7f281c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 msgr2=0x7f2824073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2814009230 tx=0x7f2814004270 comp rx=0 tx=0).stop 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 shutdown_connections 2024-04-25T00:25:49.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f28100a2db0 0x7f28100a5270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2824107720 0x7f28240744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28241020d0 0x7f2824073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 --2- 172.21.15.45:0/3158557453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2824101710 0x7f2824073a40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.581+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 >> 172.21.15.45:0/3158557453 conn(0x7f28240fd180 msgr2=0x7f28240fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.582+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 shutdown_connections 2024-04-25T00:25:49.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.582+0000 7f282977e640 1 -- 172.21.15.45:0/3158557453 wait complete. 2024-04-25T00:25:49.594 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:25:49.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:25:49.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.696+0000 7ffa18161640 1 -- 172.21.15.45:0/2799753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa1010bd60 msgr2=0x7ffa1010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.696+0000 7ffa18161640 1 --2- 172.21.15.45:0/2799753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa1010bd60 0x7ffa1010e1d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009ca0 tx=0x7ffa0802a950 comp rx=0 tx=0).stop 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 -- 172.21.15.45:0/2799753712 shutdown_connections 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- 172.21.15.45:0/2799753712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa1010bd60 0x7ffa1010e1d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- 172.21.15.45:0/2799753712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10103340 0x7ffa1010b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- 172.21.15.45:0/2799753712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa10102a20 0x7ffa10102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 -- 172.21.15.45:0/2799753712 >> 172.21.15.45:0/2799753712 conn(0x7ffa100777a0 msgr2=0x7ffa10077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 -- 172.21.15.45:0/2799753712 shutdown_connections 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 -- 172.21.15.45:0/2799753712 wait complete. 2024-04-25T00:25:49.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 Processor -- start 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 -- start start 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 0x7ffa10197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.697+0000 7ffa18161640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1010bd60 0x7ffa1019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa18161640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10111b30 con 0x7ffa10103340 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa18161640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101119b0 con 0x7ffa10102a20 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa18161640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10111cb0 con 0x7ffa1010bd60 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51734/0 (socket says 172.21.15.45:51734) 2024-04-25T00:25:49.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 -- 172.21.15.45:0/3291695395 learned_addr learned my addr 172.21.15.45:0/3291695395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa15efd640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 0x7ffa10197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa166fe640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1010bd60 0x7ffa1019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 -- 172.21.15.45:0/3291695395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1010bd60 msgr2=0x7ffa1019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1010bd60 0x7ffa1019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 -- 172.21.15.45:0/3291695395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 msgr2=0x7ffa10197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 0x7ffa10197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 -- 172.21.15.45:0/3291695395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019cfa0 con 0x7ffa10103340 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa15efd640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 0x7ffa10197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa156fc640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000d950 tx=0x7ffa0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00017070 con 0x7ffa10103340 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa00009c30 con 0x7ffa10103340 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019d290 con 0x7ffa10103340 2024-04-25T00:25:49.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000f6b0 con 0x7ffa10103340 2024-04-25T00:25:49.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.698+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a4b20 con 0x7ffa10103340 2024-04-25T00:25:49.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.699+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa100689b0 con 0x7ffa10103340 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.700+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa000109d0 con 0x7ffa10103340 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.701+0000 7ffa06ffd640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 0x7ff9fc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.701+0000 7ffa15efd640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 0x7ff9fc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.701+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ffa000cf390 con 0x7ffa10103340 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.701+0000 7ffa15efd640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 0x7ff9fc0a5240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009c20 tx=0x7ffa0c005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.705+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa00009de0 con 0x7ffa10103340 2024-04-25T00:25:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.930+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa10106950 con 0x7ffa10103340 2024-04-25T00:25:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.930+0000 7ffa06ffd640 1 -- 172.21.15.45:0/3291695395 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7ffa0009bcd0 con 0x7ffa10103340 2024-04-25T00:25:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:25:49.931 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:25:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:25:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 msgr2=0x7ff9fc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 0x7ff9fc0a5240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009c20 tx=0x7ffa0c005f90 comp rx=0 tx=0).stop 2024-04-25T00:25:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 msgr2=0x7ffa101984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000d950 tx=0x7ffa0000de10 comp rx=0 tx=0).stop 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 shutdown_connections 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9fc0a2d80 0x7ff9fc0a5240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa1010bd60 0x7ffa1019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10103340 0x7ffa101984f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 --2- 172.21.15.45:0/3291695395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa10102a20 0x7ffa10197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 >> 172.21.15.45:0/3291695395 conn(0x7ffa100777a0 msgr2=0x7ffa100fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 shutdown_connections 2024-04-25T00:25:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:49.932+0000 7ffa18161640 1 -- 172.21.15.45:0/3291695395 wait complete. 2024-04-25T00:25:49.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:25:50.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 -- 172.21.15.45:0/2644764927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100620 msgr2=0x7f60e4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 --2- 172.21.15.45:0/2644764927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100620 0x7f60e4100aa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009930 tx=0x7f60d402a420 comp rx=0 tx=0).stop 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 -- 172.21.15.45:0/2644764927 shutdown_connections 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 --2- 172.21.15.45:0/2644764927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4100fe0 0x7f60e410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 --2- 172.21.15.45:0/2644764927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100620 0x7f60e4100aa0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 --2- 172.21.15.45:0/2644764927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4106630 0x7f60e4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 -- 172.21.15.45:0/2644764927 >> 172.21.15.45:0/2644764927 conn(0x7f60e40776a0 msgr2=0x7f60e4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 -- 172.21.15.45:0/2644764927 shutdown_connections 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.048+0000 7f60ea914640 1 -- 172.21.15.45:0/2644764927 wait complete. 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 Processor -- start 2024-04-25T00:25:50.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 -- start start 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4100620 0x7f60e4196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 0x7f60e419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4110a40 con 0x7f60e4100fe0 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41108c0 con 0x7f60e4106630 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60ea914640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4110bc0 con 0x7f60e4100620 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51746/0 (socket says 172.21.15.45:51746) 2024-04-25T00:25:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60e37fe640 1 -- 172.21.15.45:0/1088175660 learned_addr learned my addr 172.21.15.45:0/1088175660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.049+0000 7f60e3fff640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4100620 0x7f60e4196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 0x7f60e419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 -- 172.21.15.45:0/1088175660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4100620 msgr2=0x7f60e4196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4100620 0x7f60e4196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 -- 172.21.15.45:0/1088175660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 msgr2=0x7f60e419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 0x7f60e419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 -- 172.21.15.45:0/1088175660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e419bf50 con 0x7f60e4100fe0 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 0x7f60e419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e17fa640 1 -- 172.21.15.45:0/1088175660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4046070 con 0x7f60e4100fe0 2024-04-25T00:25:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e17fa640 1 -- 172.21.15.45:0/1088175660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4003ad0 con 0x7f60e4100fe0 2024-04-25T00:25:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60e17fa640 1 -- 172.21.15.45:0/1088175660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4035050 con 0x7f60e4100fe0 2024-04-25T00:25:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e419c1e0 con 0x7f60e4100fe0 2024-04-25T00:25:50.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.050+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e4069d40 con 0x7f60e4100fe0 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.052+0000 7f60e17fa640 1 -- 172.21.15.45:0/1088175660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60d4049070 con 0x7f60e4100fe0 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 Processor -- start 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 -- start start 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef2313ea0 0x564ef231e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef231e570 0x564ef2320960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ef2278670 con 0x564ef2313580 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564ef2220970 con 0x564ef231e570 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60ed625740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564ef2274630 con 0x564ef2313ea0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e17fa640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 0x7f60cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e8eb1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e8eb1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51754/0 (socket says 172.21.15.45:51754) 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e8eb1640 1 -- 172.21.15.45:0/3032499550 learned_addr learned my addr 172.21.15.45:0/3032499550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e17fa640 1 -- 172.21.15.45:0/1088175660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60d40f1330 con 0x7f60e4100fe0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 0x7f60cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e37fe640 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef231e570 0x564ef2320960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.053+0000 7f60e3fff640 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef2313ea0 0x564ef231e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 -- 172.21.15.45:0/3032499550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef2313ea0 msgr2=0x564ef231e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef2313ea0 0x564ef231e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 -- 172.21.15.45:0/3032499550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef231e570 msgr2=0x564ef2320960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef231e570 0x564ef2320960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 -- 172.21.15.45:0/3032499550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef2244400 con 0x564ef2313580 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60dc012a00 tx=0x7f60dc012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618199d2012e98f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60ca7fc640 1 -- 172.21.15.45:0/3032499550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc022070 con 0x564ef2313580 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60ca7fc640 1 -- 172.21.15.45:0/3032499550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60dc004b60 con 0x564ef2313580 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 0x7f60cc0a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01f000 tx=0x7f60dc01b000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.054+0000 7f60ca7fc640 1 -- 172.21.15.45:0/3032499550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00f420 con 0x564ef2313580 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- 172.21.15.45:0/3032499550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 msgr2=0x564ef2313960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60dc012a00 tx=0x7f60dc012ec0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- 172.21.15.45:0/3032499550 shutdown_connections 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef231e570 0x564ef2320960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef2313ea0 0x564ef231e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 --2- 172.21.15.45:0/3032499550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef2313960 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- 172.21.15.45:0/3032499550 >> 172.21.15.45:0/3032499550 conn(0x564ef230f8a0 msgr2=0x564ef230fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- 172.21.15.45:0/3032499550 shutdown_connections 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- 172.21.15.45:0/3032499550 wait complete. 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 -- start start 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.055+0000 7f60ed625740 1 Processor -- start 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313580 0x564ef231df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ef2321360 con 0x564ef2313ea0 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564ef2234050 con 0x564ef2313580 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564ef23214e0 con 0x564ef231e570 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 10 client.0 _ll_register_callbacks cb 0x564ef1f8bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313580 0x564ef231df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51764/0 (socket says 172.21.15.45:51764) 2024-04-25T00:25:50.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 -- 172.21.15.45:0/2625051228 learned_addr learned my addr 172.21.15.45:0/2625051228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 -- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 msgr2=0x564ef231b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 -- 172.21.15.45:0/2625051228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313580 msgr2=0x564ef231df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313580 0x564ef231df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 -- 172.21.15.45:0/2625051228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef2205dc0 con 0x564ef2313ea0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60e37fe640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009a50 tx=0x7f60d40bd260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273665a42bbe82e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40eeb40 con 0x564ef2313ea0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60d4031bd0 con 0x564ef2313ea0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.056+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60dc004b60 con 0x564ef2313ea0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60d4038470 con 0x564ef2313ea0 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 10 client.16731 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ef1f8c3f0 con 0x564ef2313ea0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60dc004b60 con 0x564ef2313ea0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60d40d6220 con 0x564ef2313ea0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 10 client.16731 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 10 client.16731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 20 client.16731 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef232e5e0 0x564ef2330a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 4 client.16731 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef2285bb0 con 0x564ef232e5e0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.057+0000 7f60e3fff640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef232e5e0 0x564ef2330a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.058+0000 7f60c37fe640 10 client.16731 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.058+0000 7f60e3fff640 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef232e5e0 0x564ef2330a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=748652aa6b22e48e server_cookie=599138b4b40065cb in_seq=0 out_seq=0 2024-04-25T00:25:50.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.058+0000 7f60c37fe640 1 -- 172.21.15.45:0/2625051228 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ef2285bb0 con 0x564ef232e5e0 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.058+0000 7f60c37fe640 10 client.16731 handle_command_reply: tid=0 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.058+0000 7f60ed625740 1 client.16731 shutdown 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 msgr2=0x564ef231b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009a50 tx=0x7f60d40bd260 comp rx=0 tx=0).stop 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 shutdown_connections 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef232e5e0 0x564ef2330a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313ea0 0x564ef231b250 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 --2- 172.21.15.45:0/2625051228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313580 0x564ef231df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60c37fe640 0 client.16731 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 >> 172.21.15.45:0/2625051228 conn(0x564ef230f8a0 msgr2=0x564ef23156a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 shutdown_connections 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 1 -- 172.21.15.45:0/2625051228 wait complete. 2024-04-25T00:25:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.059+0000 7f60ed625740 20 client.16731 trim_cache size 0 max 16384 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 Processor -- start 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 -- start start 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef2326b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41ac7a0 con 0x564ef2313580 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41ae2e0 con 0x564ef2313ea0 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.072+0000 7f60ed625740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41ace70 con 0x564ef231e570 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51774/0 (socket says 172.21.15.45:51774) 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 -- 172.21.15.45:0/3435435000 learned_addr learned my addr 172.21.15.45:0/3435435000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e37fe640 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef2326b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 -- 172.21.15.45:0/3435435000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 msgr2=0x564ef231da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 -- 172.21.15.45:0/3435435000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 msgr2=0x564ef2326b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef2326b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 -- 172.21.15.45:0/3435435000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef20567d0 con 0x564ef2313580 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60e3fff640 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x564ef2331250 tx=0x7f60d8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3809dd99bf40994c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3435435000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8023070 con 0x564ef2313580 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3435435000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d80047a0 con 0x564ef2313580 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.073+0000 7f60c9ffb640 1 -- 172.21.15.45:0/3435435000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d801f3d0 con 0x564ef2313580 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- 172.21.15.45:0/3435435000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 msgr2=0x564ef232a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x564ef2331250 tx=0x7f60d8010800 comp rx=0 tx=0).stop 2024-04-25T00:25:50.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- 172.21.15.45:0/3435435000 shutdown_connections 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef231da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef2326b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 --2- 172.21.15.45:0/3435435000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef232a220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- 172.21.15.45:0/3435435000 >> 172.21.15.45:0/3435435000 conn(0x564ef230f8a0 msgr2=0x564ef2316740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- 172.21.15.45:0/3435435000 shutdown_connections 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- 172.21.15.45:0/3435435000 wait complete. 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 -- start start 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.074+0000 7f60ed625740 1 Processor -- start 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef238d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef2397bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41adc10 con 0x564ef2313580 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41ad540 con 0x564ef2313ea0 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41aa780 con 0x564ef231e570 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 10 client.0 _ll_register_callbacks cb 0x7f60d4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51784/0 (socket says 172.21.15.45:51784) 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 -- 172.21.15.45:0/1167980572 learned_addr learned my addr 172.21.15.45:0/1167980572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e8eb1640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef2397bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef238d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 -- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 msgr2=0x564ef2397bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef2397bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 -- 172.21.15.45:0/1167980572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 msgr2=0x564ef238d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef238d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 -- 172.21.15.45:0/1167980572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef2398150 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef238d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60e3fff640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009ca0 tx=0x7f60d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1b0e29ee125611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8023040 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60d8005020 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d80047a0 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.075+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60d8005260 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 10 client.16737 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ef2399450 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d80047a0 con 0x564ef2313580 2024-04-25T00:25:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60d8028480 con 0x564ef2313580 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 10 client.16737 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 10 client.16737 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 20 client.16737 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef2399b60 0x564ef239bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 4 client.16737 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ef227ff00 con 0x564ef2399b60 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.076+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef2399b60 0x564ef239bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60e37fe640 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef2399b60 0x564ef239bf80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbf967ab76278814 server_cookie=acb27835bf1cde4c in_seq=0 out_seq=0 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60c37fe640 10 client.16737 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60c37fe640 1 -- 172.21.15.45:0/1167980572 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ef227ff00 con 0x564ef2399b60 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60c37fe640 10 client.16737 handle_command_reply: tid=0 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 client.16737 shutdown 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 msgr2=0x564ef238cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009ca0 tx=0x7f60d8004a40 comp rx=0 tx=0).stop 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 shutdown_connections 2024-04-25T00:25:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x564ef2399b60 0x564ef239bf80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564ef231e570 0x564ef2397bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ef2313ea0 0x564ef238d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60c37fe640 0 client.16737 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 --2- 172.21.15.45:0/1167980572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ef2313580 0x564ef238cd30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 >> 172.21.15.45:0/1167980572 conn(0x564ef230f8a0 msgr2=0x564ef2315eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 shutdown_connections 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 1 -- 172.21.15.45:0/1167980572 wait complete. 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.077+0000 7f60ed625740 20 client.16737 trim_cache size 0 max 16384 2024-04-25T00:25:50.085 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 msgr2=0x7f60cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 0x7f60cc0a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01f000 tx=0x7f60dc01b000 comp rx=0 tx=0).stop 2024-04-25T00:25:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 msgr2=0x7f60e41974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009230 tx=0x7f60d4004270 comp rx=0 tx=0).stop 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 shutdown_connections 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f60cc0a2d00 0x7f60cc0a51c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e4106630 0x7f60e419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60e4100fe0 0x7f60e41974a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 --2- 172.21.15.45:0/1088175660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60e4100620 0x7f60e4196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 >> 172.21.15.45:0/1088175660 conn(0x7f60e40776a0 msgr2=0x7f60e4103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 shutdown_connections 2024-04-25T00:25:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.078+0000 7f60ea914640 1 -- 172.21.15.45:0/1088175660 wait complete. 2024-04-25T00:25:50.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:25:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- 172.21.15.45:0/3859109219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0101710 msgr2=0x7f75b0101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 --2- 172.21.15.45:0/3859109219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0101710 0x7f75b0101b90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009930 tx=0x7f75a402a420 comp rx=0 tx=0).stop 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- 172.21.15.45:0/3859109219 shutdown_connections 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 --2- 172.21.15.45:0/3859109219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 0x7f75b010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 --2- 172.21.15.45:0/3859109219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0101710 0x7f75b0101b90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 --2- 172.21.15.45:0/3859109219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0107720 0x7f75b0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- 172.21.15.45:0/3859109219 >> 172.21.15.45:0/3859109219 conn(0x7f75b0078b90 msgr2=0x7f75b0078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- 172.21.15.45:0/3859109219 shutdown_connections 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- 172.21.15.45:0/3859109219 wait complete. 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 Processor -- start 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.193+0000 7f75b625d640 1 -- start start 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0101710 0x7f75b0073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 0x7f75b0073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0111b30 con 0x7f75b0107720 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01119b0 con 0x7f75b0101710 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75b625d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0111cb0 con 0x7f75b01020d0 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51798/0 (socket says 172.21.15.45:51798) 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 -- 172.21.15.45:0/1639551091 learned_addr learned my addr 172.21.15.45:0/1639551091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75aeffd640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 0x7f75b0073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75af7fe640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0101710 0x7f75b0073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 -- 172.21.15.45:0/1639551091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 msgr2=0x7f75b0073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 0x7f75b0073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 -- 172.21.15.45:0/1639551091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0101710 msgr2=0x7f75b0073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0101710 0x7f75b0073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 -- 172.21.15.45:0/1639551091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b01a6530 con 0x7f75b0107720 2024-04-25T00:25:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.194+0000 7f75affff640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75a800eb10 tx=0x7f75a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.195+0000 7f75acff9640 1 -- 172.21.15.45:0/1639551091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8004b20 con 0x7f75b0107720 2024-04-25T00:25:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.195+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b01a6820 con 0x7f75b0107720 2024-04-25T00:25:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.195+0000 7f75acff9640 1 -- 172.21.15.45:0/1639551091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a8004cc0 con 0x7f75b0107720 2024-04-25T00:25:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.195+0000 7f75acff9640 1 -- 172.21.15.45:0/1639551091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8010760 con 0x7f75b0107720 2024-04-25T00:25:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.195+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b01a6d90 con 0x7f75b0107720 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.197+0000 7f75acff9640 1 -- 172.21.15.45:0/1639551091 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f75a8005000 con 0x7f75b0107720 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 Processor -- start 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 -- start start 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56247c330670 con 0x56247c3cb580 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56247c2d8970 con 0x56247c3d6570 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56247c32c630 con 0x56247c3cbea0 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51814/0 (socket says 172.21.15.45:51814) 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 -- 172.21.15.45:0/3648013091 learned_addr learned my addr 172.21.15.45:0/3648013091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75acff9640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 0x7f75980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75af7fe640 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75affff640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 0x7f75980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 -- 172.21.15.45:0/3648013091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 msgr2=0x56247c3d6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3d6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75acff9640 1 -- 172.21.15.45:0/1639551091 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f75a80cfe70 con 0x7f75b0107720 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 -- 172.21.15.45:0/3648013091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 msgr2=0x56247c3d8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.198+0000 7f75aeffd640 1 -- 172.21.15.45:0/3648013091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56247c2fc400 con 0x56247c3cb580 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005c40 tx=0x7f75a4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267f075f217e693f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75affff640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 0x7f75980a51f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75a80989a0 tx=0x7f75a806df90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f7595ffb640 1 -- 172.21.15.45:0/3648013091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4044030 con 0x56247c3cb580 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f7595ffb640 1 -- 172.21.15.45:0/3648013091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a40319c0 con 0x56247c3cb580 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f7595ffb640 1 -- 172.21.15.45:0/3648013091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4035040 con 0x56247c3cb580 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3648013091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 msgr2=0x56247c3cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005c40 tx=0x7f75a4005fd0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3648013091 shutdown_connections 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3d6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3648013091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3cb960 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3648013091 >> 172.21.15.45:0/3648013091 conn(0x56247c3c78a0 msgr2=0x56247c3c7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3648013091 shutdown_connections 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.199+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3648013091 wait complete. 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 -- start start 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 Processor -- start 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3d6570 0x56247c3d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56247c3d9300 con 0x56247c3cb580 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56247c2ec050 con 0x56247c3cbea0 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56247c3d9480 con 0x56247c3d6570 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75b8f6c740 10 client.0 _ll_register_callbacks cb 0x56247c043be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51820/0 (socket says 172.21.15.45:51820) 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75aeffd640 1 -- 172.21.15.45:0/3190180221 learned_addr learned my addr 172.21.15.45:0/3190180221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.200+0000 7f75af7fe640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3d6570 0x56247c3d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 -- 172.21.15.45:0/3190180221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3d6570 msgr2=0x56247c3d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3d6570 0x56247c3d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 -- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 msgr2=0x56247c3d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 -- 172.21.15.45:0/3190180221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56247c2bddc0 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75af7fe640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75aeffd640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009900 tx=0x7f75a40055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4dec768e886308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f758effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4035070 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f75a4004230 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a40319c0 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75a4044040 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75b8f6c740 10 client.16749 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56247c0443f0 con 0x56247c3cb580 2024-04-25T00:25:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.201+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a40319c0 con 0x56247c3cb580 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75a40035f0 con 0x56247c3cb580 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 10 client.16749 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 10 client.16749 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 20 client.16749 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c3e63d0 0x56247c3e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 4 client.16749 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56247c33dbb0 con 0x56247c3e63d0 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75af7fe640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c3e63d0 0x56247c3e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f758effd640 10 client.16749 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.202+0000 7f75af7fe640 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c3e63d0 0x56247c3e87c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=289efd1f997cf692 server_cookie=b161798f27641379 in_seq=0 out_seq=0 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f758effd640 1 -- 172.21.15.45:0/3190180221 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56247c33dbb0 con 0x56247c3e63d0 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f758effd640 10 client.16749 handle_command_reply: tid=0 2024-04-25T00:25:50.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 client.16749 shutdown 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 msgr2=0x56247c3d5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009900 tx=0x7f75a40055c0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 shutdown_connections 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c3e63d0 0x56247c3e87c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3d6570 0x56247c3d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f758effd640 0 client.16749 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/3190180221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3d5ea0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 >> 172.21.15.45:0/3190180221 conn(0x56247c3c78a0 msgr2=0x56247c3cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 shutdown_connections 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 1 -- 172.21.15.45:0/3190180221 wait complete. 2024-04-25T00:25:50.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.203+0000 7f75b8f6c740 20 client.16749 trim_cache size 0 max 16384 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 Processor -- start 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 -- start start 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01b0d50 con 0x56247c3cb580 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01b2890 con 0x56247c3d6570 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01b1420 con 0x56247c3cbea0 2024-04-25T00:25:50.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51828/0 (socket says 172.21.15.45:51828) 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 -- 172.21.15.45:0/1463861095 learned_addr learned my addr 172.21.15.45:0/1463861095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75af7fe640 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 -- 172.21.15.45:0/1463861095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 msgr2=0x56247c3ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 -- 172.21.15.45:0/1463861095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 msgr2=0x56247c3d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 -- 172.21.15.45:0/1463861095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56247c10e7d0 con 0x56247c3cb580 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.217+0000 7f75aeffd640 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006010 tx=0x7f75a4005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548584a0c17aa76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75957fa640 1 -- 172.21.15.45:0/1463861095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4035030 con 0x56247c3cb580 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75957fa640 1 -- 172.21.15.45:0/1463861095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4004440 con 0x56247c3cb580 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75957fa640 1 -- 172.21.15.45:0/1463861095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a40316d0 con 0x56247c3cb580 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- 172.21.15.45:0/1463861095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 msgr2=0x56247c3e1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006010 tx=0x7f75a4005e10 comp rx=0 tx=0).stop 2024-04-25T00:25:50.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- 172.21.15.45:0/1463861095 shutdown_connections 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3d6570 0x56247c3d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cbea0 0x56247c3ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/1463861095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3cb580 0x56247c3e1f60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- 172.21.15.45:0/1463861095 >> 172.21.15.45:0/1463861095 conn(0x56247c3c78a0 msgr2=0x56247c3cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- 172.21.15.45:0/1463861095 shutdown_connections 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- 172.21.15.45:0/1463861095 wait complete. 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.218+0000 7f75b8f6c740 1 -- start start 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 Processor -- start 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cb580 0x56247c3e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01b21c0 con 0x56247c3d6570 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01b1af0 con 0x56247c3cbea0 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01aed30 con 0x56247c3cb580 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75b8f6c740 10 client.0 _ll_register_callbacks cb 0x7f75a000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51832/0 (socket says 172.21.15.45:51832) 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 -- 172.21.15.45:0/733794267 learned_addr learned my addr 172.21.15.45:0/733794267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cb580 0x56247c3e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75af7fe640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 -- 172.21.15.45:0/733794267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cb580 msgr2=0x56247c3e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cb580 0x56247c3e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 -- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 msgr2=0x56247c3e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.219+0000 7f75aeffd640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75aeffd640 1 -- 172.21.15.45:0/733794267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56247c44b630 con 0x56247c3d6570 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75aeffd640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f75a4004720 tx=0x7f75a4005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4372ed9a4ecfc84c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4035070 con 0x56247c3d6570 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f75a40037f0 con 0x56247c3d6570 2024-04-25T00:25:50.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a4004440 con 0x56247c3d6570 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75a4003c40 con 0x56247c3d6570 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 10 client.16755 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56247c44c910 con 0x56247c3d6570 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4004440 con 0x56247c3d6570 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75a405a070 con 0x56247c3d6570 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 10 client.16755 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 10 client.16755 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 20 client.16755 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c44cd90 0x56247c44f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 4 client.16755 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.220+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56247c337f00 con 0x56247c44cd90 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75af7fe640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c44cd90 0x56247c44f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75af7fe640 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c44cd90 0x56247c44f180 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1c36e8704105f44 server_cookie=d5e563763637cfa6 in_seq=0 out_seq=0 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f758effd640 10 client.16755 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f758effd640 1 -- 172.21.15.45:0/733794267 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56247c337f00 con 0x56247c44cd90 2024-04-25T00:25:50.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f758effd640 10 client.16755 handle_command_reply: tid=0 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 client.16755 shutdown 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 msgr2=0x56247c3e9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f75a4004720 tx=0x7f75a4005440 comp rx=0 tx=0).stop 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 shutdown_connections 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56247c44cd90 0x56247c44f180 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56247c3d6570 0x56247c3e9dd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56247c3cbea0 0x56247c3e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f758effd640 0 client.16755 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 --2- 172.21.15.45:0/733794267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56247c3cb580 0x56247c3e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 >> 172.21.15.45:0/733794267 conn(0x56247c3c78a0 msgr2=0x56247c3c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 shutdown_connections 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 1 -- 172.21.15.45:0/733794267 wait complete. 2024-04-25T00:25:50.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.221+0000 7f75b8f6c740 20 client.16755 trim_cache size 0 max 16384 2024-04-25T00:25:50.229 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:50.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 msgr2=0x7f75980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 0x7f75980a51f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75a80989a0 tx=0x7f75a806df90 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 msgr2=0x7f75b00744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75a800eb10 tx=0x7f75a800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 shutdown_connections 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f75980a2d30 0x7f75980a51f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75b0107720 0x7f75b00744c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75b01020d0 0x7f75b0073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 --2- 172.21.15.45:0/1639551091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b0101710 0x7f75b0073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 >> 172.21.15.45:0/1639551091 conn(0x7f75b0078b90 msgr2=0x7f75b00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 shutdown_connections 2024-04-25T00:25:50.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.222+0000 7f75b625d640 1 -- 172.21.15.45:0/1639551091 wait complete. 2024-04-25T00:25:50.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 -- 172.21.15.45:0/320906683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c06a780 msgr2=0x7fe16c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 --2- 172.21.15.45:0/320906683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c06a780 0x7fe16c10e5f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe164009cd0 tx=0x7fe16402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 -- 172.21.15.45:0/320906683 shutdown_connections 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 --2- 172.21.15.45:0/320906683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c06a780 0x7fe16c10e5f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 --2- 172.21.15.45:0/320906683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c069dc0 0x7fe16c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 --2- 172.21.15.45:0/320906683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c107730 0x7fe16c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 -- 172.21.15.45:0/320906683 >> 172.21.15.45:0/320906683 conn(0x7fe16c0776a0 msgr2=0x7fe16c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 -- 172.21.15.45:0/320906683 shutdown_connections 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.337+0000 7fe173d70640 1 -- 172.21.15.45:0/320906683 wait complete. 2024-04-25T00:25:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 Processor -- start 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 -- start start 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 0x7fe16c100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 0x7fe16c1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c111b00 con 0x7fe16c069dc0 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c111980 con 0x7fe16c107730 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe173d70640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c111c80 con 0x7fe16c06a780 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51848/0 (socket says 172.21.15.45:51848) 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 -- 172.21.15.45:0/3806220011 learned_addr learned my addr 172.21.15.45:0/3806220011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe17130b640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 0x7fe16c100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe17230d640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 0x7fe16c1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 -- 172.21.15.45:0/3806220011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 msgr2=0x7fe16c100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 0x7fe16c100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 -- 172.21.15.45:0/3806220011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 msgr2=0x7fe16c1010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 0x7fe16c1010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 -- 172.21.15.45:0/3806220011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1a63e0 con 0x7fe16c069dc0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe17230d640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 0x7fe16c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe17130b640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 0x7fe16c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.338+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c920 tx=0x7fe16800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.339+0000 7fe162ffd640 1 -- 172.21.15.45:0/3806220011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168007c70 con 0x7fe16c069dc0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.339+0000 7fe162ffd640 1 -- 172.21.15.45:0/3806220011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168007e10 con 0x7fe16c069dc0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.339+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c1a66d0 con 0x7fe16c069dc0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.339+0000 7fe162ffd640 1 -- 172.21.15.45:0/3806220011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168018890 con 0x7fe16c069dc0 2024-04-25T00:25:50.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.339+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c1a6c70 con 0x7fe16c069dc0 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.341+0000 7fe176a7d740 1 Processor -- start 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.341+0000 7fe162ffd640 1 -- 172.21.15.45:0/3806220011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe168018a30 con 0x7fe16c069dc0 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.341+0000 7fe176a7d740 1 -- start start 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a60960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5613e29c5670 con 0x5613e2a6b570 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5613e296d970 con 0x5613e2a60580 2024-04-25T00:25:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe176a7d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613e29c1630 con 0x5613e2a60ea0 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51858/0 (socket says 172.21.15.45:51858) 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 -- 172.21.15.45:0/1268503286 learned_addr learned my addr 172.21.15.45:0/1268503286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe171b0c640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17230d640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe162ffd640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 0x7fe1580a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 -- 172.21.15.45:0/1268503286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 msgr2=0x5613e2a6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a6b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 -- 172.21.15.45:0/1268503286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 msgr2=0x5613e2a60960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a60960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17130b640 1 -- 172.21.15.45:0/1268503286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e2991400 con 0x5613e2a6b570 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17230d640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a60960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe17230d640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 0x7fe1580a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.342+0000 7fe162ffd640 1 -- 172.21.15.45:0/3806220011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe1680cf860 con 0x7fe16c069dc0 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe17130b640 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00bd30 tx=0x7fe15c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dec2f9c42b58f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe157fff640 1 -- 172.21.15.45:0/1268503286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c015070 con 0x5613e2a6b570 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe17230d640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 0x7fe1580a5270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe164009a60 tx=0x7fe16403f040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe157fff640 1 -- 172.21.15.45:0/1268503286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c00be20 con 0x5613e2a6b570 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe157fff640 1 -- 172.21.15.45:0/1268503286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0107f0 con 0x5613e2a6b570 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 -- 172.21.15.45:0/1268503286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 msgr2=0x5613e2a6d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00bd30 tx=0x7fe15c00f730 comp rx=0 tx=0).stop 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 -- 172.21.15.45:0/1268503286 shutdown_connections 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2a6d960 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.343+0000 7fe176a7d740 1 --2- 172.21.15.45:0/1268503286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a60960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- 172.21.15.45:0/1268503286 >> 172.21.15.45:0/1268503286 conn(0x5613e2a5c8a0 msgr2=0x5613e2a5cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- 172.21.15.45:0/1268503286 shutdown_connections 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- 172.21.15.45:0/1268503286 wait complete. 2024-04-25T00:25:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- start start 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 Processor -- start 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a6aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a6b570 0x5613e2a68710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5613e2a6e360 con 0x5613e2a60ea0 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5613e2981050 con 0x5613e2a60580 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613e2a6e4e0 con 0x5613e2a6b570 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.344+0000 7fe176a7d740 10 client.0 _ll_register_callbacks cb 0x5613e26d8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51868/0 (socket says 172.21.15.45:51868) 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 -- 172.21.15.45:0/3366041389 learned_addr learned my addr 172.21.15.45:0/3366041389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a6aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17230d640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a6b570 0x5613e2a68710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 -- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a6b570 msgr2=0x5613e2a68710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a6b570 0x5613e2a68710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 -- 172.21.15.45:0/3366041389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 msgr2=0x5613e2a6aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a6aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 -- 172.21.15.45:0/3366041389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e2952dc0 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe17130b640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011fa0 tx=0x7fe15c00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb4136a67fddcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c029060 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe15c004410 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c00be20 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe15c030050 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe176a7d740 10 client.16767 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.345+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613e26d93f0 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c00be20 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe15c010920 con 0x5613e2a60ea0 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 10 client.16767 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 20 client.16767 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2a7b500 0x5613e2a7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 4 client.16767 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e29d2bb0 con 0x5613e2a7b500 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2a7b500 0x5613e2a7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe1557fa640 10 client.16767 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.346+0000 7fe171b0c640 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2a7b500 0x5613e2a7d8f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=826e0a77832c7e60 server_cookie=70cd15d98d8ad1 in_seq=0 out_seq=0 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe1557fa640 1 -- 172.21.15.45:0/3366041389 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613e29d2bb0 con 0x5613e2a7b500 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe1557fa640 10 client.16767 handle_command_reply: tid=0 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 client.16767 shutdown 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 msgr2=0x5613e2a681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011fa0 tx=0x7fe15c00ba70 comp rx=0 tx=0).stop 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 shutdown_connections 2024-04-25T00:25:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2a7b500 0x5613e2a7d8f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a6b570 0x5613e2a68710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a681b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe1557fa640 0 client.16767 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 --2- 172.21.15.45:0/3366041389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a6aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 >> 172.21.15.45:0/3366041389 conn(0x5613e2a5c8a0 msgr2=0x5613e2a5bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.347+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 shutdown_connections 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.348+0000 7fe176a7d740 1 -- 172.21.15.45:0/3366041389 wait complete. 2024-04-25T00:25:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.348+0000 7fe176a7d740 20 client.16767 trim_cache size 0 max 16384 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 Processor -- start 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 -- start start 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60580 0x5613e2a77220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 0x5613e2a833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1b0ca0 con 0x5613e2a60ea0 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1b27e0 con 0x5613e2a6b570 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe176a7d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1b1370 con 0x5613e2a60580 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51872/0 (socket says 172.21.15.45:51872) 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 -- 172.21.15.45:0/2507137806 learned_addr learned my addr 172.21.15.45:0/2507137806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17230d640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 0x5613e2a833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe171b0c640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60580 0x5613e2a77220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 -- 172.21.15.45:0/2507137806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60580 msgr2=0x5613e2a77220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60580 0x5613e2a77220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 -- 172.21.15.45:0/2507137806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 msgr2=0x5613e2a833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 0x5613e2a833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17130b640 1 -- 172.21.15.45:0/2507137806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e27a37d0 con 0x5613e2a60ea0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.361+0000 7fe17230d640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 0x5613e2a833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe17130b640 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00fde0 tx=0x7fe15c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60b5e01f0680118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe1577fe640 1 -- 172.21.15.45:0/2507137806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c029070 con 0x5613e2a60ea0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe1577fe640 1 -- 172.21.15.45:0/2507137806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c00be20 con 0x5613e2a60ea0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe1577fe640 1 -- 172.21.15.45:0/2507137806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c015050 con 0x5613e2a60ea0 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 -- 172.21.15.45:0/2507137806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 msgr2=0x5613e2a82e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00fde0 tx=0x7fe15c010f10 comp rx=0 tx=0).stop 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 -- 172.21.15.45:0/2507137806 shutdown_connections 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a6b570 0x5613e2a833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a60ea0 0x5613e2a82e90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2507137806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60580 0x5613e2a77220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 -- 172.21.15.45:0/2507137806 >> 172.21.15.45:0/2507137806 conn(0x5613e2a5c8a0 msgr2=0x5613e2a5b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 -- 172.21.15.45:0/2507137806 shutdown_connections 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.362+0000 7fe176a7d740 1 -- 172.21.15.45:0/2507137806 wait complete. 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 -- start start 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 Processor -- start 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a70e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1b2110 con 0x5613e2a6b570 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1b1a40 con 0x5613e2a60580 2024-04-25T00:25:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1aec80 con 0x5613e2a60ea0 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe176a7d740 10 client.0 _ll_register_callbacks cb 0x7fe164009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe17230d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe17230d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51884/0 (socket says 172.21.15.45:51884) 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe171b0c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.363+0000 7fe17230d640 1 -- 172.21.15.45:0/2531385153 learned_addr learned my addr 172.21.15.45:0/2531385153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17130b640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a70e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 -- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 msgr2=0x5613e2a70e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a70e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 -- 172.21.15.45:0/2531385153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 msgr2=0x5613e2a708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 -- 172.21.15.45:0/2531385153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e2adfc40 con 0x5613e2a6b570 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe171b0c640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:50.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe17230d640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe16402ae00 tx=0x7fe164009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2382761706248f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164004510 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c00be20 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe1640046b0 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1640305a0 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe176a7d740 10 client.16773 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613e2ae0ef0 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.364+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c00be20 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe164030070 con 0x5613e2a6b570 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 10 client.16773 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 20 client.16773 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2ae1670 0x5613e2ae3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 4 client.16773 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:50.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613e29ccf00 con 0x5613e2ae1670 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe171b0c640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2ae1670 0x5613e2ae3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe171b0c640 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2ae1670 0x5613e2ae3a60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d21f0b1c1fabf21d server_cookie=477db539fe166df in_seq=0 out_seq=0 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe1557fa640 10 client.16773 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe1557fa640 1 -- 172.21.15.45:0/2531385153 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613e29ccf00 con 0x5613e2ae1670 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe1557fa640 10 client.16773 handle_command_reply: tid=0 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.365+0000 7fe176a7d740 1 client.16773 shutdown 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 msgr2=0x5613e2adf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe16402ae00 tx=0x7fe164009360 comp rx=0 tx=0).stop 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 shutdown_connections 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5613e2ae1670 0x5613e2ae3a60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613e2a6b570 0x5613e2adf700 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e2a60ea0 0x5613e2a70e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 --2- 172.21.15.45:0/2531385153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613e2a60580 0x5613e2a708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe1557fa640 0 client.16773 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:25:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 >> 172.21.15.45:0/2531385153 conn(0x5613e2a5c8a0 msgr2=0x5613e2a5bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 shutdown_connections 2024-04-25T00:25:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 1 -- 172.21.15.45:0/2531385153 wait complete. 2024-04-25T00:25:50.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe176a7d740 20 client.16773 trim_cache size 0 max 16384 2024-04-25T00:25:50.373 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 msgr2=0x7fe1580a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 0x7fe1580a5270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe164009a60 tx=0x7fe16403f040 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 msgr2=0x7fe16c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c920 tx=0x7fe16800cde0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 shutdown_connections 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe1580a2db0 0x7fe1580a5270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe16c107730 0x7fe16c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe16c06a780 0x7fe16c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 --2- 172.21.15.45:0/3806220011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe16c069dc0 0x7fe16c100670 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 >> 172.21.15.45:0/3806220011 conn(0x7fe16c0776a0 msgr2=0x7fe16c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 shutdown_connections 2024-04-25T00:25:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.366+0000 7fe173d70640 1 -- 172.21.15.45:0/3806220011 wait complete. 2024-04-25T00:25:50.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:25:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.481+0000 7f4a64f42640 1 -- 172.21.15.45:0/2511532267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60069dc0 msgr2=0x7f4a6006a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.481+0000 7f4a64f42640 1 --2- 172.21.15.45:0/2511532267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60069dc0 0x7f4a6006a240 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009930 tx=0x7f4a5402a420 comp rx=0 tx=0).stop 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 -- 172.21.15.45:0/2511532267 shutdown_connections 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 --2- 172.21.15.45:0/2511532267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a6006a780 0x7f4a6010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 --2- 172.21.15.45:0/2511532267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60069dc0 0x7f4a6006a240 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 --2- 172.21.15.45:0/2511532267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a60107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 -- 172.21.15.45:0/2511532267 >> 172.21.15.45:0/2511532267 conn(0x7f4a600776a0 msgr2=0x7f4a60077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 -- 172.21.15.45:0/2511532267 shutdown_connections 2024-04-25T00:25:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 -- 172.21.15.45:0/2511532267 wait complete. 2024-04-25T00:25:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 Processor -- start 2024-04-25T00:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 -- start start 2024-04-25T00:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 0x7f4a60197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.482+0000 7f4a64f42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a6019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111b40 con 0x7f4a6006a780 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601119c0 con 0x7f4a60069dc0 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111cc0 con 0x7f4a60107730 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51908/0 (socket says 172.21.15.45:51908) 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/1069658761 learned_addr learned my addr 172.21.15.45:0/1069658761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 0x7f4a60197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a6019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/1069658761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 msgr2=0x7f4a6019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a6019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/1069658761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 msgr2=0x7f4a60197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 0x7f4a60197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/1069658761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6019cfd0 con 0x7f4a6006a780 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a6019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 0x7f4a60197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009230 tx=0x7f4a54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a4f7fe640 1 -- 172.21.15.45:0/1069658761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5403f030 con 0x7f4a6006a780 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a4f7fe640 1 -- 172.21.15.45:0/1069658761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a54003ad0 con 0x7f4a6006a780 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a4f7fe640 1 -- 172.21.15.45:0/1069658761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54035070 con 0x7f4a6006a780 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6019d260 con 0x7f4a6006a780 2024-04-25T00:25:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.483+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a4b50 con 0x7f4a6006a780 2024-04-25T00:25:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.486+0000 7f4a4f7fe640 1 -- 172.21.15.45:0/1069658761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a54042030 con 0x7f4a6006a780 2024-04-25T00:25:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.486+0000 7f4a67c55740 1 Processor -- start 2024-04-25T00:25:50.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 -- start start 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd24960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd2f570 0x55a99dd31960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a99dc89670 con 0x55a99dd24ea0 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a99dc31970 con 0x55a99dd24580 2024-04-25T00:25:50.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a67c55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a99dc85630 con 0x55a99dd2f570 2024-04-25T00:25:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a4f7fe640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 0x7f4a480a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51914/0 (socket says 172.21.15.45:51914) 2024-04-25T00:25:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 -- 172.21.15.45:0/930137668 learned_addr learned my addr 172.21.15.45:0/930137668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 0x7f4a480a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a4f7fe640 1 -- 172.21.15.45:0/1069658761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f4a540efe50 con 0x7f4a6006a780 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd2f570 0x55a99dd31960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 -- 172.21.15.45:0/930137668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd2f570 msgr2=0x55a99dd31960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd2f570 0x55a99dd31960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 -- 172.21.15.45:0/930137668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 msgr2=0x55a99dd24960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd24960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 -- 172.21.15.45:0/930137668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99dc55400 con 0x55a99dd24ea0 2024-04-25T00:25:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd24960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.487+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a580114c0 tx=0x7f4a58011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cc95c31bc7d785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 0x7f4a480a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005c40 tx=0x7f4a50009590 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a4cff9640 1 -- 172.21.15.45:0/930137668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58004020 con 0x55a99dd24ea0 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a4cff9640 1 -- 172.21.15.45:0/930137668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a580041c0 con 0x55a99dd24ea0 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a4cff9640 1 -- 172.21.15.45:0/930137668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800f8f0 con 0x55a99dd24ea0 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 -- 172.21.15.45:0/930137668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 msgr2=0x55a99dd2f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a580114c0 tx=0x7f4a58011980 comp rx=0 tx=0).stop 2024-04-25T00:25:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 -- 172.21.15.45:0/930137668 shutdown_connections 2024-04-25T00:25:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd2f570 0x55a99dd31960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd2f030 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 --2- 172.21.15.45:0/930137668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd24960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.488+0000 7f4a67c55740 1 -- 172.21.15.45:0/930137668 >> 172.21.15.45:0/930137668 conn(0x55a99dd208a0 msgr2=0x55a99dd20c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- 172.21.15.45:0/930137668 shutdown_connections 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- 172.21.15.45:0/930137668 wait complete. 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- start start 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 Processor -- start 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd2ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24ea0 0x55a99dd2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a99dd32360 con 0x55a99dd2f570 2024-04-25T00:25:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a99dc45050 con 0x55a99dd24ea0 2024-04-25T00:25:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a99dd324e0 con 0x55a99dd24580 2024-04-25T00:25:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a67c55740 10 client.0 _ll_register_callbacks cb 0x55a99d99cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a5ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a5ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51916/0 (socket says 172.21.15.45:51916) 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/3873343691 learned_addr learned my addr 172.21.15.45:0/3873343691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24ea0 0x55a99dd2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.489+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd2ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 msgr2=0x55a99dd2ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd2ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/3873343691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24ea0 msgr2=0x55a99dd2c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24ea0 0x55a99dd2c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/3873343691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99dc16dc0 con 0x55a99dd2f570 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a50011a90 tx=0x7f4a50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118014e099b53dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5001dd40 con 0x55a99dd2f570 2024-04-25T00:25:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f4a50027470 con 0x55a99dd2f570 2024-04-25T00:25:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a580041c0 con 0x55a99dd2f570 2024-04-25T00:25:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a50030050 con 0x55a99dd2f570 2024-04-25T00:25:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 10 client.16785 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a99d99d3f0 con 0x55a99dd2f570 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a580041c0 con 0x55a99dd2f570 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a5001b5b0 con 0x55a99dd2f570 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 10 client.16785 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 20 client.16785 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99dd3f5f0 0x55a99dd41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 4 client.16785 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.490+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99dc96bb0 con 0x55a99dd3f5f0 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.491+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99dd3f5f0 0x55a99dd41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.491+0000 7f4a45ffb640 10 client.16785 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.491+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99dd3f5f0 0x55a99dd41a20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d580ee55313e10b5 server_cookie=9537d283b2decac3 in_seq=0 out_seq=0 2024-04-25T00:25:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a45ffb640 1 -- 172.21.15.45:0/3873343691 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a99dc96bb0 con 0x55a99dd3f5f0 2024-04-25T00:25:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a45ffb640 10 client.16785 handle_command_reply: tid=0 2024-04-25T00:25:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 client.16785 shutdown 2024-04-25T00:25:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 msgr2=0x55a99dd2c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a50011a90 tx=0x7f4a50004ab0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 shutdown_connections 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99dd3f5f0 0x55a99dd41a20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2c840 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24ea0 0x55a99dd2c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a45ffb640 0 client.16785 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 --2- 172.21.15.45:0/3873343691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd2ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 >> 172.21.15.45:0/3873343691 conn(0x55a99dd208a0 msgr2=0x55a99dd266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 shutdown_connections 2024-04-25T00:25:50.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.492+0000 7f4a67c55740 1 -- 172.21.15.45:0/3873343691 wait complete. 2024-04-25T00:25:50.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.493+0000 7f4a67c55740 20 client.16785 trim_cache size 0 max 16384 2024-04-25T00:25:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 Processor -- start 2024-04-25T00:25:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 -- start start 2024-04-25T00:25:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 0x55a99dd37b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601aea80 con 0x55a99dd2f570 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b05c0 con 0x55a99dd24580 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a67c55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601af150 con 0x55a99dd24ea0 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51924/0 (socket says 172.21.15.45:51924) 2024-04-25T00:25:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/1117698821 learned_addr learned my addr 172.21.15.45:0/1117698821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 0x55a99dd37b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/1117698821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 msgr2=0x55a99dd37b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 0x55a99dd37b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/1117698821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 msgr2=0x55a99dd3b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd3b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 -- 172.21.15.45:0/1117698821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99da677d0 con 0x55a99dd2f570 2024-04-25T00:25:50.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 0x55a99dd37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:50.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.506+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ac0 tx=0x7f4a500098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b594bc6f99709d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a47fff640 1 -- 172.21.15.45:0/1117698821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5001bdf0 con 0x55a99dd2f570 2024-04-25T00:25:50.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a47fff640 1 -- 172.21.15.45:0/1117698821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50022070 con 0x55a99dd2f570 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a47fff640 1 -- 172.21.15.45:0/1117698821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50005280 con 0x55a99dd2f570 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- 172.21.15.45:0/1117698821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 msgr2=0x55a99dd2ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009ac0 tx=0x7f4a500098e0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- 172.21.15.45:0/1117698821 shutdown_connections 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd2f570 0x55a99dd2ea70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24ea0 0x55a99dd37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 --2- 172.21.15.45:0/1117698821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd24580 0x55a99dd3b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- 172.21.15.45:0/1117698821 >> 172.21.15.45:0/1117698821 conn(0x55a99dd208a0 msgr2=0x55a99dd27740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- 172.21.15.45:0/1117698821 shutdown_connections 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- 172.21.15.45:0/1117698821 wait complete. 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 -- start start 2024-04-25T00:25:50.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.507+0000 7f4a67c55740 1 Processor -- start 2024-04-25T00:25:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd9db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd2f570 0x55a99dd9e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601afef0 con 0x55a99dd24ea0 2024-04-25T00:25:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601af820 con 0x55a99dd2f570 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601aca60 con 0x55a99dd24580 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 10 client.0 _ll_register_callbacks cb 0x7f4a50009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51940/0 (socket says 172.21.15.45:51940) 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/960043794 learned_addr learned my addr 172.21.15.45:0/960043794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5ed9d640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd2f570 0x55a99dd9e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd9db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 msgr2=0x55a99dd9db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd9db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/960043794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd2f570 msgr2=0x55a99dd9e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd2f570 0x55a99dd9e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 -- 172.21.15.45:0/960043794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99dda8d40 con 0x55a99dd24ea0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a5dd9b640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009900 tx=0x7f4a540dc0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9639f86d171a180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540d4070 con 0x55a99dd24ea0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f4a54003c80 con 0x55a99dd24ea0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.508+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50022070 con 0x55a99dd24ea0 2024-04-25T00:25:50.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a54049040 con 0x55a99dd24ea0 2024-04-25T00:25:50.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 10 client.16791 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a99dda9f90 con 0x55a99dd24ea0 2024-04-25T00:25:50.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50022070 con 0x55a99dd24ea0 2024-04-25T00:25:50.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a540d9070 con 0x55a99dd24ea0 2024-04-25T00:25:50.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 10 client.16791 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 10 client.16791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 20 client.16791 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99ddaa6e0 0x55a99ddacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 4 client.16791 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a99dc90f00 con 0x55a99ddaa6e0 2024-04-25T00:25:50.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.509+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99ddaa6e0 0x55a99ddacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a45ffb640 10 client.16791 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a5e59c640 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99ddaa6e0 0x55a99ddacb00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39cf1fd333292c3c server_cookie=fd60fc095f058d49 in_seq=0 out_seq=0 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a45ffb640 1 -- 172.21.15.45:0/960043794 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a99dc90f00 con 0x55a99ddaa6e0 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a45ffb640 10 client.16791 handle_command_reply: tid=0 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 client.16791 shutdown 2024-04-25T00:25:50.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 msgr2=0x55a99dd9e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009900 tx=0x7f4a540dc0b0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 shutdown_connections 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55a99ddaa6e0 0x55a99ddacb00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a99dd2f570 0x55a99dd9e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a99dd24ea0 0x55a99dd9e060 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a45ffb640 0 client.16791 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:25:50.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 --2- 172.21.15.45:0/960043794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a99dd24580 0x55a99dd9db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 >> 172.21.15.45:0/960043794 conn(0x55a99dd208a0 msgr2=0x55a99dd25cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 shutdown_connections 2024-04-25T00:25:50.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 1 -- 172.21.15.45:0/960043794 wait complete. 2024-04-25T00:25:50.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.510+0000 7f4a67c55740 20 client.16791 trim_cache size 0 max 16384 2024-04-25T00:25:50.547 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 msgr2=0x7f4a480a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 0x7f4a480a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005c40 tx=0x7f4a50009590 comp rx=0 tx=0).stop 2024-04-25T00:25:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 msgr2=0x7f4a60198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009230 tx=0x7f4a54004270 comp rx=0 tx=0).stop 2024-04-25T00:25:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 shutdown_connections 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d00 0x7f4a480a51c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107730 0x7f4a6019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a6006a780 0x7f4a60198520 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 --2- 172.21.15.45:0/1069658761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60069dc0 0x7f4a60197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 >> 172.21.15.45:0/1069658761 conn(0x7f4a600776a0 msgr2=0x7f4a600fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 shutdown_connections 2024-04-25T00:25:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.511+0000 7f4a64f42640 1 -- 172.21.15.45:0/1069658761 wait complete. 2024-04-25T00:25:50.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:25:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.656+0000 7f87938d4640 1 -- 172.21.15.45:0/337819798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 msgr2=0x7f878c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.656+0000 7f87938d4640 1 --2- 172.21.15.45:0/337819798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c102da0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a510 comp rx=0 tx=0).stop 2024-04-25T00:25:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- 172.21.15.45:0/337819798 shutdown_connections 2024-04-25T00:25:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- 172.21.15.45:0/337819798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 0x7f878c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- 172.21.15.45:0/337819798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 0x7f878c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- 172.21.15.45:0/337819798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c102da0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- 172.21.15.45:0/337819798 >> 172.21.15.45:0/337819798 conn(0x7f878c077740 msgr2=0x7f878c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- 172.21.15.45:0/337819798 shutdown_connections 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- 172.21.15.45:0/337819798 wait complete. 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 Processor -- start 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- start start 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 0x7f878c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 0x7f878c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f878c111b00 con 0x7f878c1029c0 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f878c111980 con 0x7f878c10bd00 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.657+0000 7f87938d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f878c111c80 con 0x7f878c1032e0 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51958/0 (socket says 172.21.15.45:51958) 2024-04-25T00:25:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 -- 172.21.15.45:0/2286760267 learned_addr learned my addr 172.21.15.45:0/2286760267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 0x7f878c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791e71640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 0x7f878c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 -- 172.21.15.45:0/2286760267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 msgr2=0x7f878c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 0x7f878c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 -- 172.21.15.45:0/2286760267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 msgr2=0x7f878c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 0x7f878c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 -- 172.21.15.45:0/2286760267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c19f2b0 con 0x7f878c1029c0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f8791670640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8788006010 tx=0x7f87880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f87827fc640 1 -- 172.21.15.45:0/2286760267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878803f030 con 0x7f878c1029c0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f87827fc640 1 -- 172.21.15.45:0/2286760267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8788003ce0 con 0x7f878c1029c0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c19f540 con 0x7f878c1029c0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f87827fc640 1 -- 172.21.15.45:0/2286760267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035070 con 0x7f878c1029c0 2024-04-25T00:25:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.658+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a6e30 con 0x7f878c1029c0 2024-04-25T00:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.660+0000 7f87827fc640 1 -- 172.21.15.45:0/2286760267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8788042030 con 0x7f878c1029c0 2024-04-25T00:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 Processor -- start 2024-04-25T00:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 -- start start 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0dea0 0x557874c18030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557874b72670 con 0x557874c0d580 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557874b1a970 con 0x557874c0dea0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87965e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557874b6e630 con 0x557874c18570 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f87827fc640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f8791e71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f8791e71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51974/0 (socket says 172.21.15.45:51974) 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f8791e71640 1 -- 172.21.15.45:0/3391490900 learned_addr learned my addr 172.21.15.45:0/3391490900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.661+0000 7f8790e6f640 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f87827fc640 1 -- 172.21.15.45:0/2286760267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f87880efc70 con 0x7f878c1029c0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791670640 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0dea0 0x557874c18030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 -- 172.21.15.45:0/3391490900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 msgr2=0x557874c1a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 -- 172.21.15.45:0/3391490900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0dea0 msgr2=0x557874c18030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0dea0 0x557874c18030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 -- 172.21.15.45:0/3391490900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557874b3e400 con 0x557874c0d580 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f8791e71640 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8784013940 tx=0x7f8784013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5ab0cfc4b2a06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f87777fe640 1 -- 172.21.15.45:0/3391490900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784021030 con 0x557874c0d580 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f87777fe640 1 -- 172.21.15.45:0/3391490900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87840043a0 con 0x557874c0d580 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.662+0000 7f87777fe640 1 -- 172.21.15.45:0/3391490900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878401d8e0 con 0x557874c0d580 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- 172.21.15.45:0/3391490900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 msgr2=0x557874c0d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f8791e71640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 0x7f87780a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8784021000 tx=0x7f8784020000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8784013940 tx=0x7f8784013e00 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- 172.21.15.45:0/3391490900 shutdown_connections 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0dea0 0x557874c18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 --2- 172.21.15.45:0/3391490900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0d580 0x557874c0d960 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- 172.21.15.45:0/3391490900 >> 172.21.15.45:0/3391490900 conn(0x557874c098a0 msgr2=0x557874c09c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- 172.21.15.45:0/3391490900 shutdown_connections 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- 172.21.15.45:0/3391490900 wait complete. 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 -- start start 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.663+0000 7f87965e5740 1 Processor -- start 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557874c1b360 con 0x557874c0dea0 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557874b2e050 con 0x557874c0d580 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557874c1b4e0 con 0x557874c18570 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 10 client.0 _ll_register_callbacks cb 0x557874885be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51978/0 (socket says 172.21.15.45:51978) 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 -- 172.21.15.45:0/2563240622 learned_addr learned my addr 172.21.15.45:0/2563240622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8791670640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8791e71640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 -- 172.21.15.45:0/2563240622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 msgr2=0x557874c15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 -- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 msgr2=0x557874c17e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c17e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 -- 172.21.15.45:0/2563240622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557874affdc0 con 0x557874c0dea0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8791e71640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b990 tx=0x7f877c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6700a3f31a166f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c005020 con 0x557874c0dea0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f877c0051c0 con 0x557874c0dea0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.664+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87840043a0 con 0x557874c0dea0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f877c024050 con 0x557874c0dea0 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 10 client.16803 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578748863f0 con 0x557874c0dea0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87840043a0 con 0x557874c0dea0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f877c01b470 con 0x557874c0dea0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 10 client.16803 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 20 client.16803 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c28640 0x557874c2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 4 client.16803 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557874b7fbb0 con 0x557874c28640 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.665+0000 7f8791670640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c28640 0x557874c2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.666+0000 7f8791670640 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c28640 0x557874c2aa70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c49ec50a14d3817 server_cookie=f08505e7987cf444 in_seq=0 out_seq=0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.666+0000 7f8774ff9640 10 client.16803 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.666+0000 7f8774ff9640 1 -- 172.21.15.45:0/2563240622 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557874b7fbb0 con 0x557874c28640 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.666+0000 7f8774ff9640 10 client.16803 handle_command_reply: tid=0 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.666+0000 7f87965e5740 1 client.16803 shutdown 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 msgr2=0x557874c151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b990 tx=0x7f877c00be50 comp rx=0 tx=0).stop 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 shutdown_connections 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c28640 0x557874c2aa70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c151e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f8774ff9640 0 client.16803 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 --2- 172.21.15.45:0/2563240622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c17e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 >> 172.21.15.45:0/2563240622 conn(0x557874c098a0 msgr2=0x557874c08b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 shutdown_connections 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 1 -- 172.21.15.45:0/2563240622 wait complete. 2024-04-25T00:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.667+0000 7f87965e5740 20 client.16803 trim_cache size 0 max 16384 2024-04-25T00:25:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 Processor -- start 2024-04-25T00:25:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 -- start start 2024-04-25T00:25:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c24220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b0d00 con 0x557874c0dea0 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b2840 con 0x557874c0d580 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f87965e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b13d0 con 0x557874c18570 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.680+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51984/0 (socket says 172.21.15.45:51984) 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 -- 172.21.15.45:0/2206080073 learned_addr learned my addr 172.21.15.45:0/2206080073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8791e71640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8791670640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c24220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 -- 172.21.15.45:0/2206080073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 msgr2=0x557874c10650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c10650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 -- 172.21.15.45:0/2206080073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 msgr2=0x557874c24220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c24220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 -- 172.21.15.45:0/2206080073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578749507d0 con 0x557874c0dea0 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8791670640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c24220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f877c01b840 tx=0x7f877c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6dcebd4889cea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8776ffd640 1 -- 172.21.15.45:0/2206080073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00b400 con 0x557874c0dea0 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8776ffd640 1 -- 172.21.15.45:0/2206080073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c02d070 con 0x557874c0dea0 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.681+0000 7f8776ffd640 1 -- 172.21.15.45:0/2206080073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0053e0 con 0x557874c0dea0 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- 172.21.15.45:0/2206080073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 msgr2=0x557874c092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f877c01b840 tx=0x7f877c005cd0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- 172.21.15.45:0/2206080073 shutdown_connections 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c10650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c092c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 --2- 172.21.15.45:0/2206080073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c24220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- 172.21.15.45:0/2206080073 >> 172.21.15.45:0/2206080073 conn(0x557874c098a0 msgr2=0x557874c0ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- 172.21.15.45:0/2206080073 shutdown_connections 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- 172.21.15.45:0/2206080073 wait complete. 2024-04-25T00:25:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 -- start start 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 Processor -- start 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.682+0000 7f87965e5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b2170 con 0x557874c0dea0 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b1aa0 con 0x557874c0d580 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1aece0 con 0x557874c18570 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 10 client.0 _ll_register_callbacks cb 0x7f877c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51992/0 (socket says 172.21.15.45:51992) 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 -- 172.21.15.45:0/2320122997 learned_addr learned my addr 172.21.15.45:0/2320122997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8791e71640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8791670640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 -- 172.21.15.45:0/2320122997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 msgr2=0x557874c1c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 -- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 msgr2=0x557874c20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c20920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 -- 172.21.15.45:0/2320122997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557874c1c810 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8790e6f640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f877c004010 tx=0x7f877c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e4839c51693f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035020 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f877c017040 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c02d070 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f877c01b5f0 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 10 client.16809 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.683+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557874c8e9e0 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c02d070 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f877c004b10 con 0x557874c0dea0 2024-04-25T00:25:50.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 10 client.16809 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 20 client.16809 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c8f0a0 0x557874c914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 4 client.16809 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557874b79f00 con 0x557874c8f0a0 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.684+0000 7f8791670640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c8f0a0 0x557874c914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f8774ff9640 10 client.16809 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f8791670640 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c8f0a0 0x557874c914c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570dac3ba917dd3d server_cookie=114a843f2136ca99 in_seq=0 out_seq=0 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f8774ff9640 1 -- 172.21.15.45:0/2320122997 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557874b79f00 con 0x557874c8f0a0 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f8774ff9640 10 client.16809 handle_command_reply: tid=0 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 client.16809 shutdown 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 msgr2=0x557874c1bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f877c004010 tx=0x7f877c005040 comp rx=0 tx=0).stop 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 shutdown_connections 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557874c18570 0x557874c1c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557874c8f0a0 0x557874c914c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557874c0dea0 0x557874c1bc90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 --2- 172.21.15.45:0/2320122997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557874c0d580 0x557874c20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f8774ff9640 0 client.16809 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 >> 172.21.15.45:0/2320122997 conn(0x557874c098a0 msgr2=0x557874c136b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 shutdown_connections 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 1 -- 172.21.15.45:0/2320122997 wait complete. 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.685+0000 7f87965e5740 20 client.16809 trim_cache size 0 max 16384 2024-04-25T00:25:50.692 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 msgr2=0x7f87780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 0x7f87780a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8784021000 tx=0x7f8784020000 comp rx=0 tx=0).stop 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 msgr2=0x7f878c19a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8788006010 tx=0x7f87880042f0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 shutdown_connections 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c10bd00 0x7f878c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878c1032e0 0x7f878c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 --2- 172.21.15.45:0/2286760267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878c1029c0 0x7f878c19a2c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:50.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 >> 172.21.15.45:0/2286760267 conn(0x7f878c077740 msgr2=0x7f878c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:50.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 shutdown_connections 2024-04-25T00:25:50.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:50.686+0000 7f87938d4640 1 -- 172.21.15.45:0/2286760267 wait complete. 2024-04-25T00:25:50.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:50 smithi045 ceph-mon[29194]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:50.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:50 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3291695395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:50 smithi122 ceph-mon[32959]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:50 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3291695395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:51.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:50 smithi071 ceph-mon[32667]: pgmap v119: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:25:51.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:50 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3291695395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:51.698 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:25:51.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:25:51.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:25:51.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.816+0000 7ff3c8d77640 1 -- 172.21.15.45:0/398059606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 msgr2=0x7ff3c410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.816+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/398059606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c410e5e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009cd0 tx=0x7ff3bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 -- 172.21.15.45:0/398059606 shutdown_connections 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/398059606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c410e5e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/398059606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4101710 0x7ff3c4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/398059606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4107720 0x7ff3c4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 -- 172.21.15.45:0/398059606 >> 172.21.15.45:0/398059606 conn(0x7ff3c40fd180 msgr2=0x7ff3c40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 -- 172.21.15.45:0/398059606 shutdown_connections 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 -- 172.21.15.45:0/398059606 wait complete. 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.817+0000 7ff3c8d77640 1 Processor -- start 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 -- start start 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4101710 0x7ff3c4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4107720 0x7ff3c419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4111b30 con 0x7ff3c41020d0 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41119b0 con 0x7ff3c4107720 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c8d77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4111cb0 con 0x7ff3c4101710 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52010/0 (socket says 172.21.15.45:52010) 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/3625000628 learned_addr learned my addr 172.21.15.45:0/3625000628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4107720 0x7ff3c419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c259c640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4101710 0x7ff3c4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/3625000628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4101710 msgr2=0x7ff3c4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4101710 0x7ff3c4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/3625000628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4107720 msgr2=0x7ff3c419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4107720 0x7ff3c419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/3625000628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c419cff0 con 0x7ff3c41020d0 2024-04-25T00:25:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.818+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800e9f0 tx=0x7ff3b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.819+0000 7ff3b37fe640 1 -- 172.21.15.45:0/3625000628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800cd80 con 0x7ff3c41020d0 2024-04-25T00:25:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.819+0000 7ff3b37fe640 1 -- 172.21.15.45:0/3625000628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8010910 con 0x7ff3c41020d0 2024-04-25T00:25:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.819+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c419d2e0 con 0x7ff3c41020d0 2024-04-25T00:25:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.819+0000 7ff3b37fe640 1 -- 172.21.15.45:0/3625000628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8018990 con 0x7ff3c41020d0 2024-04-25T00:25:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.819+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41a4b70 con 0x7ff3c41020d0 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 Processor -- start 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3b37fe640 1 -- 172.21.15.45:0/3625000628 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3b8010ac0 con 0x7ff3c41020d0 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 -- start start 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf5980 0x55c4b0cf3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d05810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0c582c0 con 0x55c4b0cf4290 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0c00910 con 0x55c4b0d03420 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0c54280 con 0x55c4b0cf5980 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3c259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3c259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52022/0 (socket says 172.21.15.45:52022) 2024-04-25T00:25:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.822+0000 7ff3c259c640 1 -- 172.21.15.45:0/3630995275 learned_addr learned my addr 172.21.15.45:0/3630995275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf5980 0x55c4b0cf3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d05810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3b37fe640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 0x7ff3ac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 -- 172.21.15.45:0/3630995275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf5980 msgr2=0x55c4b0cf3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf5980 0x55c4b0cf3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 -- 172.21.15.45:0/3630995275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 msgr2=0x55c4b0d05810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d05810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 -- 172.21.15.45:0/3630995275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b0c11fb0 con 0x55c4b0cf4290 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 0x7ff3ac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3b37fe640 1 -- 172.21.15.45:0/3625000628 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff3b80cfb50 con 0x7ff3c41020d0 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3c259c640 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400bc60 tx=0x7ff3b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b29dd5ac67805d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3b0ff9640 1 -- 172.21.15.45:0/3630995275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4015070 con 0x55c4b0cf4290 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3b0ff9640 1 -- 172.21.15.45:0/3630995275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4011920 con 0x55c4b0cf4290 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.823+0000 7ff3b0ff9640 1 -- 172.21.15.45:0/3630995275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40108e0 con 0x55c4b0cf4290 2024-04-25T00:25:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- 172.21.15.45:0/3630995275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 msgr2=0x55c4b0cf4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400bc60 tx=0x7ff3b400f730 comp rx=0 tx=0).stop 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 0x7ff3ac0a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009a60 tx=0x7ff3bc03e040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- 172.21.15.45:0/3630995275 shutdown_connections 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d05810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf4290 0x55c4b0cf4710 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/3630995275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf5980 0x55c4b0cf3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- 172.21.15.45:0/3630995275 >> 172.21.15.45:0/3630995275 conn(0x55c4b0cbc950 msgr2=0x55c4b0cf07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- 172.21.15.45:0/3630995275 shutdown_connections 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- 172.21.15.45:0/3630995275 wait complete. 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.824+0000 7ff3cbb65740 1 -- start start 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 Processor -- start 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d00300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d00d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0d062b0 con 0x55c4b0cf5980 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0c13e90 con 0x55c4b0d03420 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4b0d06430 con 0x55c4b0cf4290 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3cbb65740 10 client.0 _ll_register_callbacks cb 0x55c4b0c4f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52032/0 (socket says 172.21.15.45:52032) 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/1358491696 learned_addr learned my addr 172.21.15.45:0/1358491696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c259c640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d00300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d00d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/1358491696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 msgr2=0x55c4b0d00300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d00300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 msgr2=0x55c4b0d00d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d00d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/1358491696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b0a347e0 con 0x55c4b0cf5980 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d00d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c6a0 tx=0x7ff3b809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7689314f77e5b150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.825+0000 7ff3a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8018d30 con 0x55c4b0cf5980 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff3b809b160 con 0x55c4b0cf5980 2024-04-25T00:25:51.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b4011920 con 0x55c4b0cf5980 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3b809bd10 con 0x55c4b0cf5980 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 10 client.16821 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4b0a94f70 con 0x55c4b0cf5980 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4011920 con 0x55c4b0cf5980 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b809abd0 con 0x55c4b0cf5980 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 10 client.16821 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 20 client.16821 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:51.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d13530 0x55c4b0d15960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 4 client.16821 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.826+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b0c2c5a0 con 0x55c4b0d13530 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.827+0000 7ff3c259c640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d13530 0x55c4b0d15960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.827+0000 7ff3c259c640 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d13530 0x55c4b0d15960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a8e9c01475a33 server_cookie=df79f6de8298ef2f in_seq=0 out_seq=0 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.827+0000 7ff3a9ffb640 10 client.16821 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/1358491696 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4b0c2c5a0 con 0x55c4b0d13530 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3a9ffb640 10 client.16821 handle_command_reply: tid=0 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 client.16821 shutdown 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 msgr2=0x55c4b0d00840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c6a0 tx=0x7ff3b809b950 comp rx=0 tx=0).stop 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 shutdown_connections 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0d00d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d13530 0x55c4b0d15960 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d00840 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/1358491696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d00300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3a9ffb640 0 client.16821 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 >> 172.21.15.45:0/1358491696 conn(0x55c4b0cbc950 msgr2=0x55c4b0cf65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 shutdown_connections 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 1 -- 172.21.15.45:0/1358491696 wait complete. 2024-04-25T00:25:51.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.828+0000 7ff3cbb65740 20 client.16821 trim_cache size 0 max 16384 2024-04-25T00:25:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 Processor -- start 2024-04-25T00:25:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 -- start start 2024-04-25T00:25:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ac9d0 con 0x55c4b0cf5980 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ae510 con 0x55c4b0d03420 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ad0a0 con 0x55c4b0cf4290 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52044/0 (socket says 172.21.15.45:52044) 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/213253532 learned_addr learned my addr 172.21.15.45:0/213253532 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/213253532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 msgr2=0x55c4b0d0f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/213253532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 msgr2=0x55c4b0cef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/213253532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b0cfbf00 con 0x55c4b0cf5980 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.846+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff3b809bca0 tx=0x7ff3b8097940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ad19319fee4adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3abfff640 1 -- 172.21.15.45:0/213253532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b809c410 con 0x55c4b0cf5980 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3abfff640 1 -- 172.21.15.45:0/213253532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b80a00c0 con 0x55c4b0cf5980 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3abfff640 1 -- 172.21.15.45:0/213253532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b809b3c0 con 0x55c4b0cf5980 2024-04-25T00:25:51.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 -- 172.21.15.45:0/213253532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 msgr2=0x55c4b0cff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff3b809bca0 tx=0x7ff3b8097940 comp rx=0 tx=0).stop 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 -- 172.21.15.45:0/213253532 shutdown_connections 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0cff460 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/213253532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 -- 172.21.15.45:0/213253532 >> 172.21.15.45:0/213253532 conn(0x55c4b0cbc950 msgr2=0x55c4b0cef0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 -- 172.21.15.45:0/213253532 shutdown_connections 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.847+0000 7ff3cbb65740 1 -- 172.21.15.45:0/213253532 wait complete. 2024-04-25T00:25:51.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 -- start start 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 Processor -- start 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cfc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ade40 con 0x55c4b0cf5980 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ad770 con 0x55c4b0d03420 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41aa9b0 con 0x55c4b0cf4290 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3cbb65740 10 client.0 _ll_register_callbacks cb 0x7ff3b8098070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52056/0 (socket says 172.21.15.45:52056) 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/429573196 learned_addr learned my addr 172.21.15.45:0/429573196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c2d9d640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cfc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c259c640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/429573196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 msgr2=0x55c4b0d0b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 msgr2=0x55c4b0cfc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cfc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c1d9b640 1 -- 172.21.15.45:0/429573196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b092c170 con 0x55c4b0cf5980 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.848+0000 7ff3c259c640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3c1d9b640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c6a0 tx=0x7ff3b8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d192674a7f51773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8018d30 con 0x55c4b0cf5980 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff3b80a4bd0 con 0x55c4b0cf5980 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b80a00c0 con 0x55c4b0cf5980 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3b80a4dd0 con 0x55c4b0cf5980 2024-04-25T00:25:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 10 client.16827 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4b092c170 con 0x55c4b0cf5980 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b80a00c0 con 0x55c4b0cf5980 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b80a13e0 con 0x55c4b0cf5980 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 10 client.16827 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 20 client.16827 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d7aab0 0x55c4b0d7cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 4 client.16827 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.849+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4b0c65800 con 0x55c4b0d7aab0 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.850+0000 7ff3c259c640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d7aab0 0x55c4b0d7cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.850+0000 7ff3c259c640 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d7aab0 0x55c4b0d7cea0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93379826012b8fa server_cookie=61b9e95e6229fa1 in_seq=0 out_seq=0 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.850+0000 7ff3a9ffb640 10 client.16827 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3a9ffb640 1 -- 172.21.15.45:0/429573196 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4b0c65800 con 0x55c4b0d7aab0 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3a9ffb640 10 client.16827 handle_command_reply: tid=0 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 client.16827 shutdown 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 msgr2=0x55c4b0d0bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c6a0 tx=0x7ff3b8018420 comp rx=0 tx=0).stop 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 shutdown_connections 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b0d03420 0x55c4b0cfc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c4b0d7aab0 0x55c4b0d7cea0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c4b0cf5980 0x55c4b0d0bc00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 --2- 172.21.15.45:0/429573196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4b0cf4290 0x55c4b0d0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3a9ffb640 0 client.16827 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 >> 172.21.15.45:0/429573196 conn(0x55c4b0cbc950 msgr2=0x55c4b0cef0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 shutdown_connections 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.851+0000 7ff3cbb65740 1 -- 172.21.15.45:0/429573196 wait complete. 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3cbb65740 20 client.16827 trim_cache size 0 max 16384 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "73b931fe-05f9-4b12-bc47-e8a520c1acb2", 2024-04-25T00:25:51.862 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:25:51.863 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 msgr2=0x7ff3ac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 0x7ff3ac0a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009a60 tx=0x7ff3bc03e040 comp rx=0 tx=0).stop 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 msgr2=0x7ff3c4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800e9f0 tx=0x7ff3b800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 shutdown_connections 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff3ac0a2d00 0x7ff3ac0a51c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3c4107720 0x7ff3c419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3c41020d0 0x7ff3c4198590 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 --2- 172.21.15.45:0/3625000628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3c4101710 0x7ff3c4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 >> 172.21.15.45:0/3625000628 conn(0x7ff3c40fd180 msgr2=0x7ff3c40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 shutdown_connections 2024-04-25T00:25:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.852+0000 7ff3c8d77640 1 -- 172.21.15.45:0/3625000628 wait complete. 2024-04-25T00:25:51.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:51.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 -- 172.21.15.45:0/2382956256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81029c0 msgr2=0x7f21f8102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/2382956256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81029c0 0x7f21f8102da0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec02a510 comp rx=0 tx=0).stop 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 -- 172.21.15.45:0/2382956256 shutdown_connections 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/2382956256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/2382956256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81032e0 0x7f21f810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/2382956256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81029c0 0x7f21f8102da0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 -- 172.21.15.45:0/2382956256 >> 172.21.15.45:0/2382956256 conn(0x7f21f8077740 msgr2=0x7f21f8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 -- 172.21.15.45:0/2382956256 shutdown_connections 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.969+0000 7f21fe4ee640 1 -- 172.21.15.45:0/2382956256 wait complete. 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 Processor -- start 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 -- start start 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81029c0 0x7f21f8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8111b00 con 0x7f21f81032e0 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8111980 con 0x7f21f810bd00 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21fe4ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21f8111c80 con 0x7f21f81029c0 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52078/0 (socket says 172.21.15.45:52078) 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21f77fe640 1 -- 172.21.15.45:0/841077721 learned_addr learned my addr 172.21.15.45:0/841077721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.970+0000 7f21f7fff640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81029c0 0x7f21f8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21fca8b640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 -- 172.21.15.45:0/841077721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81029c0 msgr2=0x7f21f8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81029c0 0x7f21f8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 -- 172.21.15.45:0/841077721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 msgr2=0x7f21f819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 -- 172.21.15.45:0/841077721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f819d010 con 0x7f21f81032e0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21fca8b640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f77fe640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e9f0 tx=0x7f21e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f57fa640 1 -- 172.21.15.45:0/841077721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e800cd80 con 0x7f21f81032e0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f57fa640 1 -- 172.21.15.45:0/841077721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e8005500 con 0x7f21f81032e0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f819d300 con 0x7f21f81032e0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21f57fa640 1 -- 172.21.15.45:0/841077721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8010690 con 0x7f21f81032e0 2024-04-25T00:25:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.971+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f81a4b90 con 0x7f21f81032e0 2024-04-25T00:25:51.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.973+0000 7f21f57fa640 1 -- 172.21.15.45:0/841077721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f21e80046c0 con 0x7f21f81032e0 2024-04-25T00:25:51.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 Processor -- start 2024-04-25T00:25:51.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 -- start start 2024-04-25T00:25:51.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 0x564bf4ca4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bf4bfc6b0 con 0x564bf4c97f50 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564bf4ba4880 con 0x564bf4ca2620 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f22011fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564bf4bf8670 con 0x564bf4c97630 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21f57fa640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 0x7f21e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52082/0 (socket says 172.21.15.45:52082) 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21f7fff640 1 -- 172.21.15.45:0/880025119 learned_addr learned my addr 172.21.15.45:0/880025119 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21fca8b640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.974+0000 7f21f77fe640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 0x564bf4ca4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21fca8b640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 0x7f21e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 -- 172.21.15.45:0/880025119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 msgr2=0x564bf4c97a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f57fa640 1 -- 172.21.15.45:0/841077721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f21e80cfce0 con 0x7f21f81032e0 2024-04-25T00:25:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c97a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 -- 172.21.15.45:0/880025119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 msgr2=0x564bf4ca4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 0x564bf4ca4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 -- 172.21.15.45:0/880025119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf49da7d0 con 0x564bf4c97f50 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f77fe640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 0x564bf4ca4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21f7fff640 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b422f48ff5432797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21fca8b640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 0x7f21e00a51c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21f00094d0 tx=0x7f21f0009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21de7fc640 1 -- 172.21.15.45:0/880025119 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec040070 con 0x564bf4c97f50 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21de7fc640 1 -- 172.21.15.45:0/880025119 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec031870 con 0x564bf4c97f50 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.975+0000 7f21de7fc640 1 -- 172.21.15.45:0/880025119 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec035040 con 0x564bf4c97f50 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- 172.21.15.45:0/880025119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 msgr2=0x564bf4ca20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec005be0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- 172.21.15.45:0/880025119 shutdown_connections 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4ca2620 0x564bf4ca4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4ca20e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 --2- 172.21.15.45:0/880025119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- 172.21.15.45:0/880025119 >> 172.21.15.45:0/880025119 conn(0x564bf4c93950 msgr2=0x564bf4c93d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- 172.21.15.45:0/880025119 shutdown_connections 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- 172.21.15.45:0/880025119 wait complete. 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 -- start start 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.976+0000 7f22011fb740 1 Processor -- start 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4ca2620 0x564bf4c9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bf4ca53b0 con 0x564bf4c97f50 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564bf4bb7f50 con 0x564bf4c97630 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564bf4ca5530 con 0x564bf4ca2620 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 10 client.0 _ll_register_callbacks cb 0x564bf4bf3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52086/0 (socket says 172.21.15.45:52086) 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 -- 172.21.15.45:0/2805348196 learned_addr learned my addr 172.21.15.45:0/2805348196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21fca8b640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4ca2620 0x564bf4c9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 -- 172.21.15.45:0/2805348196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4ca2620 msgr2=0x564bf4c9ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4ca2620 0x564bf4c9ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 -- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 msgr2=0x564bf4c9f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 -- 172.21.15.45:0/2805348196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf49103f0 con 0x564bf4c97f50 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21f77fe640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f21e809bdd0 tx=0x7f21e806bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66625c366da03c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e809e060 con 0x564bf4c97f50 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f21e80a4820 con 0x564bf4c97f50 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec031870 con 0x564bf4c97f50 2024-04-25T00:25:51.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21e80cd500 con 0x564bf4c97f50 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 10 client.16839 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.977+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bf48d00f0 con 0x564bf4c97f50 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec031870 con 0x564bf4c97f50 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21e809b290 con 0x564bf4c97f50 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 10 client.16839 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 20 client.16839 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4cb2830 0x564bf4cb4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 4 client.16839 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bf4c09bf0 con 0x564bf4cb2830 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4cb2830 0x564bf4cb4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f21d77fe640 10 client.16839 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.978+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4cb2830 0x564bf4cb4c60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd761dc1758ad9 server_cookie=ce5acdb99d88c847 in_seq=0 out_seq=0 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f21d77fe640 1 -- 172.21.15.45:0/2805348196 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bf4c09bf0 con 0x564bf4cb2830 2024-04-25T00:25:51.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f21d77fe640 10 client.16839 handle_command_reply: tid=0 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 client.16839 shutdown 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 msgr2=0x564bf4c9fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f21e809bdd0 tx=0x7f21e806bd60 comp rx=0 tx=0).stop 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 shutdown_connections 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4ca2620 0x564bf4c9ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4cb2830 0x564bf4cb4c60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4c9fa30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 --2- 172.21.15.45:0/2805348196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f21d77fe640 0 client.16839 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 >> 172.21.15.45:0/2805348196 conn(0x564bf4c93950 msgr2=0x564bf4c99750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:51.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 shutdown_connections 2024-04-25T00:25:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.979+0000 7f22011fb740 1 -- 172.21.15.45:0/2805348196 wait complete. 2024-04-25T00:25:51.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.980+0000 7f22011fb740 20 client.16839 trim_cache size 0 max 16384 2024-04-25T00:25:52.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 Processor -- start 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 -- start start 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c99ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97f50 0x564bf4cba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81aeaa0 con 0x564bf4c93950 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81b05e0 con 0x564bf4c97f50 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f22011fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81af170 con 0x564bf4c97630 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f21f7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52102/0 (socket says 172.21.15.45:52102) 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.997+0000 7f21f7fff640 1 -- 172.21.15.45:0/2943392787 learned_addr learned my addr 172.21.15.45:0/2943392787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f77fe640 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c99ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 -- 172.21.15.45:0/2943392787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 msgr2=0x564bf4c99ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:52.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c99ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 -- 172.21.15.45:0/2943392787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97f50 msgr2=0x564bf4cba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21fca8b640 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97f50 0x564bf4cba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97f50 0x564bf4cba530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 -- 172.21.15.45:0/2943392787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf4c85090 con 0x564bf4c93950 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21f7fff640 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c6463e2d446b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21ddffb640 1 -- 172.21.15.45:0/2943392787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec056070 con 0x564bf4c93950 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21ddffb640 1 -- 172.21.15.45:0/2943392787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec031870 con 0x564bf4c93950 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f21ddffb640 1 -- 172.21.15.45:0/2943392787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec040050 con 0x564bf4c93950 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.998+0000 7f22011fb740 1 -- 172.21.15.45:0/2943392787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 msgr2=0x564bf4cae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009900 tx=0x7f21ec02a510 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- 172.21.15.45:0/2943392787 shutdown_connections 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97f50 0x564bf4cba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c97630 0x564bf4c99ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- 172.21.15.45:0/2943392787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c93950 0x564bf4cae270 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- 172.21.15.45:0/2943392787 >> 172.21.15.45:0/2943392787 conn(0x564bf4c9e130 msgr2=0x564bf4c9fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- 172.21.15.45:0/2943392787 shutdown_connections 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- 172.21.15.45:0/2943392787 wait complete. 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- start start 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 Processor -- start 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c93950 0x564bf4c9c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81aff10 con 0x564bf4c97f50 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81af840 con 0x564bf4c97630 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21f81aca80 con 0x564bf4c93950 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f22011fb740 10 client.0 _ll_register_callbacks cb 0x7f21e800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:51.999+0000 7f21fca8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52106/0 (socket says 172.21.15.45:52106) 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 -- 172.21.15.45:0/3052053274 learned_addr learned my addr 172.21.15.45:0/3052053274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:25:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21f7fff640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c93950 0x564bf4c9c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21f77fe640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 -- 172.21.15.45:0/3052053274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c93950 msgr2=0x564bf4c9c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c93950 0x564bf4c9c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 -- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 msgr2=0x564bf4c9ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 -- 172.21.15.45:0/3052053274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf4d182e0 con 0x564bf4c97f50 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21fca8b640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f21f00042b0 tx=0x7f21f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6ef72bec9d2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0011cf0 con 0x564bf4c97f50 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f21f002a440 con 0x564bf4c97f50 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ec031870 con 0x564bf4c97f50 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21f0033050 con 0x564bf4c97f50 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f22011fb740 10 client.16845 fetch_fsmap learned FSMap version 28 2024-04-25T00:25:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bf4d19450 con 0x564bf4c97f50 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.000+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec031870 con 0x564bf4c97f50 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21f0028020 con 0x564bf4c97f50 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 10 client.16845 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 20 client.16845 populate_metadata read hostname 'smithi045' 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4ca2620 0x564bf4d1bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 4 client.16845 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bf4c03f40 con 0x564bf4ca2620 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21f7fff640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4ca2620 0x564bf4d1bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21f7fff640 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4ca2620 0x564bf4d1bc70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9ef362c881d6d4 server_cookie=4c4e1b420687fbf2 in_seq=0 out_seq=0 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21d77fe640 10 client.16845 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21d77fe640 1 -- 172.21.15.45:0/3052053274 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564bf4c03f40 con 0x564bf4ca2620 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f21d77fe640 10 client.16845 handle_command_reply: tid=0 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.001+0000 7f22011fb740 1 client.16845 shutdown 2024-04-25T00:25:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 msgr2=0x564bf4d17da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f21f00042b0 tx=0x7f21f0004a40 comp rx=0 tx=0).stop 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 shutdown_connections 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564bf4ca2620 0x564bf4d1bc70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bf4c97f50 0x564bf4d17da0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21d77fe640 0 client.16845 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bf4c97630 0x564bf4c9ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 --2- 172.21.15.45:0/3052053274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564bf4c93950 0x564bf4c9c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 >> 172.21.15.45:0/3052053274 conn(0x564bf4c9e130 msgr2=0x564bf4c92260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 shutdown_connections 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 1 -- 172.21.15.45:0/3052053274 wait complete. 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f22011fb740 20 client.16845 trim_cache size 0 max 16384 2024-04-25T00:25:52.009 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "73b931fe-05f9-4b12-bc47-e8a520c1acb2": { 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "73b931fe-05f9-4b12-bc47-e8a520c1acb2", 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:25:52.010 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 msgr2=0x7f21e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 0x7f21e00a51c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21f00094d0 tx=0x7f21f0009500 comp rx=0 tx=0).stop 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 msgr2=0x7f21f8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e9f0 tx=0x7f21e800edf0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 shutdown_connections 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f21e00a2d00 0x7f21e00a51c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21f810bd00 0x7f21f819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21f81032e0 0x7f21f8198560 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 --2- 172.21.15.45:0/841077721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21f81029c0 0x7f21f8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:52.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 >> 172.21.15.45:0/841077721 conn(0x7f21f8077740 msgr2=0x7f21f80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:52.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 shutdown_connections 2024-04-25T00:25:52.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:25:52.002+0000 7f21fe4ee640 1 -- 172.21.15.45:0/841077721 wait complete. 2024-04-25T00:25:52.014 INFO:tasks.cephfs.filesystem:scrub status for tag:73b931fe-05f9-4b12-bc47-e8a520c1acb2 - {'path': '/', 'tag': '73b931fe-05f9-4b12-bc47-e8a520c1acb2', 'options': 'recursive,force'} 2024-04-25T00:25:52.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:25:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:25:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:52 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:52 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: pgmap v120: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:53.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:52 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:25:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:54 smithi045 ceph-mon[29194]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:54 smithi122 ceph-mon[32959]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:54 smithi071 ceph-mon[32667]: pgmap v121: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:25:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:56 smithi045 ceph-mon[29194]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:57.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:56 smithi122 ceph-mon[32959]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:57.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:56 smithi071 ceph-mon[32667]: pgmap v122: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:25:58.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:25:58 smithi045 ceph-mon[29194]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:59.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:25:58 smithi122 ceph-mon[32959]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:25:59.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:25:58 smithi071 ceph-mon[32667]: pgmap v123: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:00 smithi045 ceph-mon[29194]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:00 smithi122 ceph-mon[32959]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:00 smithi071 ceph-mon[32667]: pgmap v124: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:02 smithi045 ceph-mon[29194]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:02 smithi122 ceph-mon[32959]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:02 smithi071 ceph-mon[32667]: pgmap v125: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:04 smithi122 ceph-mon[32959]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:26:05.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:04 smithi071 ceph-mon[32667]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:26:05.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:04 smithi045 ceph-mon[29194]: pgmap v126: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:26:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:06 smithi122 ceph-mon[32959]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:26:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:06 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:26:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:06 smithi071 ceph-mon[32667]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:26:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:06 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:26:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:06 smithi045 ceph-mon[29194]: pgmap v127: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:26:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:06 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:26:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:08 smithi122 ceph-mon[32959]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:08 smithi071 ceph-mon[32667]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:08 smithi045 ceph-mon[29194]: pgmap v128: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:10 smithi122 ceph-mon[32959]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:26:11.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:10 smithi071 ceph-mon[32667]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:26:11.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:10 smithi045 ceph-mon[29194]: pgmap v129: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:26:12.921 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:12 smithi071 ceph-mon[32667]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:12 smithi122 ceph-mon[32959]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:13.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:12 smithi045 ceph-mon[29194]: pgmap v130: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:14 smithi122 ceph-mon[32959]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:26:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:14 smithi071 ceph-mon[32667]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:26:15.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:14 smithi045 ceph-mon[29194]: pgmap v131: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:26:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:16 smithi122 ceph-mon[32959]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:17.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:16 smithi071 ceph-mon[32667]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:16 smithi045 ceph-mon[29194]: pgmap v132: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:26:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:18 smithi122 ceph-mon[32959]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:26:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:18 smithi071 ceph-mon[32667]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:26:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:18 smithi045 ceph-mon[29194]: pgmap v133: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:26:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:20 smithi122 ceph-mon[32959]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:26:21.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:20 smithi071 ceph-mon[32667]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:26:21.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:20 smithi045 ceph-mon[29194]: pgmap v134: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:26:22.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:22.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- 172.21.15.45:0/3002123429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 msgr2=0x7f70081103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3002123429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f70081103f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f700002a950 comp rx=0 tx=0).stop 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- 172.21.15.45:0/3002123429 shutdown_connections 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3002123429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f70081103f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3002123429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3002123429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 0x7f7008069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- 172.21.15.45:0/3002123429 >> 172.21.15.45:0/3002123429 conn(0x7f7008076440 msgr2=0x7f7008076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- 172.21.15.45:0/3002123429 shutdown_connections 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- 172.21.15.45:0/3002123429 wait complete. 2024-04-25T00:26:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 Processor -- start 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.133+0000 7f700dd5c640 1 -- start start 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 0x7f700819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7008113cc0 con 0x7f700810e000 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7008113b40 con 0x7f700806a280 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7008113e40 con 0x7f70081026f0 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49430/0 (socket says 172.21.15.45:49430) 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 -- 172.21.15.45:0/3327667019 learned_addr learned my addr 172.21.15.45:0/3327667019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f70077fe640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 0x7f700819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 -- 172.21.15.45:0/3327667019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 msgr2=0x7f700819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 0x7f700819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 -- 172.21.15.45:0/3327667019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 msgr2=0x7f700819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 -- 172.21.15.45:0/3327667019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700819f170 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f70077fe640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f7007fff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7000004af0 tx=0x7f700002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f70057fa640 1 -- 172.21.15.45:0/3327667019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000034040 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f70057fa640 1 -- 172.21.15.45:0/3327667019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000030920 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f70057fa640 1 -- 172.21.15.45:0/3327667019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000004780 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.134+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700819f400 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.135+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a6cf0 con 0x7f700810e000 2024-04-25T00:26:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.137+0000 7f70057fa640 1 -- 172.21.15.45:0/3327667019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7000037020 con 0x7f700810e000 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.137+0000 7f7010a60740 1 Processor -- start 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.137+0000 7f7010a60740 1 -- start start 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 0x557bfe07ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe08ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557bfdfe36b0 con 0x557bfe07ef50 2024-04-25T00:26:22.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557bfdf8b880 con 0x557bfe07e630 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7010a60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557bfdfdf670 con 0x557bfe089620 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70057fa640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 0x7f6ff40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 0x557bfe07ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49432/0 (socket says 172.21.15.45:49432) 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 -- 172.21.15.45:0/1912527049 learned_addr learned my addr 172.21.15.45:0/1912527049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f7007fff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 0x7f6ff40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 0x557bfe07ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:53402/0 (socket says 172.21.15.45:53402) 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f6ffffff640 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe08ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70057fa640 1 -- 172.21.15.45:0/3327667019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f70000f05a0 con 0x7f700810e000 2024-04-25T00:26:22.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 -- 172.21.15.45:0/1912527049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 msgr2=0x557bfe08ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe08ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 -- 172.21.15.45:0/1912527049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 msgr2=0x557bfe07ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 0x557bfe07ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.138+0000 7f70077fe640 1 -- 172.21.15.45:0/1912527049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfddc17d0 con 0x557bfe07ef50 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f7007fff640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 0x7f6ff40a51c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70000bc7a0 tx=0x7f7000087830 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f70077fe640 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8008be0 tx=0x7f6ff800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63f01816c31428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f6ffdffb640 1 -- 172.21.15.45:0/1912527049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8014070 con 0x557bfe07ef50 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f6ffdffb640 1 -- 172.21.15.45:0/1912527049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff80047a0 con 0x557bfe07ef50 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f6ffdffb640 1 -- 172.21.15.45:0/1912527049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8010650 con 0x557bfe07ef50 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f7010a60740 1 -- 172.21.15.45:0/1912527049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 msgr2=0x557bfe0890e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.139+0000 7f7010a60740 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8008be0 tx=0x7f6ff800e720 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- 172.21.15.45:0/1912527049 shutdown_connections 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe08ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07ef50 0x557bfe0890e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- 172.21.15.45:0/1912527049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07e630 0x557bfe07ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- 172.21.15.45:0/1912527049 >> 172.21.15.45:0/1912527049 conn(0x557bfe07a950 msgr2=0x557bfe07ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- 172.21.15.45:0/1912527049 shutdown_connections 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- 172.21.15.45:0/1912527049 wait complete. 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- start start 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 Processor -- start 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe086b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe087040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557bfe08c410 con 0x557bfe07e630 2024-04-25T00:26:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557bfdf9ef50 con 0x557bfe07ef50 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557bfe08c590 con 0x557bfe089620 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.140+0000 7f7010a60740 10 client.0 _ll_register_callbacks cb 0x557bfdfda000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49448/0 (socket says 172.21.15.45:49448) 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 -- 172.21.15.45:0/1717010103 learned_addr learned my addr 172.21.15.45:0/1717010103 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f70077fe640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe086b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe087040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 -- 172.21.15.45:0/1717010103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 msgr2=0x557bfe087040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe087040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 -- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 msgr2=0x557bfe086b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe086b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 -- 172.21.15.45:0/1717010103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfdcf73f0 con 0x557bfe07e630 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6ffffff640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0003850 tx=0x7f6ff0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38d4ea9364481b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0005020 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6ff00051c0 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff80047a0 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0029050 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f7010a60740 10 client.16857 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.141+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557bfdcb70f0 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff80047a0 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ff00214e0 con 0x557bfe07e630 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 10 client.16857 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 20 client.16857 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe0998e0 0x557bfe09bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 4 client.16857 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bfdff0bf0 con 0x557bfe0998e0 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f70077fe640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe0998e0 0x557bfe09bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f70077fe640 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe0998e0 0x557bfe09bcf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa340553d1dbcc9 server_cookie=9fcadfc7812a7744 in_seq=0 out_seq=0 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.142+0000 7f6fe6ffd640 10 client.16857 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/1717010103 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bfdff0bf0 con 0x557bfe0998e0 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f6fe6ffd640 10 client.16857 handle_command_reply: tid=0 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 client.16857 shutdown 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 msgr2=0x557bfe0865c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0003850 tx=0x7f6ff0014040 comp rx=0 tx=0).stop 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 shutdown_connections 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe087040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe086b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe0998e0 0x557bfe09bcf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 --2- 172.21.15.45:0/1717010103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe0865c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f6fe6ffd640 0 client.16857 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:22.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.143+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 >> 172.21.15.45:0/1717010103 conn(0x557bfe07a950 msgr2=0x557bfe080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.144+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 shutdown_connections 2024-04-25T00:26:22.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.144+0000 7f7010a60740 1 -- 172.21.15.45:0/1717010103 wait complete. 2024-04-25T00:26:22.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.144+0000 7f7010a60740 20 client.16857 trim_cache size 0 max 16384 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 Processor -- start 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 -- start start 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe095300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe07ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b0cd0 con 0x557bfe07e630 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b2810 con 0x557bfe07ef50 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f7010a60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b13a0 con 0x557bfe089620 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.161+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49464/0 (socket says 172.21.15.45:49464) 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 -- 172.21.15.45:0/3914060011 learned_addr learned my addr 172.21.15.45:0/3914060011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe07ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f70077fe640 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe095300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 -- 172.21.15.45:0/3914060011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 msgr2=0x557bfe07ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe07ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 -- 172.21.15.45:0/3914060011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 msgr2=0x557bfe095300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe095300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 -- 172.21.15.45:0/3914060011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfe0762a0 con 0x557bfe07e630 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004700 tx=0x7f6ff0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce0d398054e61f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffd7fa640 1 -- 172.21.15.45:0/3914060011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0009530 con 0x557bfe07e630 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffd7fa640 1 -- 172.21.15.45:0/3914060011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0004430 con 0x557bfe07e630 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.162+0000 7f6ffd7fa640 1 -- 172.21.15.45:0/3914060011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00216b0 con 0x557bfe07e630 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- 172.21.15.45:0/3914060011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 msgr2=0x557bfe07a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004700 tx=0x7f6ff0009b30 comp rx=0 tx=0).stop 2024-04-25T00:26:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- 172.21.15.45:0/3914060011 shutdown_connections 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe089620 0x557bfe07ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe07ef50 0x557bfe095300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 --2- 172.21.15.45:0/3914060011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe07a270 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- 172.21.15.45:0/3914060011 >> 172.21.15.45:0/3914060011 conn(0x557bfe07a950 msgr2=0x557bfe087ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- 172.21.15.45:0/3914060011 shutdown_connections 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- 172.21.15.45:0/3914060011 wait complete. 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 -- start start 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.163+0000 7f7010a60740 1 Processor -- start 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe07ef50 0x557bfe083040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe089620 0x557bfe0835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b2140 con 0x557bfe07e630 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b1a70 con 0x557bfe089620 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70081aecb0 con 0x557bfe07ef50 2024-04-25T00:26:22.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f7010a60740 10 client.0 _ll_register_callbacks cb 0x7f6ff0009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49468/0 (socket says 172.21.15.45:49468) 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 -- 172.21.15.45:0/3034637155 learned_addr learned my addr 172.21.15.45:0/3034637155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe089620 0x557bfe0835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f70077fe640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe07ef50 0x557bfe083040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 -- 172.21.15.45:0/3034637155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe07ef50 msgr2=0x557bfe083040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe07ef50 0x557bfe083040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 -- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe089620 msgr2=0x557bfe0835e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe089620 0x557bfe0835e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 -- 172.21.15.45:0/3034637155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfe083e10 con 0x557bfe07e630 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6ffffff640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0021dd0 tx=0x7f6ff0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53c16b9830234cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.164+0000 7f6fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0036040 con 0x557bfe07e630 2024-04-25T00:26:22.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff0004430 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6ff0021670 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0012020 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 10 client.16863 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557bfe0ffeb0 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0004430 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ff0004a60 con 0x557bfe07e630 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 10 client.16863 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 20 client.16863 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe1004c0 0x557bfe1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 4 client.16863 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557bfdfeaf40 con 0x557bfe1004c0 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.165+0000 7f70077fe640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe1004c0 0x557bfe1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f6fe6ffd640 10 client.16863 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f70077fe640 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe1004c0 0x557bfe1028e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a55433a5e3591a server_cookie=48a612c1fb9e107 in_seq=0 out_seq=0 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f6fe6ffd640 1 -- 172.21.15.45:0/3034637155 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557bfdfeaf40 con 0x557bfe1004c0 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f6fe6ffd640 10 client.16863 handle_command_reply: tid=0 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 client.16863 shutdown 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 msgr2=0x557bfe091cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0021dd0 tx=0x7f6ff0005f90 comp rx=0 tx=0).stop 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 shutdown_connections 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557bfe089620 0x557bfe0835e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557bfe07ef50 0x557bfe083040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557bfe1004c0 0x557bfe1028e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 --2- 172.21.15.45:0/3034637155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557bfe07e630 0x557bfe091cf0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 >> 172.21.15.45:0/3034637155 conn(0x557bfe07a950 msgr2=0x557bfe088570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f6fe6ffd640 0 client.16863 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 shutdown_connections 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 1 -- 172.21.15.45:0/3034637155 wait complete. 2024-04-25T00:26:22.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.166+0000 7f7010a60740 20 client.16863 trim_cache size 0 max 16384 2024-04-25T00:26:22.174 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:22.174 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:26:22.174 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:26:22.174 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 msgr2=0x7f6ff40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 0x7f6ff40a51c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f70000bc7a0 tx=0x7f7000087830 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 msgr2=0x7f700819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7000004af0 tx=0x7f700002f5b0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 shutdown_connections 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6ff40a2d00 0x7f6ff40a51c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700810e000 0x7f700819ea40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70081026f0 0x7f700819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 --2- 172.21.15.45:0/3327667019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700806a280 0x7f700819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 >> 172.21.15.45:0/3327667019 conn(0x7f7008076440 msgr2=0x7f70080ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 shutdown_connections 2024-04-25T00:26:22.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.167+0000 7f700dd5c640 1 -- 172.21.15.45:0/3327667019 wait complete. 2024-04-25T00:26:22.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:26:22.179 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- 172.21.15.45:0/2780035252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105680 msgr2=0x7f86bc105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 --2- 172.21.15.45:0/2780035252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105680 0x7f86bc105a60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009930 tx=0x7f86ac02a4d0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- 172.21.15.45:0/2780035252 shutdown_connections 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 --2- 172.21.15.45:0/2780035252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 --2- 172.21.15.45:0/2780035252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc1011a0 0x7f86bc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 --2- 172.21.15.45:0/2780035252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc105680 0x7f86bc105a60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- 172.21.15.45:0/2780035252 >> 172.21.15.45:0/2780035252 conn(0x7f86bc075920 msgr2=0x7f86bc075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- 172.21.15.45:0/2780035252 shutdown_connections 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- 172.21.15.45:0/2780035252 wait complete. 2024-04-25T00:26:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 Processor -- start 2024-04-25T00:26:22.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.281+0000 7f86c187d640 1 -- start start 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc105680 0x7f86bc1a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110a00 con 0x7f86bc1011a0 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110880 con 0x7f86bc101b60 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c187d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110b80 con 0x7f86bc105680 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49496/0 (socket says 172.21.15.45:49496) 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 -- 172.21.15.45:0/2076467575 learned_addr learned my addr 172.21.15.45:0/2076467575 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86b27fc640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86bb7fe640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc105680 0x7f86bc1a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 -- 172.21.15.45:0/2076467575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc105680 msgr2=0x7f86bc1a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc105680 0x7f86bc1a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 -- 172.21.15.45:0/2076467575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 msgr2=0x7f86bc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 -- 172.21.15.45:0/2076467575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc1a4480 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86b27fc640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86baffd640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f86ac005940 tx=0x7f86ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.282+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac03f030 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.283+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac003ce0 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.283+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1a4650 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.283+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac035070 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.283+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a4a50 con 0x7f86bc1011a0 2024-04-25T00:26:22.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.284+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc0689b0 con 0x7f86bc1011a0 2024-04-25T00:26:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.285+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86ac042030 con 0x7f86bc1011a0 2024-04-25T00:26:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.285+0000 7f86c087b640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 0x7f86a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.286+0000 7f86b27fc640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 0x7f86a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.286+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f86ac0efd90 con 0x7f86bc1011a0 2024-04-25T00:26:22.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.286+0000 7f86b27fc640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 0x7f86a80a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f86a40058c0 tx=0x7f86a4005850 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.290+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86ac0bc650 con 0x7f86bc1011a0 2024-04-25T00:26:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.513+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86bc108600 con 0x7f86bc1011a0 2024-04-25T00:26:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.513+0000 7f86c087b640 1 -- 172.21.15.45:0/2076467575 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f86ac0bbf50 con 0x7f86bc1011a0 2024-04-25T00:26:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:26:22.514 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:26:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 msgr2=0x7f86a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 0x7f86a80a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f86a40058c0 tx=0x7f86a4005850 comp rx=0 tx=0).stop 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 msgr2=0x7f86bc198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f86ac005940 tx=0x7f86ac0042f0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 shutdown_connections 2024-04-25T00:26:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f86a80a2d00 0x7f86a80a51c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc105680 0x7f86bc1a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc101b60 0x7f86bc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 --2- 172.21.15.45:0/2076467575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86bc1011a0 0x7f86bc198ec0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.515+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 >> 172.21.15.45:0/2076467575 conn(0x7f86bc075920 msgr2=0x7f86bc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.516+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 shutdown_connections 2024-04-25T00:26:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.516+0000 7f86c187d640 1 -- 172.21.15.45:0/2076467575 wait complete. 2024-04-25T00:26:22.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:26:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.630+0000 7f888de9e640 1 -- 172.21.15.45:0/1344885781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88881032e0 msgr2=0x7f888810b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.630+0000 7f888de9e640 1 --2- 172.21.15.45:0/1344885781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88881032e0 0x7f888810b7c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8878009900 tx=0x7f887802a510 comp rx=0 tx=0).stop 2024-04-25T00:26:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 -- 172.21.15.45:0/1344885781 shutdown_connections 2024-04-25T00:26:22.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 --2- 172.21.15.45:0/1344885781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f888810bd00 0x7f888810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 --2- 172.21.15.45:0/1344885781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88881032e0 0x7f888810b7c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 --2- 172.21.15.45:0/1344885781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 0x7f8888102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 -- 172.21.15.45:0/1344885781 >> 172.21.15.45:0/1344885781 conn(0x7f8888077740 msgr2=0x7f8888077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 -- 172.21.15.45:0/1344885781 shutdown_connections 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 -- 172.21.15.45:0/1344885781 wait complete. 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 Processor -- start 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.631+0000 7f888de9e640 1 -- start start 2024-04-25T00:26:22.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 0x7f8888198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88881032e0 0x7f8888198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8888111b00 con 0x7f888810bd00 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8888111980 con 0x7f88881032e0 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8888111c80 con 0x7f88881029c0 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49516/0 (socket says 172.21.15.45:49516) 2024-04-25T00:26:22.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 -- 172.21.15.45:0/3390418508 learned_addr learned my addr 172.21.15.45:0/3390418508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f88877fe640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 0x7f8888198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88881032e0 0x7f8888198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 -- 172.21.15.45:0/3390418508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 msgr2=0x7f8888198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 0x7f8888198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 -- 172.21.15.45:0/3390418508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88881032e0 msgr2=0x7f8888198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88881032e0 0x7f8888198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 -- 172.21.15.45:0/3390418508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819d010 con 0x7f888810bd00 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8887fff640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8880009fd0 tx=0x7f888000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8884ff9640 1 -- 172.21.15.45:0/3390418508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880017070 con 0x7f888810bd00 2024-04-25T00:26:22.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8884ff9640 1 -- 172.21.15.45:0/3390418508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8880003e50 con 0x7f888810bd00 2024-04-25T00:26:22.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819d300 con 0x7f888810bd00 2024-04-25T00:26:22.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.632+0000 7f8884ff9640 1 -- 172.21.15.45:0/3390418508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800106c0 con 0x7f888810bd00 2024-04-25T00:26:22.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.633+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a4b90 con 0x7f888810bd00 2024-04-25T00:26:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8884ff9640 1 -- 172.21.15.45:0/3390418508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8880003970 con 0x7f888810bd00 2024-04-25T00:26:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 Processor -- start 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 -- start start 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cafe6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1caff5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1caf4b670 con 0x55b1caff5980 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b1caef3970 con 0x55b1cafe62b0 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8890baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b1caf47630 con 0x55b1cafe6c60 2024-04-25T00:26:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49524/0 (socket says 172.21.15.45:49524) 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8886ffd640 1 -- 172.21.15.45:0/3302617526 learned_addr learned my addr 172.21.15.45:0/3302617526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8884ff9640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 0x7f88700a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cafe6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.635+0000 7f88877fe640 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1caff5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 -- 172.21.15.45:0/3302617526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 msgr2=0x55b1caff5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1caff5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 -- 172.21.15.45:0/3302617526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 msgr2=0x55b1cafe6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cafe6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 -- 172.21.15.45:0/3302617526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1caf17400 con 0x55b1caff5980 2024-04-25T00:26:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8884ff9640 1 -- 172.21.15.45:0/3390418508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f88800cf3a0 con 0x7f888810bd00 2024-04-25T00:26:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8887fff640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 0x7f88700a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f887802aa30 tx=0x7f8878005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4c0997c2e553a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f886dffb640 1 -- 172.21.15.45:0/3302617526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878035070 con 0x55b1caff5980 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f886dffb640 1 -- 172.21.15.45:0/3302617526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878004030 con 0x55b1caff5980 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f8887fff640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 0x7f88700a5240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f888000b760 tx=0x7f8880086040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.636+0000 7f886dffb640 1 -- 172.21.15.45:0/3302617526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878005620 con 0x55b1caff5980 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- 172.21.15.45:0/3302617526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 msgr2=0x55b1caff7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f887802aa30 tx=0x7f8878005c00 comp rx=0 tx=0).stop 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- 172.21.15.45:0/3302617526 shutdown_connections 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff7d70 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1caff5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 --2- 172.21.15.45:0/3302617526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cafe6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- 172.21.15.45:0/3302617526 >> 172.21.15.45:0/3302617526 conn(0x55b1cafe28a0 msgr2=0x55b1cafe2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- 172.21.15.45:0/3302617526 shutdown_connections 2024-04-25T00:26:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- 172.21.15.45:0/3302617526 wait complete. 2024-04-25T00:26:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 -- start start 2024-04-25T00:26:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 Processor -- start 2024-04-25T00:26:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.637+0000 7f8890baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1caff5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1caff86e0 con 0x55b1caff5980 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b1caf07050 con 0x55b1cafe6c60 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b1caff8860 con 0x55b1cafe62b0 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 10 client.0 _ll_register_callbacks cb 0x55b1cac5ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49538/0 (socket says 172.21.15.45:49538) 2024-04-25T00:26:22.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 -- 172.21.15.45:0/1568754786 learned_addr learned my addr 172.21.15.45:0/1568754786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f88877fe640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1caff5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 -- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 msgr2=0x55b1caff5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1caff5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 -- 172.21.15.45:0/1568754786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 msgr2=0x55b1caff25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 -- 172.21.15.45:0/1568754786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1caed8dc0 con 0x55b1caff5980 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8886ffd640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8878009900 tx=0x7f8878005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb4064cc9558ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878035040 con 0x55b1caff5980 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8878031630 con 0x55b1caff5980 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8878004030 con 0x55b1caff5980 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f887805b050 con 0x55b1caff5980 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 10 client.16881 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.638+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1cac5f3f0 con 0x55b1caff5980 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878004030 con 0x55b1caff5980 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8878003660 con 0x55b1caff5980 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 10 client.16881 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 10 client.16881 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 20 client.16881 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb005890 0x55b1cb007c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 4 client.16881 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1caf58bb0 con 0x55b1cb005890 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f88877fe640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb005890 0x55b1cb007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f88877fe640 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb005890 0x55b1cb007c80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12f9821e29b54bc9 server_cookie=cfa6b53f54f34858 in_seq=0 out_seq=0 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.639+0000 7f8866ffd640 10 client.16881 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8866ffd640 1 -- 172.21.15.45:0/1568754786 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1caf58bb0 con 0x55b1cb005890 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8866ffd640 10 client.16881 handle_command_reply: tid=0 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 client.16881 shutdown 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 msgr2=0x55b1caff2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8878009900 tx=0x7f8878005260 comp rx=0 tx=0).stop 2024-04-25T00:26:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 shutdown_connections 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb005890 0x55b1cb007c80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1caff2b70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8866ffd640 0 client.16881 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 --2- 172.21.15.45:0/1568754786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1caff5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 >> 172.21.15.45:0/1568754786 conn(0x55b1cafe28a0 msgr2=0x55b1cafe8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 shutdown_connections 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.640+0000 7f8890baa740 1 -- 172.21.15.45:0/1568754786 wait complete. 2024-04-25T00:26:22.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.641+0000 7f8890baa740 20 client.16881 trim_cache size 0 max 16384 2024-04-25T00:26:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 Processor -- start 2024-04-25T00:26:22.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 -- start start 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1cafedaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ac750 con 0x55b1caff5980 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ae290 con 0x55b1cafe6c60 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8890baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ace20 con 0x55b1cafe62b0 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49546/0 (socket says 172.21.15.45:49546) 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 -- 172.21.15.45:0/3741998403 learned_addr learned my addr 172.21.15.45:0/3741998403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f88877fe640 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1cafedaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 -- 172.21.15.45:0/3741998403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 msgr2=0x55b1cafedaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1cafedaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 -- 172.21.15.45:0/3741998403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 msgr2=0x55b1caff4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.654+0000 7f8886ffd640 1 -- 172.21.15.45:0/3741998403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1cad297d0 con 0x55b1caff5980 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8886ffd640 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8878059040 tx=0x7f8878040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68db4e8b8897caa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f886d7fa640 1 -- 172.21.15.45:0/3741998403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878035030 con 0x55b1caff5980 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f886d7fa640 1 -- 172.21.15.45:0/3741998403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887804d040 con 0x55b1caff5980 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f886d7fa640 1 -- 172.21.15.45:0/3741998403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878003850 con 0x55b1caff5980 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 -- 172.21.15.45:0/3741998403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 msgr2=0x55b1cb00aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8878059040 tx=0x7f8878040040 comp rx=0 tx=0).stop 2024-04-25T00:26:22.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 -- 172.21.15.45:0/3741998403 shutdown_connections 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb00aca0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe6c60 0x55b1caff4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 --2- 172.21.15.45:0/3741998403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe62b0 0x55b1cafedaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 -- 172.21.15.45:0/3741998403 >> 172.21.15.45:0/3741998403 conn(0x55b1cafe28a0 msgr2=0x55b1cafed3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 -- 172.21.15.45:0/3741998403 shutdown_connections 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.655+0000 7f8890baa740 1 -- 172.21.15.45:0/3741998403 wait complete. 2024-04-25T00:26:22.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 -- start start 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 Processor -- start 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cb007f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1cb0084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88881adbc0 con 0x55b1caff5980 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88881ad4f0 con 0x55b1cafe62b0 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88881aa730 con 0x55b1cafe6c60 2024-04-25T00:26:22.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8890baa740 10 client.0 _ll_register_callbacks cb 0x7f8878032e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49552/0 (socket says 172.21.15.45:49552) 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8886ffd640 1 -- 172.21.15.45:0/2998019393 learned_addr learned my addr 172.21.15.45:0/2998019393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f8886ffd640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cb007f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.656+0000 7f88877fe640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1cb0084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 -- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 msgr2=0x55b1cb0084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1cb0084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 -- 172.21.15.45:0/2998019393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 msgr2=0x55b1cb007f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cb007f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 -- 172.21.15.45:0/2998019393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1cb064060 con 0x55b1caff5980 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8886ffd640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f887802aa30 tx=0x7f8878003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf456b0e6fdcc2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878035030 con 0x55b1caff5980 2024-04-25T00:26:22.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8878067670 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f887804d040 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8878067870 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 10 client.16887 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1cb06b410 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887804d040 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88780316d0 con 0x55b1caff5980 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 10 client.16887 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 10 client.16887 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:26:22.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 20 client.16887 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb06ba60 0x55b1cb06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 4 client.16887 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.657+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1caf52f00 con 0x55b1cb06ba60 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f88877fe640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb06ba60 0x55b1cb06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8866ffd640 10 client.16887 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f88877fe640 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb06ba60 0x55b1cb06de50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1507d79617966a95 server_cookie=e6f82dbd4328aeaf in_seq=0 out_seq=0 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8866ffd640 1 -- 172.21.15.45:0/2998019393 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1caf52f00 con 0x55b1cb06ba60 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8866ffd640 10 client.16887 handle_command_reply: tid=0 2024-04-25T00:26:22.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 client.16887 shutdown 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 msgr2=0x55b1cb063aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f887802aa30 tx=0x7f8878003d10 comp rx=0 tx=0).stop 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 shutdown_connections 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55b1cb06ba60 0x55b1cb06de50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1caff5980 0x55b1cb063aa0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1cafe6c60 0x55b1cb0084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 --2- 172.21.15.45:0/2998019393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1cafe62b0 0x55b1cb007f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8866ffd640 0 client.16887 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 >> 172.21.15.45:0/2998019393 conn(0x55b1cafe28a0 msgr2=0x55b1cafedc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 shutdown_connections 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.658+0000 7f8890baa740 1 -- 172.21.15.45:0/2998019393 wait complete. 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f8890baa740 20 client.16887 trim_cache size 0 max 16384 2024-04-25T00:26:22.683 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 msgr2=0x7f88700a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 0x7f88700a5240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f888000b760 tx=0x7f8880086040 comp rx=0 tx=0).stop 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 msgr2=0x7f888819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8880009fd0 tx=0x7f888000b260 comp rx=0 tx=0).stop 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 shutdown_connections 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88700a2d80 0x7f88700a5240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f888810bd00 0x7f888819c8e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88881032e0 0x7f8888198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 --2- 172.21.15.45:0/3390418508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88881029c0 0x7f8888198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 >> 172.21.15.45:0/3390418508 conn(0x7f8888077740 msgr2=0x7f88880fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 shutdown_connections 2024-04-25T00:26:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.659+0000 7f888de9e640 1 -- 172.21.15.45:0/3390418508 wait complete. 2024-04-25T00:26:22.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:26:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- 172.21.15.45:0/4272537760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 msgr2=0x7fe9dc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/4272537760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc100a90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009930 tx=0x7fe9cc02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- 172.21.15.45:0/4272537760 shutdown_connections 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/4272537760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc100fd0 0x7fe9dc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/4272537760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc100a90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/4272537760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- 172.21.15.45:0/4272537760 >> 172.21.15.45:0/4272537760 conn(0x7fe9dc077740 msgr2=0x7fe9dc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- 172.21.15.45:0/4272537760 shutdown_connections 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- 172.21.15.45:0/4272537760 wait complete. 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 Processor -- start 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- start start 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc100fd0 0x7fe9dc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1109f0 con 0x7fe9dc100610 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc110870 con 0x7fe9dc100fd0 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.792+0000 7fe9e46b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc110b70 con 0x7fe9dc1066b0 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49556/0 (socket says 172.21.15.45:49556) 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 -- 172.21.15.45:0/1530778441 learned_addr learned my addr 172.21.15.45:0/1530778441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc100fd0 0x7fe9dc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 -- 172.21.15.45:0/1530778441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 msgr2=0x7fe9dc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 -- 172.21.15.45:0/1530778441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc100fd0 msgr2=0x7fe9dc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc100fd0 0x7fe9dc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 -- 172.21.15.45:0/1530778441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc19cfb0 con 0x7fe9dc100610 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e244f640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800d770 tx=0x7fe9d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9d37fe640 1 -- 172.21.15.45:0/1530778441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8017070 con 0x7fe9dc100610 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9d37fe640 1 -- 172.21.15.45:0/1530778441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d8009c30 con 0x7fe9dc100610 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc19d2a0 con 0x7fe9dc100610 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9d37fe640 1 -- 172.21.15.45:0/1530778441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003dd0 con 0x7fe9dc100610 2024-04-25T00:26:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.793+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc1a4b30 con 0x7fe9dc100610 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.795+0000 7fe9d37fe640 1 -- 172.21.15.45:0/1530778441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe9d8003fb0 con 0x7fe9dc100610 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 Processor -- start 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 -- start start 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9d37fe640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 0x7fe9c80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9d37fe640 1 -- 172.21.15.45:0/1530778441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe9d80cf140 con 0x7fe9dc100610 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563760031670 con 0x5637600ccea0 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56375ffd9970 con 0x5637600cc580 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56376002d630 con 0x5637600d7570 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 0x7fe9c80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e1c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e1c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49560/0 (socket says 172.21.15.45:49560) 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/4086789845 learned_addr learned my addr 172.21.15.45:0/4086789845 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.796+0000 7fe9e244f640 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/4086789845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 msgr2=0x5637600d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/4086789845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 msgr2=0x5637600cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/4086789845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375fffd400 con 0x5637600ccea0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc02a940 tx=0x7fe9cc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fbdbbb464038fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 0x7fe9c80a5240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005fd0 tx=0x7fe9d4015b00 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9d0ff9640 1 -- 172.21.15.45:0/4086789845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc03f070 con 0x5637600ccea0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9d0ff9640 1 -- 172.21.15.45:0/4086789845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc004030 con 0x5637600ccea0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.797+0000 7fe9d0ff9640 1 -- 172.21.15.45:0/4086789845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc035070 con 0x5637600ccea0 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- 172.21.15.45:0/4086789845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 msgr2=0x5637600d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc02a940 tx=0x7fe9cc005e80 comp rx=0 tx=0).stop 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- 172.21.15.45:0/4086789845 shutdown_connections 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d7030 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/4086789845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- 172.21.15.45:0/4086789845 >> 172.21.15.45:0/4086789845 conn(0x5637600c88a0 msgr2=0x5637600c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- 172.21.15.45:0/4086789845 shutdown_connections 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- 172.21.15.45:0/4086789845 wait complete. 2024-04-25T00:26:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 -- start start 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 Processor -- start 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600ccea0 0x5637600d41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.798+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637600da300 con 0x5637600cc580 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56375ffed050 con 0x5637600ccea0 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5637600da480 con 0x5637600d7570 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 10 client.0 _ll_register_callbacks cb 0x56375fd44be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49576/0 (socket says 172.21.15.45:49576) 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 -- 172.21.15.45:0/3188862365 learned_addr learned my addr 172.21.15.45:0/3188862365 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600ccea0 0x5637600d41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 -- 172.21.15.45:0/3188862365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 msgr2=0x5637600d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 -- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600ccea0 msgr2=0x5637600d41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600ccea0 0x5637600d41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 -- 172.21.15.45:0/3188862365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375ffbedc0 con 0x5637600cc580 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80cc590 tx=0x7fe9d809ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7670220b9e049db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80b4060 con 0x5637600cc580 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe9d80cc7b0 con 0x5637600cc580 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9cc004030 con 0x5637600cc580 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9d807c050 con 0x5637600cc580 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 10 client.16899 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.799+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56375fd453f0 con 0x5637600cc580 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc004030 con 0x5637600cc580 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9d8097970 con 0x5637600cc580 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 10 client.16899 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 10 client.16899 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 20 client.16899 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5637600e75f0 0x5637600e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 4 client.16899 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376003ebb0 con 0x5637600e75f0 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.800+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5637600e75f0 0x5637600e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5637600e75f0 0x5637600e99e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2f5d7710851de6c server_cookie=7d2fc714a2b113ba in_seq=0 out_seq=0 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9c5ffb640 10 client.16899 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3188862365 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56376003ebb0 con 0x5637600e75f0 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9c5ffb640 10 client.16899 handle_command_reply: tid=0 2024-04-25T00:26:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 client.16899 shutdown 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 msgr2=0x5637600d6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80cc590 tx=0x7fe9d809ac60 comp rx=0 tx=0).stop 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 shutdown_connections 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5637600e75f0 0x5637600e99e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600ccea0 0x5637600d41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3188862365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600cc580 0x5637600d6ee0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9c5ffb640 0 client.16899 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.801+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 >> 172.21.15.45:0/3188862365 conn(0x5637600c88a0 msgr2=0x5637600c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.802+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 shutdown_connections 2024-04-25T00:26:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.802+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3188862365 wait complete. 2024-04-25T00:26:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.802+0000 7fe9e73c0740 20 client.16899 trim_cache size 0 max 16384 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 Processor -- start 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 -- start start 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1aeac0 con 0x5637600ccea0 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1b0600 con 0x5637600cc580 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1af190 con 0x5637600d7570 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49588/0 (socket says 172.21.15.45:49588) 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/353069136 learned_addr learned my addr 172.21.15.45:0/353069136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e244f640 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/353069136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 msgr2=0x5637600ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/353069136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 msgr2=0x5637600cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 -- 172.21.15.45:0/353069136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637600e9d50 con 0x5637600ccea0 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e244f640 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.815+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc031ba0 tx=0x7fe9cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697751ef0b6cb083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9c7fff640 1 -- 172.21.15.45:0/353069136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc035050 con 0x5637600ccea0 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9c7fff640 1 -- 172.21.15.45:0/353069136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc005600 con 0x5637600ccea0 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9c7fff640 1 -- 172.21.15.45:0/353069136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc03f040 con 0x5637600ccea0 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 -- 172.21.15.45:0/353069136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 msgr2=0x5637600d6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc031ba0 tx=0x7fe9cc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 -- 172.21.15.45:0/353069136 shutdown_connections 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600d7570 0x5637600ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600ccea0 0x5637600d6a10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/353069136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 -- 172.21.15.45:0/353069136 >> 172.21.15.45:0/353069136 conn(0x5637600c88a0 msgr2=0x5637600ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 -- 172.21.15.45:0/353069136 shutdown_connections 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.816+0000 7fe9e73c0740 1 -- 172.21.15.45:0/353069136 wait complete. 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 -- start start 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 Processor -- start 2024-04-25T00:26:22.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600ccea0 0x5637600db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1aff30 con 0x5637600d7570 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1af860 con 0x5637600cc580 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1acaa0 con 0x5637600ccea0 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e73c0740 10 client.0 _ll_register_callbacks cb 0x7fe9d800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49592/0 (socket says 172.21.15.45:49592) 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 -- 172.21.15.45:0/3620501740 learned_addr learned my addr 172.21.15.45:0/3620501740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e1c4e640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600ccea0 0x5637600db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 -- 172.21.15.45:0/3620501740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600ccea0 msgr2=0x5637600db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600ccea0 0x5637600db770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 -- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 msgr2=0x5637600db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 -- 172.21.15.45:0/3620501740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637600d4aa0 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9e2c50640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009cb0 tx=0x7fe9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ee704910622739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.817+0000 7fe9c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4025030 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fe9d4013b10 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9cc005600 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9d4031050 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 10 client.16905 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56375fe0f7d0 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc005600 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9d400f040 con 0x5637600d7570 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 10 client.16905 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 10 client.16905 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:26:22.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 20 client.16905 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56376014e2c0 0x5637601506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 4 client.16905 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563760038f00 con 0x56376014e2c0 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.818+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56376014e2c0 0x5637601506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e244f640 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56376014e2c0 0x5637601506e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dec91be14f69e8b2 server_cookie=b7a785934fc9e0b6 in_seq=0 out_seq=0 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9c5ffb640 10 client.16905 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9c5ffb640 1 -- 172.21.15.45:0/3620501740 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563760038f00 con 0x56376014e2c0 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9c5ffb640 10 client.16905 handle_command_reply: tid=0 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 client.16905 shutdown 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 msgr2=0x5637600dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009cb0 tx=0x7fe9d4004a40 comp rx=0 tx=0).stop 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 shutdown_connections 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x56376014e2c0 0x5637601506e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637600d7570 0x5637600dbcb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637600ccea0 0x5637600db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 --2- 172.21.15.45:0/3620501740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637600cc580 0x5637600db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9c5ffb640 0 client.16905 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 >> 172.21.15.45:0/3620501740 conn(0x5637600c88a0 msgr2=0x5637600cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 shutdown_connections 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 1 -- 172.21.15.45:0/3620501740 wait complete. 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.819+0000 7fe9e73c0740 20 client.16905 trim_cache size 0 max 16384 2024-04-25T00:26:22.827 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 msgr2=0x7fe9c80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 0x7fe9c80a5240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005fd0 tx=0x7fe9d4015b00 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 msgr2=0x7fe9dc197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800d770 tx=0x7fe9d800dc30 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 shutdown_connections 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9c80a2d80 0x7fe9c80a5240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9dc1066b0 0x7fe9dc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9dc100fd0 0x7fe9dc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 --2- 172.21.15.45:0/1530778441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9dc100610 0x7fe9dc197fc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 >> 172.21.15.45:0/1530778441 conn(0x7fe9dc077740 msgr2=0x7fe9dc0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 shutdown_connections 2024-04-25T00:26:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.820+0000 7fe9e46b3640 1 -- 172.21.15.45:0/1530778441 wait complete. 2024-04-25T00:26:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:22 smithi045 ceph-mon[29194]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:22.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:22 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2076467575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:22.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 -- 172.21.15.45:0/1991295898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d0105160 msgr2=0x7f29d010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 --2- 172.21.15.45:0/1991295898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d0105160 0x7f29d010cce0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009ca0 tx=0x7f29c802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 -- 172.21.15.45:0/1991295898 shutdown_connections 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 --2- 172.21.15.45:0/1991295898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d0105160 0x7f29d010cce0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 --2- 172.21.15.45:0/1991295898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d0100c90 0x7f29d0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 --2- 172.21.15.45:0/1991295898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d01002e0 0x7f29d01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 -- 172.21.15.45:0/1991295898 >> 172.21.15.45:0/1991295898 conn(0x7f29d00758d0 msgr2=0x7f29d0075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 -- 172.21.15.45:0/1991295898 shutdown_connections 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.935+0000 7f29d5716640 1 -- 172.21.15.45:0/1991295898 wait complete. 2024-04-25T00:26:22.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 Processor -- start 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 -- start start 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 0x7f29d0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 0x7f29d019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010f990 con 0x7f29d01002e0 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010f810 con 0x7f29d019c850 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29d5716640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010fb10 con 0x7f29d0105160 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49604/0 (socket says 172.21.15.45:49604) 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29ceffd640 1 -- 172.21.15.45:0/129506339 learned_addr learned my addr 172.21.15.45:0/129506339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.936+0000 7f29c7fff640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 0x7f29d0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 0x7f29d019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 -- 172.21.15.45:0/129506339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 msgr2=0x7f29d0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 0x7f29d0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 -- 172.21.15.45:0/129506339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 msgr2=0x7f29d019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 0x7f29d019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 -- 172.21.15.45:0/129506339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d019d340 con 0x7f29d01002e0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 0x7f29d019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29c7fff640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 0x7f29d0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ceffd640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f29c000d770 tx=0x7f29c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ccff9640 1 -- 172.21.15.45:0/129506339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0017070 con 0x7f29d01002e0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d019d630 con 0x7f29d01002e0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ccff9640 1 -- 172.21.15.45:0/129506339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c0009c30 con 0x7f29d01002e0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29ccff9640 1 -- 172.21.15.45:0/129506339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0003dd0 con 0x7f29d01002e0 2024-04-25T00:26:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.937+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a4e20 con 0x7f29d01002e0 2024-04-25T00:26:22.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.939+0000 7f29ccff9640 1 -- 172.21.15.45:0/129506339 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f29c0003fb0 con 0x7f29d01002e0 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.939+0000 7f29d841c740 1 Processor -- start 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 -- start start 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5580 0x5606793b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56067931a670 con 0x5606793c0570 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5606792c2970 con 0x5606793b5580 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29d841c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560679316630 con 0x5606793b5ea0 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29ccff9640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 0x7f29bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 0x7f29bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49614/0 (socket says 172.21.15.45:49614) 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 -- 172.21.15.45:0/2022019168 learned_addr learned my addr 172.21.15.45:0/2022019168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5580 0x5606793b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29ceffd640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29ccff9640 1 -- 172.21.15.45:0/129506339 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f29c00cf250 con 0x7f29d01002e0 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 -- 172.21.15.45:0/2022019168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 msgr2=0x5606793c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 -- 172.21.15.45:0/2022019168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5580 msgr2=0x5606793b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5580 0x5606793b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29c7fff640 1 -- 172.21.15.45:0/2022019168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606792e6400 con 0x5606793c0570 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.940+0000 7f29ceffd640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.941+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 0x7f29bc0a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009a60 tx=0x7f29c802f500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.941+0000 7f29c7fff640 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bd20 tx=0x7f29b800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa817515886211d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.941+0000 7f29c57fa640 1 -- 172.21.15.45:0/2022019168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b800d860 con 0x5606793c0570 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.941+0000 7f29c57fa640 1 -- 172.21.15.45:0/2022019168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b80173e0 con 0x5606793c0570 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.941+0000 7f29c57fa640 1 -- 172.21.15.45:0/2022019168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b800c630 con 0x5606793c0570 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- 172.21.15.45:0/2022019168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 msgr2=0x5606793c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bd20 tx=0x7f29b800f830 comp rx=0 tx=0).stop 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- 172.21.15.45:0/2022019168 shutdown_connections 2024-04-25T00:26:22.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793c2960 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- 172.21.15.45:0/2022019168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5580 0x5606793b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- 172.21.15.45:0/2022019168 >> 172.21.15.45:0/2022019168 conn(0x5606793b18a0 msgr2=0x5606793b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- 172.21.15.45:0/2022019168 shutdown_connections 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- 172.21.15.45:0/2022019168 wait complete. 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- start start 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 Processor -- start 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606793c3360 con 0x5606793c0570 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5606792d6050 con 0x5606793b5ea0 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5606793c34e0 con 0x5606793b5580 2024-04-25T00:26:22.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29d841c740 10 client.0 _ll_register_callbacks cb 0x56067902dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29cf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49616/0 (socket says 172.21.15.45:49616) 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.942+0000 7f29cf7fe640 1 -- 172.21.15.45:0/578474691 learned_addr learned my addr 172.21.15.45:0/578474691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29c7fff640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29ceffd640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 -- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 msgr2=0x5606793bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 -- 172.21.15.45:0/578474691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 msgr2=0x5606793bd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 -- 172.21.15.45:0/578474691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606792a7dc0 con 0x5606793c0570 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f29c8035000 tx=0x7f29c8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243e2ad249fba0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8003d00 con 0x5606793c0570 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f29c8031bb0 con 0x5606793c0570 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b80173e0 con 0x5606793c0570 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29c8031db0 con 0x5606793c0570 2024-04-25T00:26:22.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29d841c740 10 client.16917 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56067902e3f0 con 0x5606793c0570 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.943+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b80173e0 con 0x5606793c0570 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29c802f8f0 con 0x5606793c0570 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 10 client.16917 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 20 client.16917 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793d04f0 0x5606793d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 4 client.16917 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560679327bb0 con 0x5606793d04f0 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29ceffd640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793d04f0 0x5606793d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29ceffd640 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793d04f0 0x5606793d2970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1cfa0bec54949db server_cookie=81a376ce9f0e53b6 in_seq=0 out_seq=0 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.944+0000 7f29b67fc640 10 client.16917 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29b67fc640 1 -- 172.21.15.45:0/578474691 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560679327bb0 con 0x5606793d04f0 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29b67fc640 10 client.16917 handle_command_reply: tid=0 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 client.16917 shutdown 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 msgr2=0x5606793bd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f29c8035000 tx=0x7f29c8047000 comp rx=0 tx=0).stop 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 shutdown_connections 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793d04f0 0x5606793d2970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793c0570 0x5606793bd630 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29b67fc640 0 client.16917 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 --2- 172.21.15.45:0/578474691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 >> 172.21.15.45:0/578474691 conn(0x5606793b18a0 msgr2=0x5606793b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 shutdown_connections 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 1 -- 172.21.15.45:0/578474691 wait complete. 2024-04-25T00:26:22.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.945+0000 7f29d841c740 20 client.16917 trim_cache size 0 max 16384 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 Processor -- start 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 -- start start 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01ae9e0 con 0x5606793b18a0 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01b0520 con 0x5606793b5ea0 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29d841c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01af0b0 con 0x5606793b5580 2024-04-25T00:26:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49630/0 (socket says 172.21.15.45:49630) 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 -- 172.21.15.45:0/3262289582 learned_addr learned my addr 172.21.15.45:0/3262289582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 -- 172.21.15.45:0/3262289582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 msgr2=0x5606793b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 -- 172.21.15.45:0/3262289582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 msgr2=0x5606793bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 -- 172.21.15.45:0/3262289582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606790f87d0 con 0x5606793b18a0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.959+0000 7f29ceffd640 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f29c0097d80 tx=0x7f29c0016880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdbc8c8cf49518b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29c4ff9640 1 -- 172.21.15.45:0/3262289582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0020040 con 0x5606793b18a0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29c4ff9640 1 -- 172.21.15.45:0/3262289582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c009bab0 con 0x5606793b18a0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29c4ff9640 1 -- 172.21.15.45:0/3262289582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c00042a0 con 0x5606793b18a0 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 -- 172.21.15.45:0/3262289582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 msgr2=0x5606793cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f29c0097d80 tx=0x7f29c0016880 comp rx=0 tx=0).stop 2024-04-25T00:26:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 -- 172.21.15.45:0/3262289582 shutdown_connections 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b5ea0 0x5606793bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5580 0x5606793b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 --2- 172.21.15.45:0/3262289582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b18a0 0x5606793cc250 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 -- 172.21.15.45:0/3262289582 >> 172.21.15.45:0/3262289582 conn(0x5606793bd790 msgr2=0x5606793b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 -- 172.21.15.45:0/3262289582 shutdown_connections 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.960+0000 7f29d841c740 1 -- 172.21.15.45:0/3262289582 wait complete. 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 -- start start 2024-04-25T00:26:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 Processor -- start 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b18a0 0x5606793c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01afe50 con 0x5606793b5580 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01af780 con 0x5606793b18a0 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01ac9c0 con 0x5606793b5ea0 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29d841c740 10 client.0 _ll_register_callbacks cb 0x7f29c8032ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49646/0 (socket says 172.21.15.45:49646) 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 -- 172.21.15.45:0/1379411594 learned_addr learned my addr 172.21.15.45:0/1379411594 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29cf7fe640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29ceffd640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b18a0 0x5606793c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 -- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 msgr2=0x5606793c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 -- 172.21.15.45:0/1379411594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b18a0 msgr2=0x5606793c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b18a0 0x5606793c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.961+0000 7f29c7fff640 1 -- 172.21.15.45:0/1379411594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560679439f50 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29c7fff640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bd20 tx=0x7f29b80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8fba97d80b8e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801d040 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f29b800da70 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c009bab0 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29b8031050 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 10 client.16923 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56067943b280 con 0x5606793b5580 2024-04-25T00:26:22.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c009bab0 con 0x5606793b5580 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29b8016070 con 0x5606793b5580 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 10 client.16923 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 20 client.16923 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793c0570 0x56067943d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 4 client.16923 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.962+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560679321f00 con 0x5606793c0570 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29ceffd640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793c0570 0x56067943d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29ceffd640 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793c0570 0x56067943d8c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69d67f165b0af4d0 server_cookie=49776fe0dac7cbff in_seq=0 out_seq=0 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29b67fc640 10 client.16923 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29b67fc640 1 -- 172.21.15.45:0/1379411594 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560679321f00 con 0x5606793c0570 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29b67fc640 10 client.16923 handle_command_reply: tid=0 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 client.16923 shutdown 2024-04-25T00:26:22.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 msgr2=0x5606793c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29b800bd20 tx=0x7f29b80045f0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 shutdown_connections 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5606793c0570 0x56067943d8c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606793b5ea0 0x5606793c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606793b5580 0x5606793c4710 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29b67fc640 0 client.16923 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 --2- 172.21.15.45:0/1379411594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606793b18a0 0x5606793c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.963+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 >> 172.21.15.45:0/1379411594 conn(0x5606793bd790 msgr2=0x5606793b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 shutdown_connections 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d841c740 1 -- 172.21.15.45:0/1379411594 wait complete. 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d841c740 20 client.16923 trim_cache size 0 max 16384 2024-04-25T00:26:22.970 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 msgr2=0x7f29bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 0x7f29bc0a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009a60 tx=0x7f29c802f500 comp rx=0 tx=0).stop 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 msgr2=0x7f29d0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f29c000d770 tx=0x7f29c000dc30 comp rx=0 tx=0).stop 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 shutdown_connections 2024-04-25T00:26:22.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f29bc0a2d00 0x7f29bc0a51c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29d019c850 0x7f29d019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d0105160 0x7f29d0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 --2- 172.21.15.45:0/129506339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29d01002e0 0x7f29d0197fe0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 >> 172.21.15.45:0/129506339 conn(0x7f29d00758d0 msgr2=0x7f29d0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 shutdown_connections 2024-04-25T00:26:22.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:22.964+0000 7f29d5716640 1 -- 172.21.15.45:0/129506339 wait complete. 2024-04-25T00:26:22.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:26:23.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:22 smithi122 ceph-mon[32959]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:23.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:22 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2076467575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.079+0000 7fa98dc9e640 1 -- 172.21.15.45:0/1100521298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 msgr2=0x7fa988106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.079+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/1100521298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa988106a90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa978009cd0 tx=0x7fa97802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 -- 172.21.15.45:0/1100521298 shutdown_connections 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/1100521298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa98810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/1100521298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 0x7fa988100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/1100521298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa988106a90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 -- 172.21.15.45:0/1100521298 >> 172.21.15.45:0/1100521298 conn(0x7fa988077740 msgr2=0x7fa988077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 -- 172.21.15.45:0/1100521298 shutdown_connections 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 -- 172.21.15.45:0/1100521298 wait complete. 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 Processor -- start 2024-04-25T00:26:23.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.080+0000 7fa98dc9e640 1 -- start start 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 0x7fa988198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa9881993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa988110a80 con 0x7fa9881066b0 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa988110900 con 0x7fa988100fd0 2024-04-25T00:26:23.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa988110c00 con 0x7fa988100610 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49670/0 (socket says 172.21.15.45:49670) 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 -- 172.21.15.45:0/2962049249 learned_addr learned my addr 172.21.15.45:0/2962049249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa9877fe640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 0x7fa988198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa9881993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 -- 172.21.15.45:0/2962049249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 msgr2=0x7fa988198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 0x7fa988198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 -- 172.21.15.45:0/2962049249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 msgr2=0x7fa9881993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa9881993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 -- 172.21.15.45:0/2962049249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9881a4420 con 0x7fa9881066b0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa9881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa987fff640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9b0 tx=0x7fa98000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa984ff9640 1 -- 172.21.15.45:0/2962049249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000cd80 con 0x7fa9881066b0 2024-04-25T00:26:23.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa984ff9640 1 -- 172.21.15.45:0/2962049249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9800035f0 con 0x7fa9881066b0 2024-04-25T00:26:23.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa984ff9640 1 -- 172.21.15.45:0/2962049249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980010670 con 0x7fa9881066b0 2024-04-25T00:26:23.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881a45f0 con 0x7fa9881066b0 2024-04-25T00:26:23.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.081+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881a4af0 con 0x7fa9881066b0 2024-04-25T00:26:23.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.084+0000 7fa984ff9640 1 -- 172.21.15.45:0/2962049249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa9800037a0 con 0x7fa9881066b0 2024-04-25T00:26:23.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.084+0000 7fa9909aa740 1 Processor -- start 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.084+0000 7fa9909aa740 1 -- start start 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13f9e2b0 0x559e13f9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 0x559e13fafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e13f03670 con 0x559e13f9ec60 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559e13eab970 con 0x559e13fad980 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9909aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559e13eff630 con 0x559e13f9e2b0 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49672/0 (socket says 172.21.15.45:49672) 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 -- 172.21.15.45:0/1320266936 learned_addr learned my addr 172.21.15.45:0/1320266936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13f9e2b0 0x559e13f9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa984ff9640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 0x7fa9700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 0x559e13fafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 -- 172.21.15.45:0/1320266936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13f9e2b0 msgr2=0x559e13f9e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13f9e2b0 0x559e13f9e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 -- 172.21.15.45:0/1320266936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 msgr2=0x559e13fafd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 0x559e13fafd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 -- 172.21.15.45:0/1320266936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e13ecf400 con 0x559e13f9ec60 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 0x559e13fafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa987fff640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 0x7fa9700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa984ff9640 1 -- 172.21.15.45:0/2962049249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa9800cfb30 con 0x7fa9881066b0 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.085+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa978009ca0 tx=0x7fa978009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87da6032845bfcfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa96dffb640 1 -- 172.21.15.45:0/1320266936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978031570 con 0x559e13f9ec60 2024-04-25T00:26:23.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa96dffb640 1 -- 172.21.15.45:0/1320266936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa978036040 con 0x559e13f9ec60 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa96dffb640 1 -- 172.21.15.45:0/1320266936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978004280 con 0x559e13f9ec60 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 -- 172.21.15.45:0/1320266936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 msgr2=0x559e13fad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa987fff640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 0x7fa9700a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa98009b990 tx=0x7fa980005bc0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa978009ca0 tx=0x7fa978009a20 comp rx=0 tx=0).stop 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 -- 172.21.15.45:0/1320266936 shutdown_connections 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13fad980 0x559e13fafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13fad440 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1320266936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13f9e2b0 0x559e13f9e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 -- 172.21.15.45:0/1320266936 >> 172.21.15.45:0/1320266936 conn(0x559e13f9a8a0 msgr2=0x559e13f9ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 -- 172.21.15.45:0/1320266936 shutdown_connections 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.086+0000 7fa9909aa740 1 -- 172.21.15.45:0/1320266936 wait complete. 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 -- start start 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 Processor -- start 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9e2b0 0x559e13fad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13faab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e13fb0680 con 0x559e13f9ec60 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559e13ebf050 con 0x559e13f9e2b0 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559e13fb0800 con 0x559e13fad980 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9909aa740 10 client.0 _ll_register_callbacks cb 0x559e13c16be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49682/0 (socket says 172.21.15.45:49682) 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa9877fe640 1 -- 172.21.15.45:0/2680211569 learned_addr learned my addr 172.21.15.45:0/2680211569 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.087+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9e2b0 0x559e13fad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 -- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 msgr2=0x559e13faab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13faab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13faab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 -- 172.21.15.45:0/2680211569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9e2b0 msgr2=0x559e13fad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9e2b0 0x559e13fad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 -- 172.21.15.45:0/2680211569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e13e90dc0 con 0x559e13f9ec60 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13faab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:23.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9877fe640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa97802aed0 tx=0x7fa97802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8952b7dd0632903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978053060 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa978031ad0 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa978036040 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa97805a050 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9909aa740 10 client.16935 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e13c173f0 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa978036040 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.088+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa978030070 con 0x559e13f9ec60 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 10 client.16935 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 20 client.16935 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e13fbd740 0x559e13fbfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 4 client.16935 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e13f10bb0 con 0x559e13fbd740 2024-04-25T00:26:23.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e13fbd740 0x559e13fbfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa966ffd640 10 client.16935 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.089+0000 7fa986ffd640 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e13fbd740 0x559e13fbfb30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b88955879e7a1bcb server_cookie=3ebd109262228580 in_seq=0 out_seq=0 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa966ffd640 1 -- 172.21.15.45:0/2680211569 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e13f10bb0 con 0x559e13fbd740 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa966ffd640 10 client.16935 handle_command_reply: tid=0 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 client.16935 shutdown 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 msgr2=0x559e13faa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa97802aed0 tx=0x7fa97802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 shutdown_connections 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e13fbd740 0x559e13fbfb30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13faab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9ec60 0x559e13faa600 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 --2- 172.21.15.45:0/2680211569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9e2b0 0x559e13fad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 >> 172.21.15.45:0/2680211569 conn(0x559e13f9a8a0 msgr2=0x559e13fa0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa966ffd640 0 client.16935 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 shutdown_connections 2024-04-25T00:26:23.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.090+0000 7fa9909aa740 1 -- 172.21.15.45:0/2680211569 wait complete. 2024-04-25T00:26:23.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.091+0000 7fa9909aa740 20 client.16935 trim_cache size 0 max 16384 2024-04-25T00:26:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 Processor -- start 2024-04-25T00:26:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 -- start start 2024-04-25T00:26:23.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13face40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fc5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881ac8c0 con 0x559e13f9e2b0 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881ae400 con 0x559e13f9ec60 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9909aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881acf90 con 0x559e13fad980 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49684/0 (socket says 172.21.15.45:49684) 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 -- 172.21.15.45:0/3313871082 learned_addr learned my addr 172.21.15.45:0/3313871082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fc5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa9877fe640 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13face40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 -- 172.21.15.45:0/3313871082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 msgr2=0x559e13fc5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fc5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 -- 172.21.15.45:0/3313871082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 msgr2=0x559e13face40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13face40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.104+0000 7fa986ffd640 1 -- 172.21.15.45:0/3313871082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e13ce17d0 con 0x559e13f9e2b0 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa986ffd640 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d3f9ee5c2cab21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa96d7fa640 1 -- 172.21.15.45:0/3313871082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011b90 con 0x559e13f9e2b0 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa96d7fa640 1 -- 172.21.15.45:0/3313871082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97c011d30 con 0x559e13f9e2b0 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa96d7fa640 1 -- 172.21.15.45:0/3313871082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c01f0f0 con 0x559e13f9e2b0 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 -- 172.21.15.45:0/3313871082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 msgr2=0x559e13fb9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 -- 172.21.15.45:0/3313871082 shutdown_connections 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fc5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13face40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 --2- 172.21.15.45:0/3313871082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb9580 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 -- 172.21.15.45:0/3313871082 >> 172.21.15.45:0/3313871082 conn(0x559e13f9a8a0 msgr2=0x559e13fa5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 -- 172.21.15.45:0/3313871082 shutdown_connections 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.105+0000 7fa9909aa740 1 -- 172.21.15.45:0/3313871082 wait complete. 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 -- start start 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 Processor -- start 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13fa6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fa74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881add30 con 0x559e13f9e2b0 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881ad660 con 0x559e13f9ec60 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881aa8a0 con 0x559e13fad980 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9909aa740 10 client.0 _ll_register_callbacks cb 0x7fa978009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa986ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa986ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49698/0 (socket says 172.21.15.45:49698) 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa986ffd640 1 -- 172.21.15.45:0/1085998661 learned_addr learned my addr 172.21.15.45:0/1085998661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13fa6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.106+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fa74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 -- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 msgr2=0x559e13fa74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fa74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 -- 172.21.15.45:0/1085998661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 msgr2=0x559e13fa6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13fa6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 -- 172.21.15.45:0/1085998661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e13fa7c20 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13fa6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa986ffd640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa97c004980 tx=0x7fa97c004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6538c38f4eb8248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0052a0 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa97c005440 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa97c011d30 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa97c005640 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 10 client.16941 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e14023850 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa97c011d30 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa97c02c040 con 0x559e13f9e2b0 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 10 client.16941 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:23.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 20 client.16941 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e14023ef0 0x559e14026310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 4 client.16941 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.107+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e13f0af00 con 0x559e14023ef0 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e14023ef0 0x559e14026310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa9877fe640 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e14023ef0 0x559e14026310 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2186c46127730cba server_cookie=e3e7a7c16bd2c1b in_seq=0 out_seq=0 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa966ffd640 10 client.16941 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa966ffd640 1 -- 172.21.15.45:0/1085998661 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e13f0af00 con 0x559e14023ef0 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa966ffd640 10 client.16941 handle_command_reply: tid=0 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.108+0000 7fa9909aa740 1 client.16941 shutdown 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 msgr2=0x559e13fb5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa97c004980 tx=0x7fa97c004b70 comp rx=0 tx=0).stop 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 shutdown_connections 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x559e14023ef0 0x559e14026310 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559e13fad980 0x559e13fa74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e13f9ec60 0x559e13fa6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa966ffd640 0 client.16941 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 --2- 172.21.15.45:0/1085998661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e13f9e2b0 0x559e13fb5d80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 >> 172.21.15.45:0/1085998661 conn(0x559e13f9a8a0 msgr2=0x559e13f9fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 shutdown_connections 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 1 -- 172.21.15.45:0/1085998661 wait complete. 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa9909aa740 20 client.16941 trim_cache size 0 max 16384 2024-04-25T00:26:23.116 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 msgr2=0x7fa9700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 0x7fa9700a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa98009b990 tx=0x7fa980005bc0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 msgr2=0x7fa9881a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e9b0 tx=0x7fa98000ee70 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 shutdown_connections 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa9700a2d00 0x7fa9700a51c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9881066b0 0x7fa9881a3ee0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa988100fd0 0x7fa9881993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 --2- 172.21.15.45:0/2962049249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa988100610 0x7fa988198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 >> 172.21.15.45:0/2962049249 conn(0x7fa988077740 msgr2=0x7fa9880ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 shutdown_connections 2024-04-25T00:26:23.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.109+0000 7fa98dc9e640 1 -- 172.21.15.45:0/2962049249 wait complete. 2024-04-25T00:26:23.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:22 smithi071 ceph-mon[32667]: pgmap v135: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:23.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:22 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2076467575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:23.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:26:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 -- 172.21.15.45:0/3307817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 msgr2=0x7f14d810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 --2- 172.21.15.45:0/3307817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d810dc10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009900 tx=0x7f14c002a410 comp rx=0 tx=0).stop 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 -- 172.21.15.45:0/3307817983 shutdown_connections 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 --2- 172.21.15.45:0/3307817983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d810e150 0x7f14d8110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 --2- 172.21.15.45:0/3307817983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d810dc10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 --2- 172.21.15.45:0/3307817983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d8069dc0 0x7f14d806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 -- 172.21.15.45:0/3307817983 >> 172.21.15.45:0/3307817983 conn(0x7f14d80776a0 msgr2=0x7f14d8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 -- 172.21.15.45:0/3307817983 shutdown_connections 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.224+0000 7f14dd841640 1 -- 172.21.15.45:0/3307817983 wait complete. 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 Processor -- start 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 -- start start 2024-04-25T00:26:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d8069dc0 0x7f14d819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d810e150 0x7f14d819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113d30 con 0x7f14d806a6e0 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113bb0 con 0x7f14d8069dc0 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14dd841640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8113eb0 con 0x7f14d810e150 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49712/0 (socket says 172.21.15.45:49712) 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14ce7fc640 1 -- 172.21.15.45:0/3832868887 learned_addr learned my addr 172.21.15.45:0/3832868887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.225+0000 7f14d77fe640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d810e150 0x7f14d819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d8069dc0 0x7f14d819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 -- 172.21.15.45:0/3832868887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d810e150 msgr2=0x7f14d819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d810e150 0x7f14d819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 -- 172.21.15.45:0/3832868887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d8069dc0 msgr2=0x7f14d819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d8069dc0 0x7f14d819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 -- 172.21.15.45:0/3832868887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819f2e0 con 0x7f14d806a6e0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f14c0006010 tx=0x7f14c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14dc83f640 1 -- 172.21.15.45:0/3832868887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c003f030 con 0x7f14d806a6e0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14dc83f640 1 -- 172.21.15.45:0/3832868887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c0003ce0 con 0x7f14d806a6e0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819f570 con 0x7f14d806a6e0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14dc83f640 1 -- 172.21.15.45:0/3832868887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0035070 con 0x7f14d806a6e0 2024-04-25T00:26:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.226+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a6e60 con 0x7f14d806a6e0 2024-04-25T00:26:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14dc83f640 1 -- 172.21.15.45:0/3832868887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14c0042030 con 0x7f14d806a6e0 2024-04-25T00:26:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 Processor -- start 2024-04-25T00:26:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 -- start start 2024-04-25T00:26:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 0x5583e1752030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1754960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583e16ac670 con 0x5583e1747580 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5583e1654970 con 0x5583e1747ea0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14e0620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583e16a8630 con 0x5583e1752570 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49728/0 (socket says 172.21.15.45:49728) 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14d77fe640 1 -- 172.21.15.45:0/709620437 learned_addr learned my addr 172.21.15.45:0/709620437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.229+0000 7f14dc83f640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 0x7f14c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1754960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 0x5583e1752030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 0x7f14c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 -- 172.21.15.45:0/709620437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 msgr2=0x5583e1754960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1754960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 -- 172.21.15.45:0/709620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 msgr2=0x5583e1752030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14dc83f640 1 -- 172.21.15.45:0/3832868887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f14c00ef970 con 0x7f14d806a6e0 2024-04-25T00:26:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 0x5583e1752030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 -- 172.21.15.45:0/709620437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e1678400 con 0x5583e1747580 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 0x5583e1752030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14d0014a20 tx=0x7f14d0014ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca66efbc2e71e072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14d77fe640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 0x7f14c40a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14d0012820 tx=0x7f14d001e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14cdffb640 1 -- 172.21.15.45:0/709620437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0012c10 con 0x5583e1747580 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14cdffb640 1 -- 172.21.15.45:0/709620437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0022020 con 0x5583e1747580 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.230+0000 7f14cdffb640 1 -- 172.21.15.45:0/709620437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d001ed90 con 0x5583e1747580 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- 172.21.15.45:0/709620437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 msgr2=0x5583e1747960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f14d0014a20 tx=0x7f14d0014ee0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- 172.21.15.45:0/709620437 shutdown_connections 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1754960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747ea0 0x5583e1752030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 --2- 172.21.15.45:0/709620437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e1747960 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- 172.21.15.45:0/709620437 >> 172.21.15.45:0/709620437 conn(0x5583e17438a0 msgr2=0x5583e1743c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- 172.21.15.45:0/709620437 shutdown_connections 2024-04-25T00:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- 172.21.15.45:0/709620437 wait complete. 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.231+0000 7f14e0620740 1 -- start start 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 Processor -- start 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e1751f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e174f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583e1755300 con 0x5583e1752570 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5583e1668050 con 0x5583e1747580 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583e1755480 con 0x5583e1747ea0 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14e0620740 10 client.0 _ll_register_callbacks cb 0x5583e13bfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49740/0 (socket says 172.21.15.45:49740) 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 -- 172.21.15.45:0/2485795360 learned_addr learned my addr 172.21.15.45:0/2485795360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e1751f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e174f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 -- 172.21.15.45:0/2485795360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 msgr2=0x5583e174f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e174f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 -- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 msgr2=0x5583e1751f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e1751f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 -- 172.21.15.45:0/2485795360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e1639dc0 con 0x5583e1752570 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e174f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14d77fe640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009bb0 tx=0x7f14d000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6d03f9fe8fdbfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.232+0000 7f14b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0012e10 con 0x5583e1752570 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f14d001eb10 con 0x5583e1752570 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d0022020 con 0x5583e1752570 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d001ed10 con 0x5583e1752570 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 10 client.16953 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583e13c03f0 con 0x5583e1752570 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0022020 con 0x5583e1752570 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14d000f040 con 0x5583e1752570 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 10 client.16953 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 20 client.16953 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e1762520 0x5583e1764910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 4 client.16953 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e16b9bb0 con 0x5583e1762520 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.233+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e1762520 0x5583e1764910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.234+0000 7f14b6ffd640 10 client.16953 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.234+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e1762520 0x5583e1764910 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca43631273307acf server_cookie=bcd4118c9a4b08a4 in_seq=0 out_seq=0 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.234+0000 7f14b6ffd640 1 -- 172.21.15.45:0/2485795360 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583e16b9bb0 con 0x5583e1762520 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.234+0000 7f14b6ffd640 10 client.16953 handle_command_reply: tid=0 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 client.16953 shutdown 2024-04-25T00:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 msgr2=0x5583e174f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009bb0 tx=0x7f14d000ebf0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 shutdown_connections 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e1762520 0x5583e1764910 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1752570 0x5583e174f770 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e174f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14b6ffd640 0 client.16953 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 --2- 172.21.15.45:0/2485795360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e1751f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 >> 172.21.15.45:0/2485795360 conn(0x5583e17438a0 msgr2=0x5583e1742b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 shutdown_connections 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 1 -- 172.21.15.45:0/2485795360 wait complete. 2024-04-25T00:26:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.235+0000 7f14e0620740 20 client.16953 trim_cache size 0 max 16384 2024-04-25T00:26:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 Processor -- start 2024-04-25T00:26:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 -- start start 2024-04-25T00:26:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e175e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1769f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b0d20 con 0x5583e1747ea0 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b2860 con 0x5583e1747580 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.248+0000 7f14e0620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b13f0 con 0x5583e1752570 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49748/0 (socket says 172.21.15.45:49748) 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 -- 172.21.15.45:0/842904441 learned_addr learned my addr 172.21.15.45:0/842904441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e175e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 -- 172.21.15.45:0/842904441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 msgr2=0x5583e1769f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1769f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 -- 172.21.15.45:0/842904441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 msgr2=0x5583e175e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e175e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 -- 172.21.15.45:0/842904441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e148a7d0 con 0x5583e1747ea0 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e175e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f14c00ecdc0 tx=0x7f14c008bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a36bc46991289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14cd7fa640 1 -- 172.21.15.45:0/842904441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0049040 con 0x5583e1747ea0 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14cd7fa640 1 -- 172.21.15.45:0/842904441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c003b580 con 0x5583e1747ea0 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.249+0000 7f14cd7fa640 1 -- 172.21.15.45:0/842904441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00bbbf0 con 0x5583e1747ea0 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- 172.21.15.45:0/842904441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 msgr2=0x5583e1751a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f14c00ecdc0 tx=0x7f14c008bb10 comp rx=0 tx=0).stop 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- 172.21.15.45:0/842904441 shutdown_connections 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1752570 0x5583e1769f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747ea0 0x5583e1751a30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 --2- 172.21.15.45:0/842904441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1747580 0x5583e175e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- 172.21.15.45:0/842904441 >> 172.21.15.45:0/842904441 conn(0x5583e17438a0 msgr2=0x5583e1749560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- 172.21.15.45:0/842904441 shutdown_connections 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- 172.21.15.45:0/842904441 wait complete. 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 -- start start 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 Processor -- start 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.250+0000 7f14e0620740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e1755cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1752570 0x5583e1756210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b2190 con 0x5583e1747580 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81b1ac0 con 0x5583e1752570 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81aed00 con 0x5583e1747ea0 2024-04-25T00:26:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 10 client.0 _ll_register_callbacks cb 0x7f14d0014820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:23.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49762/0 (socket says 172.21.15.45:49762) 2024-04-25T00:26:23.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 -- 172.21.15.45:0/4221575022 learned_addr learned my addr 172.21.15.45:0/4221575022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:23.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d77fe640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1752570 0x5583e1756210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e1755cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 -- 172.21.15.45:0/4221575022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 msgr2=0x5583e1755cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e1755cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 -- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1752570 msgr2=0x5583e1756210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1752570 0x5583e1756210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 -- 172.21.15.45:0/4221575022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e1756900 con 0x5583e1747580 2024-04-25T00:26:23.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e1755cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14d6ffd640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x5583e1764ee0 tx=0x7f14c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ddffe740202c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c801e460 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f14c801e600 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583e16b3f00 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14c802e050 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 10 client.16959 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.251+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583e17c8670 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5583e16b3f00 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14c8011440 con 0x5583e1747580 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 10 client.16959 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:23.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 20 client.16959 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e17c8df0 0x5583e17cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 4 client.16959 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583e1631820 con 0x5583e17c8df0 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e17c8df0 0x5583e17cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14ce7fc640 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e17c8df0 0x5583e17cb210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b699cdfafcdfc40c server_cookie=c8e5533e6be66a20 in_seq=0 out_seq=0 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.252+0000 7f14b6ffd640 10 client.16959 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14b6ffd640 1 -- 172.21.15.45:0/4221575022 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583e1631820 con 0x5583e17c8df0 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14b6ffd640 10 client.16959 handle_command_reply: tid=0 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 client.16959 shutdown 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 msgr2=0x5583e175a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x5583e1764ee0 tx=0x7f14c8012820 comp rx=0 tx=0).stop 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 shutdown_connections 2024-04-25T00:26:23.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5583e1752570 0x5583e1756210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583e1747ea0 0x5583e1755cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5583e17c8df0 0x5583e17cb210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 --2- 172.21.15.45:0/4221575022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e1747580 0x5583e175a960 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14b6ffd640 0 client.16959 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 >> 172.21.15.45:0/4221575022 conn(0x5583e17438a0 msgr2=0x5583e174ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 shutdown_connections 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 1 -- 172.21.15.45:0/4221575022 wait complete. 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14e0620740 20 client.16959 trim_cache size 0 max 16384 2024-04-25T00:26:23.265 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 msgr2=0x7f14c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 0x7f14c40a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14d0012820 tx=0x7f14d001e000 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 msgr2=0x7f14d819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.253+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f14c0006010 tx=0x7f14c00042f0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 shutdown_connections 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14c40a2d00 0x7f14c40a51c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14d810e150 0x7f14d819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14d806a6e0 0x7f14d819a830 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 --2- 172.21.15.45:0/3832868887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14d8069dc0 0x7f14d819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 >> 172.21.15.45:0/3832868887 conn(0x7f14d80776a0 msgr2=0x7f14d810f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:23.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 shutdown_connections 2024-04-25T00:26:23.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:23.254+0000 7f14dd841640 1 -- 172.21.15.45:0/3832868887 wait complete. 2024-04-25T00:26:24.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:26:24.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:26:24.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 -- 172.21.15.45:0/445001470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1029c0 msgr2=0x7f5ccc102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 --2- 172.21.15.45:0/445001470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc102da0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc02a510 comp rx=0 tx=0).stop 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 -- 172.21.15.45:0/445001470 shutdown_connections 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 --2- 172.21.15.45:0/445001470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc10bd00 0x7f5ccc10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 --2- 172.21.15.45:0/445001470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 --2- 172.21.15.45:0/445001470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc102da0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.387+0000 7f5cd1163640 1 -- 172.21.15.45:0/445001470 >> 172.21.15.45:0/445001470 conn(0x7f5ccc077740 msgr2=0x7f5ccc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.388+0000 7f5cd1163640 1 -- 172.21.15.45:0/445001470 shutdown_connections 2024-04-25T00:26:24.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.388+0000 7f5cd1163640 1 -- 172.21.15.45:0/445001470 wait complete. 2024-04-25T00:26:24.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.388+0000 7f5cd1163640 1 Processor -- start 2024-04-25T00:26:24.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 -- start start 2024-04-25T00:26:24.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc10bd00 0x7f5ccc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111b00 con 0x7f5ccc1032e0 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111980 con 0x7f5ccc10bd00 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cd1163640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111c80 con 0x7f5ccc1029c0 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48724/0 (socket says 172.21.15.45:48724) 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 -- 172.21.15.45:0/2826443923 learned_addr learned my addr 172.21.15.45:0/2826443923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc10bd00 0x7f5ccc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 -- 172.21.15.45:0/2826443923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc1029c0 msgr2=0x7f5ccc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 -- 172.21.15.45:0/2826443923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc10bd00 msgr2=0x7f5ccc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc10bd00 0x7f5ccc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 -- 172.21.15.45:0/2826443923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc19d010 con 0x7f5ccc1032e0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.389+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000d950 tx=0x7f5cc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/2826443923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0017030 con 0x7f5ccc1032e0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.390+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc19d300 con 0x7f5ccc1032e0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/2826443923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc0004020 con 0x7f5ccc1032e0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.390+0000 7f5cb3fff640 1 -- 172.21.15.45:0/2826443923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00047a0 con 0x7f5ccc1032e0 2024-04-25T00:26:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.390+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a4b90 con 0x7f5ccc1032e0 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.392+0000 7f5cb3fff640 1 -- 172.21.15.45:0/2826443923 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5cc00041d0 con 0x7f5ccc1032e0 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.392+0000 7f5cd3e73740 1 Processor -- start 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.392+0000 7f5cd3e73740 1 -- start start 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d61cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c65cc62c0 con 0x562c65d62290 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562c65c6e910 con 0x562c65d63980 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c65cc2280 con 0x562c65d6d010 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cb3fff640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 0x7f5cb40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48736/0 (socket says 172.21.15.45:48736) 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 -- 172.21.15.45:0/1824999569 learned_addr learned my addr 172.21.15.45:0/1824999569 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d61cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cca59c640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 0x7f5cb40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cb3fff640 1 -- 172.21.15.45:0/2826443923 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5cc00cf140 con 0x7f5ccc1032e0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 -- 172.21.15.45:0/1824999569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 msgr2=0x562c65d6f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 -- 172.21.15.45:0/1824999569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 msgr2=0x562c65d61cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d61cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 -- 172.21.15.45:0/1824999569 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c65c7ffb0 con 0x562c65d62290 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5cca59c640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d61cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.393+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7e5f45714aea27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 0x7f5cb40a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fd0 tx=0x7f5cc40096a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cb17fa640 1 -- 172.21.15.45:0/1824999569 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc040070 con 0x562c65d62290 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cb17fa640 1 -- 172.21.15.45:0/1824999569 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc0319f0 con 0x562c65d62290 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cb17fa640 1 -- 172.21.15.45:0/1824999569 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc035040 con 0x562c65d62290 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 -- 172.21.15.45:0/1824999569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 msgr2=0x562c65d62710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc005e80 comp rx=0 tx=0).stop 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 -- 172.21.15.45:0/1824999569 shutdown_connections 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d62710 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/1824999569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d61cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 -- 172.21.15.45:0/1824999569 >> 172.21.15.45:0/1824999569 conn(0x562c65d2a950 msgr2=0x562c65d5e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 -- 172.21.15.45:0/1824999569 shutdown_connections 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.394+0000 7f5cd3e73740 1 -- 172.21.15.45:0/1824999569 wait complete. 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 -- start start 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 Processor -- start 2024-04-25T00:26:24.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562c65d6fe40 con 0x562c65d62290 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562c65c81e90 con 0x562c65d63980 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c65d6ffc0 con 0x562c65d6d010 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cd3e73740 10 client.0 _ll_register_callbacks cb 0x562c65cbd970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48744/0 (socket says 172.21.15.45:48744) 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 -- 172.21.15.45:0/2208436175 learned_addr learned my addr 172.21.15.45:0/2208436175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 -- 172.21.15.45:0/2208436175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 msgr2=0x562c65d6a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 -- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 msgr2=0x562c65d6a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d6a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.395+0000 7f5ccad9d640 1 -- 172.21.15.45:0/2208436175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c65aa27e0 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc02aa30 tx=0x7f5cbc005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31f9b4fd39f47fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc04c020 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5cbc004960 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc0319f0 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cbc057050 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 10 client.16971 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c65b02f70 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc0319f0 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cbc03ccc0 con 0x562c65d62290 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 10 client.16971 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 20 client.16971 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65d7cec0 0x562c65d7f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 4 client.16971 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.396+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c65c9a5a0 con 0x562c65d7cec0 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.397+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65d7cec0 0x562c65d7f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.397+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65d7cec0 0x562c65d7f2b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359338d752a56f0c server_cookie=b61b3d0c941f918 in_seq=0 out_seq=0 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.397+0000 7f5caeff5640 10 client.16971 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5caeff5640 1 -- 172.21.15.45:0/2208436175 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c65c9a5a0 con 0x562c65d7cec0 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5caeff5640 10 client.16971 handle_command_reply: tid=0 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 client.16971 shutdown 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 msgr2=0x562c65d69e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc02aa30 tx=0x7f5cbc005250 comp rx=0 tx=0).stop 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 shutdown_connections 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d6d010 0x562c65d6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d63980 0x562c65d6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65d7cec0 0x562c65d7f2b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2208436175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d62290 0x562c65d69e90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5caeff5640 0 client.16971 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 >> 172.21.15.45:0/2208436175 conn(0x562c65d2a950 msgr2=0x562c65d645f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 shutdown_connections 2024-04-25T00:26:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2208436175 wait complete. 2024-04-25T00:26:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.398+0000 7f5cd3e73740 20 client.16971 trim_cache size 0 max 16384 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 Processor -- start 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 -- start start 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1aeb10 con 0x562c65d63980 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1b0650 con 0x562c65d6d010 2024-04-25T00:26:24.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1af1e0 con 0x562c65d62290 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48756/0 (socket says 172.21.15.45:48756) 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.416+0000 7f5cca59c640 1 -- 172.21.15.45:0/2724736991 learned_addr learned my addr 172.21.15.45:0/2724736991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 -- 172.21.15.45:0/2724736991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 msgr2=0x562c65d78d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 -- 172.21.15.45:0/2724736991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 msgr2=0x562c65d848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 -- 172.21.15.45:0/2724736991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c65d68e40 con 0x562c65d63980 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cca59c640 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009610 tx=0x7f5cc0017ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83ad905fe142b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cb0ff9640 1 -- 172.21.15.45:0/2724736991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0097540 con 0x562c65d63980 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cb0ff9640 1 -- 172.21.15.45:0/2724736991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc009b9a0 con 0x562c65d63980 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cb0ff9640 1 -- 172.21.15.45:0/2724736991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0021ca0 con 0x562c65d63980 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2724736991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 msgr2=0x562c65d5d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009610 tx=0x7f5cc0017ba0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2724736991 shutdown_connections 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d5d7d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.417+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/2724736991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2724736991 >> 172.21.15.45:0/2724736991 conn(0x562c65d2a950 msgr2=0x562c65d5d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2724736991 shutdown_connections 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- 172.21.15.45:0/2724736991 wait complete. 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- start start 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 Processor -- start 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d75290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d70310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1aff80 con 0x562c65d63980 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1af8b0 con 0x562c65d6d010 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1acaf0 con 0x562c65d62290 2024-04-25T00:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cd3e73740 10 client.0 _ll_register_callbacks cb 0x7f5cbc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48762/0 (socket says 172.21.15.45:48762) 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5cca59c640 1 -- 172.21.15.45:0/3012502616 learned_addr learned my addr 172.21.15.45:0/3012502616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d75290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.418+0000 7f5ccb59e640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d70310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 -- 172.21.15.45:0/3012502616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 msgr2=0x562c65d75290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d75290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 -- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 msgr2=0x562c65d70310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d70310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 -- 172.21.15.45:0/3012502616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6599a170 con 0x562c65d63980 2024-04-25T00:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cca59c640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00bb010 tx=0x7f5cc00a4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e6870db9c1422a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5caeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00b4060 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5cc0097980 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc009b9a0 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cc009b2a0 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cd3e73740 10 client.16977 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c65de3b80 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.419+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc009b9a0 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cc0017ca0 con 0x562c65d63980 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 10 client.16977 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 20 client.16977 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:24.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65de4230 0x562c65de6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 4 client.16977 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c65cd3800 con 0x562c65de4230 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65de4230 0x562c65de6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5ccad9d640 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65de4230 0x562c65de6650 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a31a7c4db67608 server_cookie=f5cbb942ccb4a4a1 in_seq=0 out_seq=0 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.420+0000 7f5caeff5640 10 client.16977 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.421+0000 7f5caeff5640 1 -- 172.21.15.45:0/3012502616 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c65cd3800 con 0x562c65de4230 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.421+0000 7f5caeff5640 10 client.16977 handle_command_reply: tid=0 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.421+0000 7f5cd3e73740 1 client.16977 shutdown 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 msgr2=0x562c65d757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00bb010 tx=0x7f5cc00a4000 comp rx=0 tx=0).stop 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 shutdown_connections 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c65d6d010 0x562c65d70310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x562c65de4230 0x562c65de6650 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562c65d63980 0x562c65d757d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 --2- 172.21.15.45:0/3012502616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c65d62290 0x562c65d75290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5caeff5640 0 client.16977 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 >> 172.21.15.45:0/3012502616 conn(0x562c65d2a950 msgr2=0x562c65d5d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 shutdown_connections 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 1 -- 172.21.15.45:0/3012502616 wait complete. 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd3e73740 20 client.16977 trim_cache size 0 max 16384 2024-04-25T00:26:24.429 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:26:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "073f5ead-475c-41c7-9468-1f0084e719a4", 2024-04-25T00:26:24.430 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:26:24.430 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 msgr2=0x7f5cb40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 0x7f5cb40a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fd0 tx=0x7f5cc40096a0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 msgr2=0x7f5ccc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000d950 tx=0x7f5cc000de10 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 shutdown_connections 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5cb40a2d00 0x7f5cb40a51c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ccc10bd00 0x7f5ccc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ccc1032e0 0x7f5ccc198560 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 --2- 172.21.15.45:0/2826443923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ccc1029c0 0x7f5ccc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.422+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 >> 172.21.15.45:0/2826443923 conn(0x7f5ccc077740 msgr2=0x7f5ccc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.423+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 shutdown_connections 2024-04-25T00:26:24.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.423+0000 7f5cd1163640 1 -- 172.21.15.45:0/2826443923 wait complete. 2024-04-25T00:26:24.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:24.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 -- 172.21.15.45:0/3518632271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 msgr2=0x7fc30410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 --2- 172.21.15.45:0/3518632271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30410dbb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009900 tx=0x7fc2f802a410 comp rx=0 tx=0).stop 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 -- 172.21.15.45:0/3518632271 shutdown_connections 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 --2- 172.21.15.45:0/3518632271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 0x7fc3041104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 --2- 172.21.15.45:0/3518632271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30410dbb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 --2- 172.21.15.45:0/3518632271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc304100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 -- 172.21.15.45:0/3518632271 >> 172.21.15.45:0/3518632271 conn(0x7fc304077740 msgr2=0x7fc304077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 -- 172.21.15.45:0/3518632271 shutdown_connections 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 -- 172.21.15.45:0/3518632271 wait complete. 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.538+0000 7fc309f14640 1 Processor -- start 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 -- start start 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc30419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 0x7fc30419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113d20 con 0x7fc304100e60 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113ba0 con 0x7fc30410e0f0 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc309f14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc304113ea0 con 0x7fc304100540 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48774/0 (socket says 172.21.15.45:48774) 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 -- 172.21.15.45:0/1914858418 learned_addr learned my addr 172.21.15.45:0/1914858418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc303fff640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 0x7fc30419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc3037fe640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc30419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 -- 172.21.15.45:0/1914858418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 msgr2=0x7fc30419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc30419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 -- 172.21.15.45:0/1914858418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 msgr2=0x7fc30419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 0x7fc30419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 -- 172.21.15.45:0/1914858418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30419f220 con 0x7fc304100e60 2024-04-25T00:26:24.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc3037fe640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc30419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.539+0000 7fc302ffd640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006010 tx=0x7fc2f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.540+0000 7fc300ff9640 1 -- 172.21.15.45:0/1914858418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f803f030 con 0x7fc304100e60 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.540+0000 7fc300ff9640 1 -- 172.21.15.45:0/1914858418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f8003ad0 con 0x7fc304100e60 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.540+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30419f4b0 con 0x7fc304100e60 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.540+0000 7fc300ff9640 1 -- 172.21.15.45:0/1914858418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8035070 con 0x7fc304100e60 2024-04-25T00:26:24.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.540+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a6da0 con 0x7fc304100e60 2024-04-25T00:26:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc300ff9640 1 -- 172.21.15.45:0/1914858418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc2f8042030 con 0x7fc304100e60 2024-04-25T00:26:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 Processor -- start 2024-04-25T00:26:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 -- start start 2024-04-25T00:26:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df575a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df575f50 0x5557df5800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557df4da6b0 con 0x5557df580620 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5557df482880 con 0x5557df575630 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc30cc28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557df4d6670 con 0x5557df575f50 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc300ff9640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 0x7fc2ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48786/0 (socket says 172.21.15.45:48786) 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.542+0000 7fc302ffd640 1 -- 172.21.15.45:0/4214407078 learned_addr learned my addr 172.21.15.45:0/4214407078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc3037fe640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df575f50 0x5557df5800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc303fff640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df575a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc303fff640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 0x7fc2ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 -- 172.21.15.45:0/4214407078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df575f50 msgr2=0x5557df5800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df575f50 0x5557df5800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc300ff9640 1 -- 172.21.15.45:0/1914858418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc2f80eff90 con 0x7fc304100e60 2024-04-25T00:26:24.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 -- 172.21.15.45:0/4214407078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 msgr2=0x5557df575a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df575a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 -- 172.21.15.45:0/4214407078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557df2b87d0 con 0x5557df580620 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc303fff640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df575a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc302ffd640 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80c38a0 tx=0x7fc2f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62768ad29f4f8597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc303fff640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 0x7fc2ec0a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005c40 tx=0x7fc2fc009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc2e9ffb640 1 -- 172.21.15.45:0/4214407078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8049030 con 0x5557df580620 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc2e9ffb640 1 -- 172.21.15.45:0/4214407078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f803b460 con 0x5557df580620 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.543+0000 7fc2e9ffb640 1 -- 172.21.15.45:0/4214407078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003c80 con 0x5557df580620 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- 172.21.15.45:0/4214407078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 msgr2=0x5557df582a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80c38a0 tx=0x7fc2f802a410 comp rx=0 tx=0).stop 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- 172.21.15.45:0/4214407078 shutdown_connections 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df580620 0x5557df582a10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df575f50 0x5557df5800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 --2- 172.21.15.45:0/4214407078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df575a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- 172.21.15.45:0/4214407078 >> 172.21.15.45:0/4214407078 conn(0x5557df571950 msgr2=0x5557df571d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- 172.21.15.45:0/4214407078 shutdown_connections 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- 172.21.15.45:0/4214407078 wait complete. 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 -- start start 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.544+0000 7fc30cc28740 1 Processor -- start 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df57d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df580620 0x5557df57e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557df4da6b0 con 0x5557df575f50 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5557df495f50 con 0x5557df575630 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557df5831d0 con 0x5557df580620 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 10 client.0 _ll_register_callbacks cb 0x5557df4d1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48796/0 (socket says 172.21.15.45:48796) 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 -- 172.21.15.45:0/3793320619 learned_addr learned my addr 172.21.15.45:0/3793320619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc303fff640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df580620 0x5557df57e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc3037fe640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df57d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 -- 172.21.15.45:0/3793320619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df580620 msgr2=0x5557df57e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df580620 0x5557df57e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 -- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 msgr2=0x5557df57d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df57d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 -- 172.21.15.45:0/3793320619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557df1ee3f0 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc302ffd640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ed980 tx=0x7fc2f80b85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de0115ecbaa095a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8049040 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc2f80ed600 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.545+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557df4e7bf0 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2f803be30 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 10 client.16989 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557df1ae0f0 con 0x5557df575f50 2024-04-25T00:26:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5557df4e7bf0 con 0x5557df575f50 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2f80c49f0 con 0x5557df575f50 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 10 client.16989 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 20 client.16989 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df590450 0x5557df592840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 4 client.16989 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557df4e1f40 con 0x5557df590450 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.546+0000 7fc3037fe640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df590450 0x5557df592840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.547+0000 7fc2e2ffd640 10 client.16989 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.547+0000 7fc3037fe640 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df590450 0x5557df592840 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c91b8f6b3cbcf4 server_cookie=56b9c497214d7834 in_seq=0 out_seq=0 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.547+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/3793320619 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557df4e1f40 con 0x5557df590450 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.547+0000 7fc2e2ffd640 10 client.16989 handle_command_reply: tid=0 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.547+0000 7fc30cc28740 1 client.16989 shutdown 2024-04-25T00:26:24.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 msgr2=0x5557df57db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ed980 tx=0x7fc2f80b85a0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 shutdown_connections 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df580620 0x5557df57e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df590450 0x5557df592840 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df57db50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 --2- 172.21.15.45:0/3793320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df57d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc2e2ffd640 0 client.16989 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 >> 172.21.15.45:0/3793320619 conn(0x5557df571950 msgr2=0x5557df577750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 shutdown_connections 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 1 -- 172.21.15.45:0/3793320619 wait complete. 2024-04-25T00:26:24.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.548+0000 7fc30cc28740 20 client.16989 trim_cache size 0 max 16384 2024-04-25T00:26:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 Processor -- start 2024-04-25T00:26:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 -- start start 2024-04-25T00:26:24.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df58bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 0x5557df598020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b0d70 con 0x5557df575630 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b28b0 con 0x5557df575f50 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc30cc28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b1440 con 0x5557df571950 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48804/0 (socket says 172.21.15.45:48804) 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.565+0000 7fc302ffd640 1 -- 172.21.15.45:0/2141786893 learned_addr learned my addr 172.21.15.45:0/2141786893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc303fff640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 0x5557df598020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc3037fe640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df58bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 -- 172.21.15.45:0/2141786893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 msgr2=0x5557df58bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df58bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 -- 172.21.15.45:0/2141786893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 msgr2=0x5557df598020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 0x5557df598020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 -- 172.21.15.45:0/2141786893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557df563090 con 0x5557df575630 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc303fff640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 0x5557df598020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc302ffd640 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beae3884c54b363a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc2e97fa640 1 -- 172.21.15.45:0/2141786893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8049020 con 0x5557df575630 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc2e97fa640 1 -- 172.21.15.45:0/2141786893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f80bc210 con 0x5557df575630 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.566+0000 7fc2e97fa640 1 -- 172.21.15.45:0/2141786893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80c4b70 con 0x5557df575630 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- 172.21.15.45:0/2141786893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 msgr2=0x5557df577ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000c00 tx=0x7fc2f802a410 comp rx=0 tx=0).stop 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- 172.21.15.45:0/2141786893 shutdown_connections 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575f50 0x5557df598020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575630 0x5557df577ea0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- 172.21.15.45:0/2141786893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df58bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- 172.21.15.45:0/2141786893 >> 172.21.15.45:0/2141786893 conn(0x5557df57c250 msgr2=0x5557df57de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- 172.21.15.45:0/2141786893 shutdown_connections 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- 172.21.15.45:0/2141786893 wait complete. 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- start start 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 Processor -- start 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df5887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df583b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b21e0 con 0x5557df575f50 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041b1b10 con 0x5557df575630 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3041aed50 con 0x5557df571950 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc30cc28740 10 client.0 _ll_register_callbacks cb 0x7fc2f4009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.567+0000 7fc303fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48808/0 (socket says 172.21.15.45:48808) 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 -- 172.21.15.45:0/1674800265 learned_addr learned my addr 172.21.15.45:0/1674800265 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc3037fe640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df5887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc302ffd640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df583b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 -- 172.21.15.45:0/1674800265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 msgr2=0x5557df5887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df5887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 -- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 msgr2=0x5557df583b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df583b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 -- 172.21.15.45:0/1674800265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557df584850 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc302ffd640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df583b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc303fff640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d8e0 tx=0x7fc2fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc660c5e2dd7c3da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc018070 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc2fc009040 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f80bc210 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc011bd0 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc30cc28740 10 client.16995 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557df5f68c0 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.568+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f80bc210 con 0x5557df575f50 2024-04-25T00:26:24.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2fc01b070 con 0x5557df575f50 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 10 client.16995 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 20 client.16995 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df580620 0x5557df5f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 4 client.16995 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557df45f6f0 con 0x5557df580620 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc3037fe640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df580620 0x5557df5f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc3037fe640 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df580620 0x5557df5f9100 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88343766179ddcc server_cookie=fe7537d00975aaeb in_seq=0 out_seq=0 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.569+0000 7fc2e2ffd640 10 client.16995 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc2e2ffd640 1 -- 172.21.15.45:0/1674800265 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557df45f6f0 con 0x5557df580620 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc2e2ffd640 10 client.16995 handle_command_reply: tid=0 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 client.16995 shutdown 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 msgr2=0x5557df584050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d8e0 tx=0x7fc2fc004a40 comp rx=0 tx=0).stop 2024-04-25T00:26:24.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 shutdown_connections 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5557df580620 0x5557df5f9100 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557df575f50 0x5557df584050 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5557df575630 0x5557df583b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc2e2ffd640 0 client.16995 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 --2- 172.21.15.45:0/1674800265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557df571950 0x5557df5887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 >> 172.21.15.45:0/1674800265 conn(0x5557df57c250 msgr2=0x5557df57ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 shutdown_connections 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 1 -- 172.21.15.45:0/1674800265 wait complete. 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc30cc28740 20 client.16995 trim_cache size 0 max 16384 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:24.578 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "073f5ead-475c-41c7-9468-1f0084e719a4": { 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "073f5ead-475c-41c7-9468-1f0084e719a4", 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:26:24.579 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 msgr2=0x7fc2ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 0x7fc2ec0a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005c40 tx=0x7fc2fc009500 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 msgr2=0x7fc30419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.570+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006010 tx=0x7fc2f8004270 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 shutdown_connections 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2ec0a2d00 0x7fc2ec0a51c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc30410e0f0 0x7fc30419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc304100e60 0x7fc30419a7c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 --2- 172.21.15.45:0/1914858418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc304100540 0x7fc30419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 >> 172.21.15.45:0/1914858418 conn(0x7fc304077740 msgr2=0x7fc30410f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 shutdown_connections 2024-04-25T00:26:24.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:24.571+0000 7fc309f14640 1 -- 172.21.15.45:0/1914858418 wait complete. 2024-04-25T00:26:24.582 INFO:tasks.cephfs.filesystem:scrub status for tag:073f5ead-475c-41c7-9468-1f0084e719a4 - {'path': '/', 'tag': '073f5ead-475c-41c7-9468-1f0084e719a4', 'options': 'recursive,force'} 2024-04-25T00:26:24.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:24 smithi045 ceph-mon[29194]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T00:26:24.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:24 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:26:24.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:24 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:24.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:24 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:26:24.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:24 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:24 smithi122 ceph-mon[32959]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T00:26:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:24 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:26:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:24 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:24 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:26:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:24 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:25.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:24 smithi071 ceph-mon[32667]: pgmap v136: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-25T00:26:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:24 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:26:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:24 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:24 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:26:25.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:24 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:27.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:26 smithi122 ceph-mon[32959]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:26 smithi071 ceph-mon[32667]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:26 smithi045 ceph-mon[29194]: pgmap v137: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:27 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:26:28.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:27 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:26:28.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:27 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:26:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:28 smithi122 ceph-mon[32959]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T00:26:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:28 smithi071 ceph-mon[32667]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T00:26:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:28 smithi045 ceph-mon[29194]: pgmap v138: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T00:26:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:30 smithi122 ceph-mon[32959]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T00:26:31.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:30 smithi071 ceph-mon[32667]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T00:26:31.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:30 smithi045 ceph-mon[29194]: pgmap v139: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T00:26:32.951 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:32 smithi045 ceph-mon[29194]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T00:26:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:32 smithi122 ceph-mon[32959]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T00:26:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:32 smithi071 ceph-mon[32667]: pgmap v140: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T00:26:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:34 smithi122 ceph-mon[32959]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:26:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:34 smithi071 ceph-mon[32667]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:26:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:34 smithi045 ceph-mon[29194]: pgmap v141: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T00:26:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:36 smithi122 ceph-mon[32959]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:26:37.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:36 smithi071 ceph-mon[32667]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:26:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:36 smithi045 ceph-mon[29194]: pgmap v142: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:26:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:38 smithi122 ceph-mon[32959]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:39.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:38 smithi071 ceph-mon[32667]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:39.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:38 smithi045 ceph-mon[29194]: pgmap v143: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:26:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:40 smithi122 ceph-mon[32959]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:41.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:40 smithi071 ceph-mon[32667]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:41.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:40 smithi045 ceph-mon[29194]: pgmap v144: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:26:43.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:42 smithi122 ceph-mon[32959]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:43.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:42 smithi071 ceph-mon[32667]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:43.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:42 smithi045 ceph-mon[29194]: pgmap v145: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:45.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:44 smithi122 ceph-mon[32959]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:45.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:44 smithi071 ceph-mon[32667]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:45.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:44 smithi045 ceph-mon[29194]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:47.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:46 smithi122 ceph-mon[32959]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:47.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:46 smithi071 ceph-mon[32667]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:47.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:46 smithi045 ceph-mon[29194]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:49.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:48 smithi122 ceph-mon[32959]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:49.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:48 smithi071 ceph-mon[32667]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:49.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:48 smithi045 ceph-mon[29194]: pgmap v148: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:51.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:50 smithi122 ceph-mon[32959]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:26:51.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:50 smithi071 ceph-mon[32667]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:26:51.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:50 smithi045 ceph-mon[29194]: pgmap v149: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:26:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:52 smithi122 ceph-mon[32959]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:53.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:52 smithi071 ceph-mon[32667]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:53.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:52 smithi045 ceph-mon[29194]: pgmap v150: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:54.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:54.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:54.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 -- 172.21.15.45:0/703226937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6984106aa0 msgr2=0x7f698410e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 --2- 172.21.15.45:0/703226937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6984106aa0 0x7f698410e6b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f697c009ca0 tx=0x7f697c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 -- 172.21.15.45:0/703226937 shutdown_connections 2024-04-25T00:26:54.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 --2- 172.21.15.45:0/703226937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6984106aa0 0x7f698410e6b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 --2- 172.21.15.45:0/703226937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984100c90 0x7f69841063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 --2- 172.21.15.45:0/703226937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 -- 172.21.15.45:0/703226937 >> 172.21.15.45:0/703226937 conn(0x7f69840758d0 msgr2=0x7f6984075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 -- 172.21.15.45:0/703226937 shutdown_connections 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.702+0000 7f698c363640 1 -- 172.21.15.45:0/703226937 wait complete. 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 Processor -- start 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 -- start start 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106aa0 0x7f69841985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6984111b90 con 0x7f698419c970 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6984111a10 con 0x7f69841002e0 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698c363640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6984111d10 con 0x7f6984106aa0 2024-04-25T00:26:54.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46414/0 (socket says 172.21.15.45:46414) 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 -- 172.21.15.45:0/3714366801 learned_addr learned my addr 172.21.15.45:0/3714366801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f69898fe640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106aa0 0x7f69841985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a0ff640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 -- 172.21.15.45:0/3714366801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106aa0 msgr2=0x7f69841985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106aa0 0x7f69841985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 -- 172.21.15.45:0/3714366801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 msgr2=0x7f69841980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a900640 1 -- 172.21.15.45:0/3714366801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698419d460 con 0x7f698419c970 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.703+0000 7f698a0ff640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f698a900640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f697c031a20 tx=0x7f697c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f697b7fe640 1 -- 172.21.15.45:0/3714366801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c003c30 con 0x7f698419c970 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f697b7fe640 1 -- 172.21.15.45:0/3714366801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c003dd0 con 0x7f698419c970 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698419d6f0 con 0x7f698419c970 2024-04-25T00:26:54.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f697b7fe640 1 -- 172.21.15.45:0/3714366801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c02fcf0 con 0x7f698419c970 2024-04-25T00:26:54.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.704+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a4f40 con 0x7f698419c970 2024-04-25T00:26:54.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.706+0000 7f697b7fe640 1 -- 172.21.15.45:0/3714366801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f697c004050 con 0x7f698419c970 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.706+0000 7f698f073740 1 Processor -- start 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 -- start start 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7630 0x55e5ab5c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f697b7fe640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 0x7f69700a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab52c6b0 con 0x55e5ab5c7f50 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab4d4880 con 0x55e5ab5d2620 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698f073740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab528670 con 0x55e5ab5c7630 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a900640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 0x7f69700a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46428/0 (socket says 172.21.15.45:46428) 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 -- 172.21.15.45:0/632526723 learned_addr learned my addr 172.21.15.45:0/632526723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f69898fe640 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7630 0x55e5ab5c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f69898fe640 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f697b7fe640 1 -- 172.21.15.45:0/3714366801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f697c0f0be0 con 0x7f698419c970 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 -- 172.21.15.45:0/632526723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7630 msgr2=0x55e5ab5c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7630 0x55e5ab5c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.707+0000 7f698a0ff640 1 -- 172.21.15.45:0/632526723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 msgr2=0x55e5ab5d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f698a0ff640 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f698a0ff640 1 -- 172.21.15.45:0/632526723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ab30a7d0 con 0x55e5ab5c7f50 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f698a900640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 0x7f69700a5240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f697c0bcc40 tx=0x7f697c02a8c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f698a0ff640 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6980005fd0 tx=0x7f698000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a82a1b8e298f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f6978bfe640 1 -- 172.21.15.45:0/632526723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980016070 con 0x55e5ab5c7f50 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f6978bfe640 1 -- 172.21.15.45:0/632526723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980005500 con 0x55e5ab5c7f50 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.708+0000 7f6978bfe640 1 -- 172.21.15.45:0/632526723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980011470 con 0x55e5ab5c7f50 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- 172.21.15.45:0/632526723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 msgr2=0x55e5ab5d20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6980005fd0 tx=0x7f698000e6c0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- 172.21.15.45:0/632526723 shutdown_connections 2024-04-25T00:26:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d20e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 --2- 172.21.15.45:0/632526723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7630 0x55e5ab5c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- 172.21.15.45:0/632526723 >> 172.21.15.45:0/632526723 conn(0x55e5ab5c3950 msgr2=0x55e5ab5c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- 172.21.15.45:0/632526723 shutdown_connections 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- 172.21.15.45:0/632526723 wait complete. 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 -- start start 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.709+0000 7f698f073740 1 Processor -- start 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab5d5410 con 0x55e5ab5c7630 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab4e7f50 con 0x55e5ab5d2620 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e5ab5d5590 con 0x55e5ab5c7f50 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 10 client.0 _ll_register_callbacks cb 0x55e5ab523000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46434/0 (socket says 172.21.15.45:46434) 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 -- 172.21.15.45:0/3904496552 learned_addr learned my addr 172.21.15.45:0/3904496552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698a0ff640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 -- 172.21.15.45:0/3904496552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 msgr2=0x55e5ab5cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 -- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 msgr2=0x55e5ab5cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 -- 172.21.15.45:0/3904496552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ab2403f0 con 0x55e5ab5c7630 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f69898fe640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f697400ef40 tx=0x7f697400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e31852bf19e542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f696dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740142d0 con 0x55e5ab5c7630 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6974005020 con 0x55e5ab5c7630 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.710+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6980005500 con 0x55e5ab5c7630 2024-04-25T00:26:54.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6974026050 con 0x55e5ab5c7630 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 10 client.17007 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ab2000f0 con 0x55e5ab5c7630 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980005500 con 0x55e5ab5c7630 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f697401e020 con 0x55e5ab5c7630 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 10 client.17007 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 20 client.17007 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab5e28c0 0x55e5ab5e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 4 client.17007 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ab539bf0 con 0x55e5ab5e28c0 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.711+0000 7f698a0ff640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab5e28c0 0x55e5ab5e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.712+0000 7f698a0ff640 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab5e28c0 0x55e5ab5e4d10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d4671df62be8c1 server_cookie=1d1f2979dd3bcaca in_seq=0 out_seq=0 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.712+0000 7f696dffb640 10 client.17007 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f696dffb640 1 -- 172.21.15.45:0/3904496552 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5ab539bf0 con 0x55e5ab5e28c0 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f696dffb640 10 client.17007 handle_command_reply: tid=0 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 client.17007 shutdown 2024-04-25T00:26:54.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 msgr2=0x55e5ab5cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f697400ef40 tx=0x7f697400ebb0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 shutdown_connections 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab5e28c0 0x55e5ab5e4d10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 --2- 172.21.15.45:0/3904496552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5cf4d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f696dffb640 0 client.17007 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 >> 172.21.15.45:0/3904496552 conn(0x55e5ab5c3950 msgr2=0x55e5ab5c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 shutdown_connections 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 1 -- 172.21.15.45:0/3904496552 wait complete. 2024-04-25T00:26:54.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.713+0000 7f698f073740 20 client.17007 trim_cache size 0 max 16384 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 Processor -- start 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 -- start start 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69841aeb60 con 0x55e5ab5c7630 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f69841b06a0 con 0x55e5ab5d2620 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.731+0000 7f698f073740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69841af230 con 0x55e5ab5c7f50 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:42566/0 (socket says 172.21.15.45:42566) 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46446/0 (socket says 172.21.15.45:46446) 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 -- 172.21.15.45:0/2493088973 learned_addr learned my addr 172.21.15.45:0/2493088973 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f69898fe640 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 -- 172.21.15.45:0/2493088973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 msgr2=0x55e5ab5cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 -- 172.21.15.45:0/2493088973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 msgr2=0x55e5ab5de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 -- 172.21.15.45:0/2493088973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ab5c7d80 con 0x55e5ab5c7f50 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f69898fe640 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6980000c00 tx=0x7f6980004c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3819f69700b1497e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f696ffff640 1 -- 172.21.15.45:0/2493088973 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980016040 con 0x55e5ab5c7f50 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f696ffff640 1 -- 172.21.15.45:0/2493088973 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69800097e0 con 0x55e5ab5c7f50 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.732+0000 7f696ffff640 1 -- 172.21.15.45:0/2493088973 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004c90 con 0x55e5ab5c7f50 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- 172.21.15.45:0/2493088973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 msgr2=0x55e5ab5c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6980000c00 tx=0x7f6980004c20 comp rx=0 tx=0).stop 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- 172.21.15.45:0/2493088973 shutdown_connections 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5d2620 0x55e5ab5cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5c9d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 --2- 172.21.15.45:0/2493088973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- 172.21.15.45:0/2493088973 >> 172.21.15.45:0/2493088973 conn(0x55e5ab5c3950 msgr2=0x55e5ab5cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- 172.21.15.45:0/2493088973 shutdown_connections 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- 172.21.15.45:0/2493088973 wait complete. 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.733+0000 7f698f073740 1 -- start start 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 Processor -- start 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69841affd0 con 0x55e5ab5c7630 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f69841af900 con 0x55e5ab5c7f50 2024-04-25T00:26:54.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69841acb40 con 0x55e5ab5d2620 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698f073740 10 client.0 _ll_register_callbacks cb 0x7f697400e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46460/0 (socket says 172.21.15.45:46460) 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 -- 172.21.15.45:0/2514526324 learned_addr learned my addr 172.21.15.45:0/2514526324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 -- 172.21.15.45:0/2514526324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5d2620 msgr2=0x55e5ab5d6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 -- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 msgr2=0x55e5ab5d5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f69898fe640 1 -- 172.21.15.45:0/2514526324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ab5d6970 con 0x55e5ab5c7630 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.734+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f69898fe640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6974000c00 tx=0x7f6974022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d56f56ddabc9852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974012d50 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6974004640 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69800097e0 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6974037050 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 10 client.17013 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ab648c70 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69800097e0 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f697401e040 con 0x55e5ab5c7630 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 10 client.17013 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 20 client.17013 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab649390 0x55e5ab64b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 4 client.17013 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.735+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5ab533f40 con 0x55e5ab649390 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab649390 0x55e5ab64b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f696dffb640 10 client.17013 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698a0ff640 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab649390 0x55e5ab64b7b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f7bd74df8fb95ae server_cookie=4bdb1fd08ea0382d in_seq=0 out_seq=0 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f696dffb640 1 -- 172.21.15.45:0/2514526324 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e5ab533f40 con 0x55e5ab649390 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f696dffb640 10 client.17013 handle_command_reply: tid=0 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 client.17013 shutdown 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 msgr2=0x55e5ab5da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6974000c00 tx=0x7f6974022040 comp rx=0 tx=0).stop 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 shutdown_connections 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e5ab5d2620 0x55e5ab5d6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e5ab5c7f50 0x55e5ab5d5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e5ab649390 0x55e5ab64b7b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 --2- 172.21.15.45:0/2514526324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5ab5c7630 0x55e5ab5da9c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 >> 172.21.15.45:0/2514526324 conn(0x55e5ab5c3950 msgr2=0x55e5ab5cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f696dffb640 0 client.17013 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 shutdown_connections 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 1 -- 172.21.15.45:0/2514526324 wait complete. 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.736+0000 7f698f073740 20 client.17013 trim_cache size 0 max 16384 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:26:54.745 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 msgr2=0x7f69700a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 0x7f69700a5240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f697c0bcc40 tx=0x7f697c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 msgr2=0x7f698419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f697c031a20 tx=0x7f697c031a50 comp rx=0 tx=0).stop 2024-04-25T00:26:54.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 shutdown_connections 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f69700a2d80 0x7f69700a5240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f698419c970 0x7f698419ce20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6984106aa0 0x7f69841985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 --2- 172.21.15.45:0/3714366801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69841002e0 0x7f69841980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 >> 172.21.15.45:0/3714366801 conn(0x7f69840758d0 msgr2=0x7f69840fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 shutdown_connections 2024-04-25T00:26:54.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.737+0000 7f698c363640 1 -- 172.21.15.45:0/3714366801 wait complete. 2024-04-25T00:26:54.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:26:54.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:26:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 -- 172.21.15.45:0/1575355591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1029d0 msgr2=0x7f90fc102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 --2- 172.21.15.45:0/1575355591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1029d0 0x7f90fc102db0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009900 tx=0x7f90ec02a510 comp rx=0 tx=0).stop 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 -- 172.21.15.45:0/1575355591 shutdown_connections 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 --2- 172.21.15.45:0/1575355591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 0x7f90fc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 --2- 172.21.15.45:0/1575355591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1032f0 0x7f90fc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 --2- 172.21.15.45:0/1575355591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1029d0 0x7f90fc102db0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 -- 172.21.15.45:0/1575355591 >> 172.21.15.45:0/1575355591 conn(0x7f90fc077740 msgr2=0x7f90fc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 -- 172.21.15.45:0/1575355591 shutdown_connections 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 -- 172.21.15.45:0/1575355591 wait complete. 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.851+0000 7f9101c38640 1 Processor -- start 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 -- start start 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 0x7f90fc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 0x7f90fc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc111b10 con 0x7f90fc1032f0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc111990 con 0x7f90fc1029d0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f9101c38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc111c90 con 0x7f90fc10bd10 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46488/0 (socket says 172.21.15.45:46488) 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 -- 172.21.15.45:0/4213967927 learned_addr learned my addr 172.21.15.45:0/4213967927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90fbfff640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 0x7f90fc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90fb7fe640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 0x7f90fc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 -- 172.21.15.45:0/4213967927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 msgr2=0x7f90fc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 0x7f90fc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 -- 172.21.15.45:0/4213967927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 msgr2=0x7f90fc198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 0x7f90fc198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 -- 172.21.15.45:0/4213967927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc19d000 con 0x7f90fc1032f0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90fb7fe640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 0x7f90fc198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.852+0000 7f90faffd640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.853+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000cd80 con 0x7f90fc1032f0 2024-04-25T00:26:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.853+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f0010910 con 0x7f90fc1032f0 2024-04-25T00:26:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.853+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc19d2f0 con 0x7f90fc1032f0 2024-04-25T00:26:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.853+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018990 con 0x7f90fc1032f0 2024-04-25T00:26:54.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.853+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc1a4b80 con 0x7f90fc1032f0 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.854+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc068560 con 0x7f90fc1032f0 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.855+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f90f0010ac0 con 0x7f90fc1032f0 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.855+0000 7f90f8ff9640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 0x7f90e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.855+0000 7f90fb7fe640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 0x7f90e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.855+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f90f00cfb50 con 0x7f90fc1032f0 2024-04-25T00:26:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.856+0000 7f90fb7fe640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 0x7f90e40a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005ab0 tx=0x7f90ec005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:54.860+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90f009c410 con 0x7f90fc1032f0 2024-04-25T00:26:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:54 smithi045 ceph-mon[29194]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:55.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:54 smithi122 ceph-mon[32959]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:55.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.083+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90fc02d170 con 0x7f90fc1032f0 2024-04-25T00:26:55.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.083+0000 7f90f8ff9640 1 -- 172.21.15.45:0/4213967927 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f90f009bd10 con 0x7f90fc1032f0 2024-04-25T00:26:55.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:26:55.084 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:26:55.084 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 msgr2=0x7f90e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 0x7f90e40a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005ab0 tx=0x7f90ec005a20 comp rx=0 tx=0).stop 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 msgr2=0x7f90fc198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 shutdown_connections 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f90e40a2d00 0x7f90e40a51c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc10bd10 0x7f90fc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90fc1032f0 0x7f90fc198550 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 --2- 172.21.15.45:0/4213967927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc1029d0 0x7f90fc198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 >> 172.21.15.45:0/4213967927 conn(0x7f90fc077740 msgr2=0x7f90fc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 shutdown_connections 2024-04-25T00:26:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.085+0000 7f9101c38640 1 -- 172.21.15.45:0/4213967927 wait complete. 2024-04-25T00:26:55.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:26:55.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:54 smithi071 ceph-mon[32667]: pgmap v151: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:26:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- 172.21.15.45:0/3968884656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 msgr2=0x7f7050100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- 172.21.15.45:0/3968884656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050100a90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7040009930 tx=0x7f704002a420 comp rx=0 tx=0).stop 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- 172.21.15.45:0/3968884656 shutdown_connections 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- 172.21.15.45:0/3968884656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050100fd0 0x7f705010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- 172.21.15.45:0/3968884656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050100a90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- 172.21.15.45:0/3968884656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70501066b0 0x7f7050106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- 172.21.15.45:0/3968884656 >> 172.21.15.45:0/3968884656 conn(0x7f7050077740 msgr2=0x7f7050077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- 172.21.15.45:0/3968884656 shutdown_connections 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- 172.21.15.45:0/3968884656 wait complete. 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 Processor -- start 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- start start 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7050100fd0 0x7f7050198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70501066b0 0x7f705019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.201+0000 7f7058742640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7050111b80 con 0x7f7050100610 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7058742640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7050111a00 con 0x7f70501066b0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7058742640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7050111d00 con 0x7f7050100fd0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46502/0 (socket says 172.21.15.45:46502) 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 -- 172.21.15.45:0/4214579179 learned_addr learned my addr 172.21.15.45:0/4214579179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70501066b0 0x7f705019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7055cdd640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7050100fd0 0x7f7050198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 -- 172.21.15.45:0/4214579179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7050100fd0 msgr2=0x7f7050198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7050100fd0 0x7f7050198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 -- 172.21.15.45:0/4214579179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70501066b0 msgr2=0x7f705019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70501066b0 0x7f705019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 -- 172.21.15.45:0/4214579179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705019cfa0 con 0x7f7050100610 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70564de640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f704c00d950 tx=0x7f704c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70477fe640 1 -- 172.21.15.45:0/4214579179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c017070 con 0x7f7050100610 2024-04-25T00:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70477fe640 1 -- 172.21.15.45:0/4214579179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c009c30 con 0x7f7050100610 2024-04-25T00:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705019d290 con 0x7f7050100610 2024-04-25T00:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f70477fe640 1 -- 172.21.15.45:0/4214579179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00f600 con 0x7f7050100610 2024-04-25T00:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.202+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501a4b20 con 0x7f7050100610 2024-04-25T00:26:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.204+0000 7f70477fe640 1 -- 172.21.15.45:0/4214579179 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f704c01a070 con 0x7f7050100610 2024-04-25T00:26:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 Processor -- start 2024-04-25T00:26:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 -- start start 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b969440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b96bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561e5b8bf670 con 0x561e5b95e990 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561e5b867970 con 0x561e5b95f2b0 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f705b44f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e5b8bb630 con 0x561e5b969980 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46508/0 (socket says 172.21.15.45:46508) 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f7056cdf640 1 -- 172.21.15.45:0/4077186083 learned_addr learned my addr 172.21.15.45:0/4077186083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.205+0000 7f70477fe640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 0x7f703c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f70564de640 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b969440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7055cdd640 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b96bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 0x7f703c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f70477fe640 1 -- 172.21.15.45:0/4214579179 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f704c0cf190 con 0x7f7050100610 2024-04-25T00:26:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 -- 172.21.15.45:0/4077186083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 msgr2=0x561e5b96bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b96bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 -- 172.21.15.45:0/4077186083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 msgr2=0x561e5b969440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b969440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 -- 172.21.15.45:0/4077186083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5b88b400 con 0x561e5b95e990 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7048005f40 tx=0x7f7048014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139fee51032eac32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7044ff9640 1 -- 172.21.15.45:0/4077186083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048012d90 con 0x561e5b95e990 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7044ff9640 1 -- 172.21.15.45:0/4077186083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048023070 con 0x561e5b95e990 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7044ff9640 1 -- 172.21.15.45:0/4077186083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048005690 con 0x561e5b95e990 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.206+0000 7f7056cdf640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 0x7f703c0a5240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7048022000 tx=0x7f7048020000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- 172.21.15.45:0/4077186083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 msgr2=0x561e5b95ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7048005f40 tx=0x7f7048014f70 comp rx=0 tx=0).stop 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- 172.21.15.45:0/4077186083 shutdown_connections 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b96bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b969440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- 172.21.15.45:0/4077186083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95e990 0x561e5b95ed70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- 172.21.15.45:0/4077186083 >> 172.21.15.45:0/4077186083 conn(0x561e5b95acb0 msgr2=0x561e5b95b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- 172.21.15.45:0/4077186083 shutdown_connections 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- 172.21.15.45:0/4077186083 wait complete. 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 -- start start 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 Processor -- start 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b9692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.207+0000 7f705b44f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95f2b0 0x561e5b9665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561e5b96c770 con 0x561e5b969980 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561e5b87b050 con 0x561e5b95e990 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e5b96c8f0 con 0x561e5b95f2b0 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 10 client.0 _ll_register_callbacks cb 0x561e5b5d2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46522/0 (socket says 172.21.15.45:46522) 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 -- 172.21.15.45:0/3446890004 learned_addr learned my addr 172.21.15.45:0/3446890004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f70564de640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b9692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7055cdd640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95f2b0 0x561e5b9665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 -- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95f2b0 msgr2=0x561e5b9665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95f2b0 0x561e5b9665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 -- 172.21.15.45:0/3446890004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 msgr2=0x561e5b9692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b9692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 -- 172.21.15.45:0/3446890004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5b84cdc0 con 0x561e5b969980 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f70564de640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b9692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7048005370 tx=0x7f7048009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a189fd584e9c6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048012d90 con 0x561e5b969980 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7048005970 con 0x561e5b969980 2024-04-25T00:26:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7048023070 con 0x561e5b969980 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70480144f0 con 0x561e5b969980 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.208+0000 7f705b44f740 10 client.17031 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e5b5d33f0 con 0x561e5b969980 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048023070 con 0x561e5b969980 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f704800f040 con 0x561e5b969980 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 10 client.17031 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 10 client.17031 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 20 client.17031 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b979950 0x561e5b97bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 4 client.17031 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5b8ccbb0 con 0x561e5b979950 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.209+0000 7f70564de640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b979950 0x561e5b97bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.210+0000 7f70564de640 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b979950 0x561e5b97bd40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecc83a18e79f5b3e server_cookie=dbac88e149ee6781 in_seq=0 out_seq=0 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.210+0000 7f7039ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.210+0000 7f7039ffb640 1 -- 172.21.15.45:0/3446890004 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e5b8ccbb0 con 0x561e5b979950 2024-04-25T00:26:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.210+0000 7f7039ffb640 10 client.17031 handle_command_reply: tid=0 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 client.17031 shutdown 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 msgr2=0x561e5b966af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7048005370 tx=0x7f7048009b40 comp rx=0 tx=0).stop 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 shutdown_connections 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b979950 0x561e5b97bd40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b966af0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95f2b0 0x561e5b9665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 --2- 172.21.15.45:0/3446890004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b9692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f7039ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 >> 172.21.15.45:0/3446890004 conn(0x561e5b95acb0 msgr2=0x561e5b955b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 shutdown_connections 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 1 -- 172.21.15.45:0/3446890004 wait complete. 2024-04-25T00:26:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.211+0000 7f705b44f740 20 client.17031 trim_cache size 0 max 16384 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 Processor -- start 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 -- start start 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95e990 0x561e5b975630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b9552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70501ac830 con 0x561e5b969980 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70501ae370 con 0x561e5b95f2b0 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f705b44f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70501acf00 con 0x561e5b95e990 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f7056cdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46536/0 (socket says 172.21.15.45:46536) 2024-04-25T00:26:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f7056cdf640 1 -- 172.21.15.45:0/3670435115 learned_addr learned my addr 172.21.15.45:0/3670435115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.224+0000 7f70564de640 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95e990 0x561e5b975630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 -- 172.21.15.45:0/3670435115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95e990 msgr2=0x561e5b975630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7055cdd640 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b9552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95e990 0x561e5b975630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 -- 172.21.15.45:0/3670435115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 msgr2=0x561e5b9552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b9552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 -- 172.21.15.45:0/3670435115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5b95f0e0 con 0x561e5b969980 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f7056cdf640 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f70480056f0 tx=0x7f7048009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51fcc87a0dd3ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f703bfff640 1 -- 172.21.15.45:0/3670435115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048005050 con 0x561e5b969980 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f703bfff640 1 -- 172.21.15.45:0/3670435115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048023070 con 0x561e5b969980 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f703bfff640 1 -- 172.21.15.45:0/3670435115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048005970 con 0x561e5b969980 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 -- 172.21.15.45:0/3670435115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 msgr2=0x561e5b968df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f70480056f0 tx=0x7f7048009680 comp rx=0 tx=0).stop 2024-04-25T00:26:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 -- 172.21.15.45:0/3670435115 shutdown_connections 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b969980 0x561e5b968df0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95f2b0 0x561e5b9552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 --2- 172.21.15.45:0/3670435115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b95e990 0x561e5b975630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 -- 172.21.15.45:0/3670435115 >> 172.21.15.45:0/3670435115 conn(0x561e5b95acb0 msgr2=0x561e5b960970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 -- 172.21.15.45:0/3670435115 shutdown_connections 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.225+0000 7f705b44f740 1 -- 172.21.15.45:0/3670435115 wait complete. 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 -- start start 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 Processor -- start 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b971d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b963360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70501adca0 con 0x561e5b95f2b0 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70501ad5d0 con 0x561e5b95e990 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70501aa810 con 0x561e5b969980 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f705b44f740 10 client.0 _ll_register_callbacks cb 0x7f70480148e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46540/0 (socket says 172.21.15.45:46540) 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 -- 172.21.15.45:0/1734860715 learned_addr learned my addr 172.21.15.45:0/1734860715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f70564de640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b971d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7056cdf640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b963360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 -- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 msgr2=0x561e5b963360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b963360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 -- 172.21.15.45:0/1734860715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 msgr2=0x561e5b971d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b971d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7055cdd640 1 -- 172.21.15.45:0/1734860715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5b69d7d0 con 0x561e5b95f2b0 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f70564de640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b971d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.226+0000 7f7056cdf640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b963360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7055cdd640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f704002aa10 tx=0x7f7040005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3962b69c5096f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040035070 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f70400046c0 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7048023070 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7040005d50 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 10 client.17037 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e5b9df920 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048023070 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7040042020 con 0x561e5b95f2b0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 10 client.17037 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 10 client.17037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 20 client.17037 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b9e00d0 0x561e5b9e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 4 client.17037 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.227+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e5b8c6f00 con 0x561e5b9e00d0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f70564de640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b9e00d0 0x561e5b9e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f70564de640 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b9e00d0 0x561e5b9e24f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab7d336f2385682e server_cookie=93fd095bc2f12bf0 in_seq=0 out_seq=0 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f7039ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f7039ffb640 1 -- 172.21.15.45:0/1734860715 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e5b8c6f00 con 0x561e5b9e00d0 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f7039ffb640 10 client.17037 handle_command_reply: tid=0 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 client.17037 shutdown 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 msgr2=0x561e5b962e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f704002aa10 tx=0x7f7040005270 comp rx=0 tx=0).stop 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 shutdown_connections 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561e5b9e00d0 0x561e5b9e24f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e5b969980 0x561e5b963360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e5b95f2b0 0x561e5b962e20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f7039ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 --2- 172.21.15.45:0/1734860715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e5b95e990 0x561e5b971d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 >> 172.21.15.45:0/1734860715 conn(0x561e5b95acb0 msgr2=0x561e5b9600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 shutdown_connections 2024-04-25T00:26:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.228+0000 7f705b44f740 1 -- 172.21.15.45:0/1734860715 wait complete. 2024-04-25T00:26:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f705b44f740 20 client.17037 trim_cache size 0 max 16384 2024-04-25T00:26:55.237 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 msgr2=0x7f703c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 0x7f703c0a5240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7048022000 tx=0x7f7048020000 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 msgr2=0x7f7050198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f704c00d950 tx=0x7f704c00de10 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 shutdown_connections 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f703c0a2d80 0x7f703c0a5240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70501066b0 0x7f705019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7050100fd0 0x7f7050198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 --2- 172.21.15.45:0/4214579179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7050100610 0x7f7050198000 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 >> 172.21.15.45:0/4214579179 conn(0x7f7050077740 msgr2=0x7f70500ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 shutdown_connections 2024-04-25T00:26:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.229+0000 7f7058742640 1 -- 172.21.15.45:0/4214579179 wait complete. 2024-04-25T00:26:55.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:26:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 -- 172.21.15.45:0/3131338156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 msgr2=0x7fc35c06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 --2- 172.21.15.45:0/3131338156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c06a240 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc350009930 tx=0x7fc35002a420 comp rx=0 tx=0).stop 2024-04-25T00:26:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 -- 172.21.15.45:0/3131338156 shutdown_connections 2024-04-25T00:26:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 --2- 172.21.15.45:0/3131338156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 0x7fc35c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 --2- 172.21.15.45:0/3131338156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c06a240 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 --2- 172.21.15.45:0/3131338156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 0x7fc35c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 -- 172.21.15.45:0/3131338156 >> 172.21.15.45:0/3131338156 conn(0x7fc35c0776a0 msgr2=0x7fc35c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 -- 172.21.15.45:0/3131338156 shutdown_connections 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.343+0000 7fc361bdb640 1 -- 172.21.15.45:0/3131338156 wait complete. 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 Processor -- start 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 -- start start 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 0x7fc35c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 0x7fc35c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111b40 con 0x7fc35c069dc0 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1119c0 con 0x7fc35c06a780 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc361bdb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111cc0 con 0x7fc35c107730 2024-04-25T00:26:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46558/0 (socket says 172.21.15.45:46558) 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc35b7fe640 1 -- 172.21.15.45:0/2574225920 learned_addr learned my addr 172.21.15.45:0/2574225920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.344+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 0x7fc35c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35affd640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 0x7fc35c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 -- 172.21.15.45:0/2574225920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 msgr2=0x7fc35c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 0x7fc35c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 -- 172.21.15.45:0/2574225920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 msgr2=0x7fc35c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 0x7fc35c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 -- 172.21.15.45:0/2574225920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c19cff0 con 0x7fc35c069dc0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea80 tx=0x7fc34c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc358ff9640 1 -- 172.21.15.45:0/2574225920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00ce40 con 0x7fc35c069dc0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c19d2e0 con 0x7fc35c069dc0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc358ff9640 1 -- 172.21.15.45:0/2574225920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c005020 con 0x7fc35c069dc0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc358ff9640 1 -- 172.21.15.45:0/2574225920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c011760 con 0x7fc35c069dc0 2024-04-25T00:26:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.345+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a4b70 con 0x7fc35c069dc0 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.347+0000 7fc358ff9640 1 -- 172.21.15.45:0/2574225920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc34c0051d0 con 0x7fc35c069dc0 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.347+0000 7fc3649bd740 1 Processor -- start 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.347+0000 7fc3649bd740 1 -- start start 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562bce11d670 con 0x562bce1b8580 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bce0c5970 con 0x562bce1c3570 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc3649bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bce119630 con 0x562bce1b8ea0 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46566/0 (socket says 172.21.15.45:46566) 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 -- 172.21.15.45:0/3519202419 learned_addr learned my addr 172.21.15.45:0/3519202419 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc358ff9640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 0x7fc3440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35affd640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 -- 172.21.15.45:0/3519202419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 msgr2=0x562bce1c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 -- 172.21.15.45:0/3519202419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 msgr2=0x562bce1c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 -- 172.21.15.45:0/3519202419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bce0e9400 con 0x562bce1b8580 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35affd640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 0x7fc3440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc358ff9640 1 -- 172.21.15.45:0/2574225920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc34c0cfdc0 con 0x7fc35c069dc0 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.348+0000 7fc35bfff640 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc35400b7a0 tx=0x7fc35400bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541b450044329084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc341ffb640 1 -- 172.21.15.45:0/3519202419 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354024070 con 0x562bce1b8580 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc341ffb640 1 -- 172.21.15.45:0/3519202419 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35401eb60 con 0x562bce1b8580 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc341ffb640 1 -- 172.21.15.45:0/3519202419 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540118a0 con 0x562bce1b8580 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 0x7fc3440a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc354021000 tx=0x7fc35401e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 -- 172.21.15.45:0/3519202419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 msgr2=0x562bce1b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc35400b7a0 tx=0x7fc35400bc60 comp rx=0 tx=0).stop 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 -- 172.21.15.45:0/3519202419 shutdown_connections 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 --2- 172.21.15.45:0/3519202419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1b8960 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 -- 172.21.15.45:0/3519202419 >> 172.21.15.45:0/3519202419 conn(0x562bce1b48a0 msgr2=0x562bce1b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 -- 172.21.15.45:0/3519202419 shutdown_connections 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.349+0000 7fc3649bd740 1 -- 172.21.15.45:0/3519202419 wait complete. 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 -- start start 2024-04-25T00:26:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 Processor -- start 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1b8580 0x562bce1c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562bce1c6360 con 0x562bce1c3570 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562bce0d9050 con 0x562bce1b8580 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bce1c64e0 con 0x562bce1b8ea0 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc3649bd740 10 client.0 _ll_register_callbacks cb 0x562bcde30be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46580/0 (socket says 172.21.15.45:46580) 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.350+0000 7fc35bfff640 1 -- 172.21.15.45:0/2982421149 learned_addr learned my addr 172.21.15.45:0/2982421149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1b8580 0x562bce1c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35affd640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 -- 172.21.15.45:0/2982421149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 msgr2=0x562bce1c0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 -- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1b8580 msgr2=0x562bce1c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1b8580 0x562bce1c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 -- 172.21.15.45:0/2982421149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bce0aadc0 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc35bfff640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc354013e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d0e0b1dd5cc1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc33affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354024040 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc354011df0 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35401eb60 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc354037050 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc3649bd740 10 client.17049 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bcde313f0 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.351+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35401eb60 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc354030060 con 0x562bce1c3570 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 10 client.17049 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 10 client.17049 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:26:55.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 20 client.17049 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce1d3510 0x562bce1d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 4 client.17049 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bce12abb0 con 0x562bce1d3510 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce1d3510 0x562bce1d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc33affd640 10 client.17049 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.352+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce1d3510 0x562bce1d5900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9bb3126f2434cdfc server_cookie=54d3b51cdae18efe in_seq=0 out_seq=0 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc33affd640 1 -- 172.21.15.45:0/2982421149 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bce12abb0 con 0x562bce1d3510 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc33affd640 10 client.17049 handle_command_reply: tid=0 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 client.17049 shutdown 2024-04-25T00:26:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 msgr2=0x562bce1c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc354013e80 comp rx=0 tx=0).stop 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 shutdown_connections 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce1d3510 0x562bce1d5900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1c3570 0x562bce1c07a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc33affd640 0 client.17049 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 --2- 172.21.15.45:0/2982421149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1b8580 0x562bce1c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 >> 172.21.15.45:0/2982421149 conn(0x562bce1b48a0 msgr2=0x562bce1ba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 shutdown_connections 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 1 -- 172.21.15.45:0/2982421149 wait complete. 2024-04-25T00:26:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.353+0000 7fc3649bd740 20 client.17049 trim_cache size 0 max 16384 2024-04-25T00:26:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 Processor -- start 2024-04-25T00:26:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 -- start start 2024-04-25T00:26:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1aeb10 con 0x562bce1b8580 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b0650 con 0x562bce1c3570 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3649bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1af1e0 con 0x562bce1b8ea0 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46584/0 (socket says 172.21.15.45:46584) 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 -- 172.21.15.45:0/1407402759 learned_addr learned my addr 172.21.15.45:0/1407402759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 -- 172.21.15.45:0/1407402759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 msgr2=0x562bce1c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35affd640 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 -- 172.21.15.45:0/1407402759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 msgr2=0x562bce1db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 -- 172.21.15.45:0/1407402759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bcdefb7d0 con 0x562bce1b8580 2024-04-25T00:26:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35affd640 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0cd1b0 tx=0x7fc34c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e444060ecb11ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.367+0000 7fc3417fa640 1 -- 172.21.15.45:0/1407402759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01f040 con 0x562bce1b8580 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3417fa640 1 -- 172.21.15.45:0/1407402759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c09bf20 con 0x562bce1b8580 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3417fa640 1 -- 172.21.15.45:0/1407402759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0cd3d0 con 0x562bce1b8580 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- 172.21.15.45:0/1407402759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 msgr2=0x562bce1ba4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0cd1b0 tx=0x7fc34c0bc040 comp rx=0 tx=0).stop 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- 172.21.15.45:0/1407402759 shutdown_connections 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 --2- 172.21.15.45:0/1407402759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1ba4b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- 172.21.15.45:0/1407402759 >> 172.21.15.45:0/1407402759 conn(0x562bce1b48a0 msgr2=0x562bce1c0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- 172.21.15.45:0/1407402759 shutdown_connections 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- 172.21.15.45:0/1407402759 wait complete. 2024-04-25T00:26:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.368+0000 7fc3649bd740 1 -- start start 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 Processor -- start 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1aff80 con 0x562bce1b8580 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1af8b0 con 0x562bce1c3570 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1acaf0 con 0x562bce1b8ea0 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc3649bd740 10 client.0 _ll_register_callbacks cb 0x7fc354022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46600/0 (socket says 172.21.15.45:46600) 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 -- 172.21.15.45:0/234772201 learned_addr learned my addr 172.21.15.45:0/234772201 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35affd640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35bfff640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 -- 172.21.15.45:0/234772201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 msgr2=0x562bce1c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 -- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 msgr2=0x562bce1c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35b7fe640 1 -- 172.21.15.45:0/234772201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bce1c79e0 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.369+0000 7fc35affd640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc35b7fe640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00efd0 tx=0x7fc34c0bc120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99312e91f30e94e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01f040 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc34c0cd3d0 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34c09bf20 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc34c0a4b90 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 10 client.17055 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562bce2397c0 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c09bf20 con 0x562bce1b8580 2024-04-25T00:26:55.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc34c0d3020 con 0x562bce1b8580 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 10 client.17055 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 10 client.17055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 20 client.17055 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce239f00 0x562bce23c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 4 client.17055 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bce124f00 con 0x562bce239f00 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.370+0000 7fc35affd640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce239f00 0x562bce23c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc33affd640 10 client.17055 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc35affd640 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce239f00 0x562bce23c320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a33b836ea38b79c server_cookie=7e1b8d9b0d6f4b0d in_seq=0 out_seq=0 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc33affd640 1 -- 172.21.15.45:0/234772201 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bce124f00 con 0x562bce239f00 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc33affd640 10 client.17055 handle_command_reply: tid=0 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 client.17055 shutdown 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 msgr2=0x562bce1cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00efd0 tx=0x7fc34c0bc120 comp rx=0 tx=0).stop 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 shutdown_connections 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562bce239f00 0x562bce23c320 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bce1c3570 0x562bce1c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bce1b8ea0 0x562bce1c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 --2- 172.21.15.45:0/234772201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562bce1b8580 0x562bce1cb9f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc33affd640 0 client.17055 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 >> 172.21.15.45:0/234772201 conn(0x562bce1b48a0 msgr2=0x562bce1b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 shutdown_connections 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 1 -- 172.21.15.45:0/234772201 wait complete. 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.371+0000 7fc3649bd740 20 client.17055 trim_cache size 0 max 16384 2024-04-25T00:26:55.379 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 msgr2=0x7fc3440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 0x7fc3440a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc354021000 tx=0x7fc35401e000 comp rx=0 tx=0).stop 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 msgr2=0x7fc35c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ea80 tx=0x7fc34c00ef40 comp rx=0 tx=0).stop 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 shutdown_connections 2024-04-25T00:26:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3440a2d00 0x7fc3440a51c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc35c107730 0x7fc35c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc35c06a780 0x7fc35c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 --2- 172.21.15.45:0/2574225920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc35c069dc0 0x7fc35c198000 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 >> 172.21.15.45:0/2574225920 conn(0x7fc35c0776a0 msgr2=0x7fc35c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 shutdown_connections 2024-04-25T00:26:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.372+0000 7fc361bdb640 1 -- 172.21.15.45:0/2574225920 wait complete. 2024-04-25T00:26:55.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:26:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 -- 172.21.15.45:0/2588214521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1032e0 msgr2=0x7ffa0c10b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 --2- 172.21.15.45:0/2588214521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c10b7c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009900 tx=0x7ffa0402a510 comp rx=0 tx=0).stop 2024-04-25T00:26:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 -- 172.21.15.45:0/2588214521 shutdown_connections 2024-04-25T00:26:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 --2- 172.21.15.45:0/2588214521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c10bd00 0x7ffa0c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 --2- 172.21.15.45:0/2588214521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c10b7c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 --2- 172.21.15.45:0/2588214521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 -- 172.21.15.45:0/2588214521 >> 172.21.15.45:0/2588214521 conn(0x7ffa0c077740 msgr2=0x7ffa0c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 -- 172.21.15.45:0/2588214521 shutdown_connections 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.488+0000 7ffa14464640 1 -- 172.21.15.45:0/2588214521 wait complete. 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 Processor -- start 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 -- start start 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa0c10bd00 0x7ffa0c101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c111b50 con 0x7ffa0c1029c0 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1119d0 con 0x7ffa0c1032e0 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa14464640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c111cd0 con 0x7ffa0c10bd00 2024-04-25T00:26:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46618/0 (socket says 172.21.15.45:46618) 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 -- 172.21.15.45:0/3403609389 learned_addr learned my addr 172.21.15.45:0/3403609389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa119ff640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 -- 172.21.15.45:0/3403609389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa0c10bd00 msgr2=0x7ffa0c101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa0c10bd00 0x7ffa0c101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 -- 172.21.15.45:0/3403609389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 msgr2=0x7ffa0c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa12200640 1 -- 172.21.15.45:0/3403609389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c1a6530 con 0x7ffa0c1029c0 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.489+0000 7ffa119ff640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa12200640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00db80 tx=0x7ff9fc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa0b7fe640 1 -- 172.21.15.45:0/3403609389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc017070 con 0x7ffa0c1029c0 2024-04-25T00:26:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa0b7fe640 1 -- 172.21.15.45:0/3403609389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc003b70 con 0x7ffa0c1029c0 2024-04-25T00:26:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa0b7fe640 1 -- 172.21.15.45:0/3403609389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0106c0 con 0x7ffa0c1029c0 2024-04-25T00:26:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c1a6820 con 0x7ffa0c1029c0 2024-04-25T00:26:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.490+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c1a6d20 con 0x7ffa0c1029c0 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.492+0000 7ffa0b7fe640 1 -- 172.21.15.45:0/3403609389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff9fc004020 con 0x7ffa0c1029c0 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.492+0000 7ffa17171740 1 Processor -- start 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 -- start start 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa0b7fe640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 0x7ff9f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bf9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e35b51670 con 0x564e35becea0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564e35af9970 con 0x564e35bf7570 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa17171740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564e35b4d630 con 0x564e35bec580 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa12a01640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 0x7ff9f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46628/0 (socket says 172.21.15.45:46628) 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 -- 172.21.15.45:0/2014669829 learned_addr learned my addr 172.21.15.45:0/2014669829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa0b7fe640 1 -- 172.21.15.45:0/3403609389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9fc0cf630 con 0x7ffa0c1029c0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa12200640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa12a01640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bf9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 -- 172.21.15.45:0/2014669829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 msgr2=0x564e35bec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 -- 172.21.15.45:0/2014669829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 msgr2=0x564e35bf9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bf9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa119ff640 1 -- 172.21.15.45:0/2014669829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e35b1d400 con 0x564e35becea0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.493+0000 7ffa12200640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.494+0000 7ffa12a01640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 0x7ff9f80a51f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400acc0 tx=0x7ff9f40137d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.494+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ffa0402aa30 tx=0x7ffa04004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcd62eee2ffa6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.494+0000 7ffa08bfe640 1 -- 172.21.15.45:0/2014669829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403f070 con 0x564e35becea0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.494+0000 7ffa08bfe640 1 -- 172.21.15.45:0/2014669829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04005520 con 0x564e35becea0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.494+0000 7ffa08bfe640 1 -- 172.21.15.45:0/2014669829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035030 con 0x564e35becea0 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- 172.21.15.45:0/2014669829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 msgr2=0x564e35bf7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ffa0402aa30 tx=0x7ffa04004170 comp rx=0 tx=0).stop 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- 172.21.15.45:0/2014669829 shutdown_connections 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bf9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35bf7030 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- 172.21.15.45:0/2014669829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- 172.21.15.45:0/2014669829 >> 172.21.15.45:0/2014669829 conn(0x564e35be88a0 msgr2=0x564e35be8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- 172.21.15.45:0/2014669829 shutdown_connections 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- 172.21.15.45:0/2014669829 wait complete. 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- start start 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 Processor -- start 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bf6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35becea0 0x564e35bf41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e35bfa300 con 0x564e35bf7570 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564e35b0d050 con 0x564e35becea0 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.495+0000 7ffa17171740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564e35bfa480 con 0x564e35bec580 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa17171740 10 client.0 _ll_register_callbacks cb 0x564e35864be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46638/0 (socket says 172.21.15.45:46638) 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 -- 172.21.15.45:0/1555906802 learned_addr learned my addr 172.21.15.45:0/1555906802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa119ff640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35becea0 0x564e35bf41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12200640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bf6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 -- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 msgr2=0x564e35bf6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bf6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 -- 172.21.15.45:0/1555906802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35becea0 msgr2=0x564e35bf41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35becea0 0x564e35bf41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 -- 172.21.15.45:0/1555906802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e35adedc0 con 0x564e35bf7570 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12200640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bf6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa12a01640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4010ac0 tx=0x7ff9f40058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fb8d7ad420a502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4023030 con 0x564e35bf7570 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9f4011b70 con 0x564e35bf7570 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa04005520 con 0x564e35bf7570 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9f4030050 con 0x564e35bf7570 2024-04-25T00:26:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa17171740 10 client.17067 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.496+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e358653f0 con 0x564e35bf7570 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04005520 con 0x564e35bf7570 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9f401c050 con 0x564e35bf7570 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 20 client.17067 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 4 client.17067 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e35b5ebb0 con 0x564e35c07620 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.497+0000 7ffa12200640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.498+0000 7ffa01ffb640 10 client.17067 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.498+0000 7ffa12200640 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c09a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a55f552316223bf0 server_cookie=4d3912c0871942f4 in_seq=0 out_seq=0 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.498+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1555906802 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e35b5ebb0 con 0x564e35c07620 2024-04-25T00:26:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.498+0000 7ffa01ffb640 10 client.17067 handle_command_reply: tid=0 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.498+0000 7ffa17171740 1 client.17067 shutdown 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 msgr2=0x564e35bf4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4010ac0 tx=0x7ff9f40058c0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 shutdown_connections 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c09a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa01ffb640 0 client.17067 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bf7570 0x564e35bf4770 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35becea0 0x564e35bf41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 --2- 172.21.15.45:0/1555906802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35bf6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 >> 172.21.15.45:0/1555906802 conn(0x564e35be88a0 msgr2=0x564e35bee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 shutdown_connections 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 1 -- 172.21.15.45:0/1555906802 wait complete. 2024-04-25T00:26:55.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.499+0000 7ffa17171740 20 client.17067 trim_cache size 0 max 16384 2024-04-25T00:26:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 Processor -- start 2024-04-25T00:26:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 -- start start 2024-04-25T00:26:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35c031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35c0f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1b0d90 con 0x564e35becea0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1b28d0 con 0x564e35bf7570 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa17171740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1b1460 con 0x564e35bec580 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa119ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa119ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46648/0 (socket says 172.21.15.45:46648) 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.512+0000 7ffa119ff640 1 -- 172.21.15.45:0/2499322660 learned_addr learned my addr 172.21.15.45:0/2499322660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa12a01640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35c0f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa12200640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35c031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 -- 172.21.15.45:0/2499322660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 msgr2=0x564e35c031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35c031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 -- 172.21.15.45:0/2499322660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 msgr2=0x564e35c0f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35c0f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 -- 172.21.15.45:0/2499322660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e35bf31a0 con 0x564e35becea0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa12a01640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35c0f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa119ff640 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa040058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf8155208a084a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa03fff640 1 -- 172.21.15.45:0/2499322660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403f050 con 0x564e35becea0 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa03fff640 1 -- 172.21.15.45:0/2499322660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa040055e0 con 0x564e35becea0 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.513+0000 7ffa03fff640 1 -- 172.21.15.45:0/2499322660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035040 con 0x564e35becea0 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- 172.21.15.45:0/2499322660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 msgr2=0x564e35c0f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa040058d0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- 172.21.15.45:0/2499322660 shutdown_connections 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35c0f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35becea0 0x564e35c0f340 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- 172.21.15.45:0/2499322660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35bec580 0x564e35c031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- 172.21.15.45:0/2499322660 >> 172.21.15.45:0/2499322660 conn(0x564e35be88a0 msgr2=0x564e35be7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- 172.21.15.45:0/2499322660 shutdown_connections 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- 172.21.15.45:0/2499322660 wait complete. 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- start start 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 Processor -- start 2024-04-25T00:26:55.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 0x564e35bfd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bfdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1b2200 con 0x564e35bec580 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1b1b30 con 0x564e35bf7570 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c1aed70 con 0x564e35becea0 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa17171740 10 client.0 _ll_register_callbacks cb 0x7ff9f40107c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46662/0 (socket says 172.21.15.45:46662) 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.514+0000 7ffa12200640 1 -- 172.21.15.45:0/1527684700 learned_addr learned my addr 172.21.15.45:0/1527684700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 -- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 msgr2=0x564e35bfd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12a01640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bfdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa119ff640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 0x564e35bfd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 0x564e35bfd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 -- 172.21.15.45:0/1527684700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 msgr2=0x564e35bfdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bfdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 -- 172.21.15.45:0/1527684700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e3592f7d0 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12a01640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bfdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa119ff640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 0x564e35bfd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa12200640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09b720 tx=0x7ff9fc097af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbd25f3af21570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0b4070 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9fc099a90 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa040055e0 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc09fb40 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 10 client.17073 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e35c6df40 con 0x564e35bec580 2024-04-25T00:26:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa040055e0 con 0x564e35bec580 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9fc0ccc40 con 0x564e35bec580 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 10 client.17073 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 20 client.17073 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.515+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 4 client.17073 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e35b58f00 con 0x564e35c07620 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa119ff640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa01ffb640 10 client.17073 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa119ff640 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c706d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8288d0c5be9a9bd server_cookie=215eb2d01ae0c5a7 in_seq=0 out_seq=0 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa01ffb640 1 -- 172.21.15.45:0/1527684700 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e35b58f00 con 0x564e35c07620 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa01ffb640 10 client.17073 handle_command_reply: tid=0 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 client.17073 shutdown 2024-04-25T00:26:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 msgr2=0x564e35bfd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09b720 tx=0x7ff9fc097af0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 shutdown_connections 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x564e35c07620 0x564e35c706d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e35bf7570 0x564e35bfdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564e35becea0 0x564e35bfd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa01ffb640 0 client.17073 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 --2- 172.21.15.45:0/1527684700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e35bec580 0x564e35bfd310 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.516+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 >> 172.21.15.45:0/1527684700 conn(0x564e35be88a0 msgr2=0x564e35c021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 shutdown_connections 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa17171740 1 -- 172.21.15.45:0/1527684700 wait complete. 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa17171740 20 client.17073 trim_cache size 0 max 16384 2024-04-25T00:26:55.524 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 msgr2=0x7ff9f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 0x7ff9f80a51f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400acc0 tx=0x7ff9f40137d0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 msgr2=0x7ffa0c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00db80 tx=0x7ff9fc00b630 comp rx=0 tx=0).stop 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 shutdown_connections 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9f80a2d30 0x7ff9f80a51f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa0c10bd00 0x7ffa0c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0c1032e0 0x7ffa0c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 --2- 172.21.15.45:0/3403609389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa0c1029c0 0x7ffa0c100610 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 >> 172.21.15.45:0/3403609389 conn(0x7ffa0c077740 msgr2=0x7ffa0c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 shutdown_connections 2024-04-25T00:26:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.517+0000 7ffa14464640 1 -- 172.21.15.45:0/3403609389 wait complete. 2024-04-25T00:26:55.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 -- 172.21.15.45:0/1270590475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 msgr2=0x7f60b0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 --2- 172.21.15.45:0/1270590475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b0107b10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009cd0 tx=0x7f60ac02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 -- 172.21.15.45:0/1270590475 shutdown_connections 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 --2- 172.21.15.45:0/1270590475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b006a780 0x7f60b010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 --2- 172.21.15.45:0/1270590475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b0069dc0 0x7f60b006a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 --2- 172.21.15.45:0/1270590475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b0107b10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 -- 172.21.15.45:0/1270590475 >> 172.21.15.45:0/1270590475 conn(0x7f60b00776a0 msgr2=0x7f60b0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 -- 172.21.15.45:0/1270590475 shutdown_connections 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.633+0000 7f60b819d640 1 -- 172.21.15.45:0/1270590475 wait complete. 2024-04-25T00:26:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 Processor -- start 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 -- start start 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 0x7f60b0100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b006a780 0x7f60b0100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0111b00 con 0x7f60b0107730 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0111980 con 0x7f60b006a780 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b819d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0111c80 con 0x7f60b0069dc0 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46674/0 (socket says 172.21.15.45:46674) 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 -- 172.21.15.45:0/3713224011 learned_addr learned my addr 172.21.15.45:0/3713224011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b5738640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b006a780 0x7f60b0100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b5f39640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 0x7f60b0100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 -- 172.21.15.45:0/3713224011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 msgr2=0x7f60b0100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 0x7f60b0100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 -- 172.21.15.45:0/3713224011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b006a780 msgr2=0x7f60b0100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b006a780 0x7f60b0100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b673a640 1 -- 172.21.15.45:0/3713224011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01a6540 con 0x7f60b0107730 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.634+0000 7f60b5f39640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 0x7f60b0100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60b673a640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60a6ffd640 1 -- 172.21.15.45:0/3713224011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8017030 con 0x7f60b0107730 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a6830 con 0x7f60b0107730 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60a6ffd640 1 -- 172.21.15.45:0/3713224011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a8004020 con 0x7f60b0107730 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60a6ffd640 1 -- 172.21.15.45:0/3713224011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8003c00 con 0x7f60b0107730 2024-04-25T00:26:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.635+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01a6d30 con 0x7f60b0107730 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.637+0000 7f60a6ffd640 1 -- 172.21.15.45:0/3713224011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60a801a070 con 0x7f60b0107730 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.637+0000 7f60baf81740 1 Processor -- start 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.637+0000 7f60baf81740 1 -- start start 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 0x563cf1308030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf130a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563cf1262670 con 0x563cf12fd580 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563cf120a970 con 0x563cf1308570 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60baf81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563cf125e630 con 0x563cf12fdea0 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46682/0 (socket says 172.21.15.45:46682) 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 -- 172.21.15.45:0/1312217172 learned_addr learned my addr 172.21.15.45:0/1312217172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 0x563cf1308030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf130a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60a6ffd640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 0x7f609c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 -- 172.21.15.45:0/1312217172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 msgr2=0x563cf1308030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 0x563cf1308030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 -- 172.21.15.45:0/1312217172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 msgr2=0x563cf130a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf130a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 -- 172.21.15.45:0/1312217172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf122e400 con 0x563cf12fd580 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 0x563cf1308030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b673a640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 0x7f609c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60b5738640 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f60a0013580 tx=0x7f60a000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57794f80dcc15e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f60a6ffd640 1 -- 172.21.15.45:0/3713224011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60a80cefc0 con 0x7f60b0107730 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f609bfff640 1 -- 172.21.15.45:0/1312217172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0019030 con 0x563cf12fd580 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f609bfff640 1 -- 172.21.15.45:0/1312217172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a0005500 con 0x563cf12fd580 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.638+0000 7f609bfff640 1 -- 172.21.15.45:0/1312217172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000f890 con 0x563cf12fd580 2024-04-25T00:26:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 -- 172.21.15.45:0/1312217172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 msgr2=0x563cf12fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60b673a640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 0x7f609c0a51f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097b50 tx=0x7f60a80974a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f60a0013580 tx=0x7f60a000a760 comp rx=0 tx=0).stop 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 -- 172.21.15.45:0/1312217172 shutdown_connections 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf130a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fdea0 0x563cf1308030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 --2- 172.21.15.45:0/1312217172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf12fd960 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 -- 172.21.15.45:0/1312217172 >> 172.21.15.45:0/1312217172 conn(0x563cf12f98a0 msgr2=0x563cf12f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 -- 172.21.15.45:0/1312217172 shutdown_connections 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.639+0000 7f60baf81740 1 -- 172.21.15.45:0/1312217172 wait complete. 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 -- start start 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 Processor -- start 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf13050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1305610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563cf130b360 con 0x563cf12fd580 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563cf121e050 con 0x563cf12fdea0 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563cf130b4e0 con 0x563cf1308570 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60baf81740 10 client.0 _ll_register_callbacks cb 0x563cf0f75be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46686/0 (socket says 172.21.15.45:46686) 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 -- 172.21.15.45:0/1330883089 learned_addr learned my addr 172.21.15.45:0/1330883089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf13050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1305610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 -- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 msgr2=0x563cf1305610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1305610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 -- 172.21.15.45:0/1330883089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 msgr2=0x563cf13050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf13050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.640+0000 7f60b5738640 1 -- 172.21.15.45:0/1330883089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf11efdc0 con 0x563cf12fd580 2024-04-25T00:26:55.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60b5738640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f60a000a310 tx=0x7f60a0004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b4e4ecd570e582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0019040 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60a0004020 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a0005500 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60a0004220 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 10 client.17085 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563cf0f763f0 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a0005500 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60a0003710 con 0x563cf12fd580 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 10 client.17085 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 20 client.17085 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf1318400 0x563cf131a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 4 client.17085 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.641+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf126fbb0 con 0x563cf1318400 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.642+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf1318400 0x563cf131a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.642+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf1318400 0x563cf131a7f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e30c333edfdd2b7 server_cookie=70b46c1e677c982b in_seq=0 out_seq=0 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.642+0000 7f60997fa640 10 client.17085 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60997fa640 1 -- 172.21.15.45:0/1330883089 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563cf126fbb0 con 0x563cf1318400 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60997fa640 10 client.17085 handle_command_reply: tid=0 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 client.17085 shutdown 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 msgr2=0x563cf1307de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f60a000a310 tx=0x7f60a0004c90 comp rx=0 tx=0).stop 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 shutdown_connections 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf1318400 0x563cf131a7f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1305610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf13050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 --2- 172.21.15.45:0/1330883089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1307de0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60997fa640 0 client.17085 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 >> 172.21.15.45:0/1330883089 conn(0x563cf12f98a0 msgr2=0x563cf12ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 shutdown_connections 2024-04-25T00:26:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 1 -- 172.21.15.45:0/1330883089 wait complete. 2024-04-25T00:26:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.643+0000 7f60baf81740 20 client.17085 trim_cache size 0 max 16384 2024-04-25T00:26:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.656+0000 7f60baf81740 1 Processor -- start 2024-04-25T00:26:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.656+0000 7f60baf81740 1 -- start start 2024-04-25T00:26:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fd580 0x563cf13049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf131ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b0ce0 con 0x563cf12fdea0 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b2820 con 0x563cf1308570 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60baf81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b13b0 con 0x563cf12fd580 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46690/0 (socket says 172.21.15.45:46690) 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 -- 172.21.15.45:0/1244616389 learned_addr learned my addr 172.21.15.45:0/1244616389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5738640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fd580 0x563cf13049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5738640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf131ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 -- 172.21.15.45:0/1244616389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fd580 msgr2=0x563cf13049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fd580 0x563cf13049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 -- 172.21.15.45:0/1244616389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 msgr2=0x563cf131ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf131ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 -- 172.21.15.45:0/1244616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf10407d0 con 0x563cf12fdea0 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5738640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf131ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f60b5f39640 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f60ac02ae00 tx=0x7f60ac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d58413fe508b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f609b7fe640 1 -- 172.21.15.45:0/1244616389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0318c0 con 0x563cf12fdea0 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f609b7fe640 1 -- 172.21.15.45:0/1244616389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ac036040 con 0x563cf12fdea0 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.657+0000 7f609b7fe640 1 -- 172.21.15.45:0/1244616389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0045a0 con 0x563cf12fdea0 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- 172.21.15.45:0/1244616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 msgr2=0x563cf1307910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f60ac02ae00 tx=0x7f60ac009920 comp rx=0 tx=0).stop 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- 172.21.15.45:0/1244616389 shutdown_connections 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf1308570 0x563cf131ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fdea0 0x563cf1307910 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 --2- 172.21.15.45:0/1244616389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf12fd580 0x563cf13049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- 172.21.15.45:0/1244616389 >> 172.21.15.45:0/1244616389 conn(0x563cf12f98a0 msgr2=0x563cf12ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- 172.21.15.45:0/1244616389 shutdown_connections 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- 172.21.15.45:0/1244616389 wait complete. 2024-04-25T00:26:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 -- start start 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.658+0000 7f60baf81740 1 Processor -- start 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf1310c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1301770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b2150 con 0x563cf12fd580 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b1a80 con 0x563cf12fdea0 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01aecc0 con 0x563cf1308570 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60baf81740 10 client.0 _ll_register_callbacks cb 0x7f60a000abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46694/0 (socket says 172.21.15.45:46694) 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 -- 172.21.15.45:0/3751006335 learned_addr learned my addr 172.21.15.45:0/3751006335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5f39640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf1310c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 -- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 msgr2=0x563cf1301770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1301770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 -- 172.21.15.45:0/3751006335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 msgr2=0x563cf1310c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf1310c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 -- 172.21.15.45:0/3751006335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf1301fa0 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60b5738640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f60a0003b60 tx=0x7f60a0004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5d436c7678d24b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.659+0000 7f60997fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0039020 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f60a000a720 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ac036040 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60a0019040 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 10 client.17091 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563cf137e1b0 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ac036040 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60a000f890 con 0x563cf12fd580 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 10 client.17091 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 20 client.17091 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf137e960 0x563cf1380d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 4 client.17091 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.660+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cf1269f00 con 0x563cf137e960 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60b5f39640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf137e960 0x563cf1380d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60997fa640 10 client.17091 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60b5f39640 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf137e960 0x563cf1380d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec0e1b0ff3f0c6e5 server_cookie=67471fd6bfdba9af in_seq=0 out_seq=0 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60997fa640 1 -- 172.21.15.45:0/3751006335 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cf1269f00 con 0x563cf137e960 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60997fa640 10 client.17091 handle_command_reply: tid=0 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 client.17091 shutdown 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 msgr2=0x563cf1310730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f60a0003b60 tx=0x7f60a0004bf0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 shutdown_connections 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x563cf137e960 0x563cf1380d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563cf1308570 0x563cf1301770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563cf12fdea0 0x563cf1310c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 --2- 172.21.15.45:0/3751006335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cf12fd580 0x563cf1310730 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60997fa640 0 client.17091 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 >> 172.21.15.45:0/3751006335 conn(0x563cf12f98a0 msgr2=0x563cf12fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 shutdown_connections 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.661+0000 7f60baf81740 1 -- 172.21.15.45:0/3751006335 wait complete. 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60baf81740 20 client.17091 trim_cache size 0 max 16384 2024-04-25T00:26:55.668 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 msgr2=0x7f609c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 0x7f609c0a51f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f60a8097b50 tx=0x7f60a80974a0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 msgr2=0x7f60b01010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800de10 comp rx=0 tx=0).stop 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 shutdown_connections 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f609c0a2d30 0x7f609c0a51f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60b0107730 0x7f60b01010f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b006a780 0x7f60b0100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 --2- 172.21.15.45:0/3713224011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60b0069dc0 0x7f60b0100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 >> 172.21.15.45:0/3713224011 conn(0x7f60b00776a0 msgr2=0x7f60b00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 shutdown_connections 2024-04-25T00:26:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.662+0000 7f60b819d640 1 -- 172.21.15.45:0/3713224011 wait complete. 2024-04-25T00:26:55.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 -- 172.21.15.45:0/1865479682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3670109930 msgr2=0x7f3670109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 --2- 172.21.15.45:0/1865479682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3670109930 0x7f3670109d10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f366c009930 tx=0x7f366c02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 -- 172.21.15.45:0/1865479682 shutdown_connections 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 --2- 172.21.15.45:0/1865479682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36701042e0 0x7f36701107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 --2- 172.21.15.45:0/1865479682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 0x7f3670103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 --2- 172.21.15.45:0/1865479682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3670109930 0x7f3670109d10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 -- 172.21.15.45:0/1865479682 >> 172.21.15.45:0/1865479682 conn(0x7f36700ff3b0 msgr2=0x7f36701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 -- 172.21.15.45:0/1865479682 shutdown_connections 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.777+0000 7f36780cd640 1 -- 172.21.15.45:0/1865479682 wait complete. 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 Processor -- start 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 -- start start 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 0x7f367019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 0x7f367019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3670113d40 con 0x7f36701042e0 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3670113bc0 con 0x7f3670103920 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f36780cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3670113ec0 con 0x7f3670109930 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46712/0 (socket says 172.21.15.45:46712) 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 -- 172.21.15.45:0/3408966272 learned_addr learned my addr 172.21.15.45:0/3408966272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675e69640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 0x7f367019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f367666a640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 0x7f367019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 -- 172.21.15.45:0/3408966272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 msgr2=0x7f367019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 0x7f367019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 -- 172.21.15.45:0/3408966272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 msgr2=0x7f367019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.778+0000 7f3675668640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 0x7f367019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f3675668640 1 -- 172.21.15.45:0/3408966272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367019f260 con 0x7f36701042e0 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f367666a640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 0x7f367019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f3675668640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f366000d950 tx=0x7f366000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f3666ffd640 1 -- 172.21.15.45:0/3408966272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660017070 con 0x7f36701042e0 2024-04-25T00:26:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f3666ffd640 1 -- 172.21.15.45:0/3408966272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3660009c30 con 0x7f36701042e0 2024-04-25T00:26:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367019f550 con 0x7f36701042e0 2024-04-25T00:26:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f3666ffd640 1 -- 172.21.15.45:0/3408966272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000f6b0 con 0x7f36701042e0 2024-04-25T00:26:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.779+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a6de0 con 0x7f36701042e0 2024-04-25T00:26:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.781+0000 7f3666ffd640 1 -- 172.21.15.45:0/3408966272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36600109d0 con 0x7f36701042e0 2024-04-25T00:26:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 Processor -- start 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 -- start start 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f917c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9189960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6f90e1670 con 0x55d6f917cea0 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d6f9089970 con 0x55d6f9187570 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367add9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6f90dd630 con 0x55d6f917c580 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3666ffd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 0x7f365c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46714/0 (socket says 172.21.15.45:46714) 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 -- 172.21.15.45:0/666565955 learned_addr learned my addr 172.21.15.45:0/666565955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367666a640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f917c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675668640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9189960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f367666a640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 0x7f365c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 -- 172.21.15.45:0/666565955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 msgr2=0x55d6f917c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f917c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3666ffd640 1 -- 172.21.15.45:0/3408966272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f36600cf4b0 con 0x7f36701042e0 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 -- 172.21.15.45:0/666565955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 msgr2=0x55d6f9189960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9189960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675e69640 1 -- 172.21.15.45:0/666565955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f90ad400 con 0x55d6f917cea0 2024-04-25T00:26:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.782+0000 7f3675668640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9189960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f3675e69640 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f366c02a940 tx=0x7f366c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dc97f995895217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f365bfff640 1 -- 172.21.15.45:0/666565955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c03f070 con 0x55d6f917cea0 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f365bfff640 1 -- 172.21.15.45:0/666565955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c004030 con 0x55d6f917cea0 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f367666a640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 0x7f365c0a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3668005c40 tx=0x7f3668009200 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f365bfff640 1 -- 172.21.15.45:0/666565955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c035070 con 0x55d6f917cea0 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f367add9740 1 -- 172.21.15.45:0/666565955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 msgr2=0x55d6f9187030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.783+0000 7f367add9740 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f366c02a940 tx=0x7f366c005e80 comp rx=0 tx=0).stop 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- 172.21.15.45:0/666565955 shutdown_connections 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9189960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9187030 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- 172.21.15.45:0/666565955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f917c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- 172.21.15.45:0/666565955 >> 172.21.15.45:0/666565955 conn(0x55d6f91788a0 msgr2=0x55d6f9178c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- 172.21.15.45:0/666565955 shutdown_connections 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- 172.21.15.45:0/666565955 wait complete. 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- start start 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 Processor -- start 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f9186ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9184720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6f918a300 con 0x55d6f917cea0 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d6f909d050 con 0x55d6f9187570 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6f918a480 con 0x55d6f917c580 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.784+0000 7f367add9740 10 client.0 _ll_register_callbacks cb 0x55d6f8df4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46726/0 (socket says 172.21.15.45:46726) 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 -- 172.21.15.45:0/1462016268 learned_addr learned my addr 172.21.15.45:0/1462016268 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675e69640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f9186ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f367666a640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9184720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 -- 172.21.15.45:0/1462016268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 msgr2=0x55d6f9186ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f9186ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 -- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 msgr2=0x55d6f9184720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9184720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 -- 172.21.15.45:0/1462016268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f906edc0 con 0x55d6f917cea0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f3675668640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f36600ccea0 tx=0x7f3660010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac27f9a5b850981e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f36597fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660020040 con 0x55d6f917cea0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f36600b4610 con 0x55d6f917cea0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f366c004030 con 0x55d6f917cea0 2024-04-25T00:26:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3660009de0 con 0x55d6f917cea0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f367add9740 10 client.17103 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6f8df53f0 con 0x55d6f917cea0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.785+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c004030 con 0x55d6f917cea0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3660097ce0 con 0x55d6f917cea0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 10 client.17103 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 20 client.17103 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f9197580 0x55d6f91999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 4 client.17103 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6f90eebb0 con 0x55d6f9197580 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f3675e69640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f9197580 0x55d6f91999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f36597fa640 10 client.17103 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.786+0000 7f3675e69640 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f9197580 0x55d6f91999b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c40738645c58d6 server_cookie=aeea18fdda77f408 in_seq=0 out_seq=0 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f36597fa640 1 -- 172.21.15.45:0/1462016268 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6f90eebb0 con 0x55d6f9197580 2024-04-25T00:26:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f36597fa640 10 client.17103 handle_command_reply: tid=0 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 client.17103 shutdown 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 msgr2=0x55d6f91841c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f36600ccea0 tx=0x7f3660010d70 comp rx=0 tx=0).stop 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 shutdown_connections 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f9184720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f9197580 0x55d6f91999b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f91841c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 --2- 172.21.15.45:0/1462016268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917c580 0x55d6f9186ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f36597fa640 0 client.17103 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 >> 172.21.15.45:0/1462016268 conn(0x55d6f91788a0 msgr2=0x55d6f9177b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 shutdown_connections 2024-04-25T00:26:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.787+0000 7f367add9740 1 -- 172.21.15.45:0/1462016268 wait complete. 2024-04-25T00:26:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.788+0000 7f367add9740 20 client.17103 trim_cache size 0 max 16384 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 Processor -- start 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 -- start start 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f917c580 0x55d6f9193270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f9187570 0x55d6f918f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36701b0ca0 con 0x55d6f917cea0 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36701b27e0 con 0x55d6f917c580 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f367add9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36701b1370 con 0x55d6f9187570 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46742/0 (socket says 172.21.15.45:46742) 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 -- 172.21.15.45:0/1509440737 learned_addr learned my addr 172.21.15.45:0/1509440737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675e69640 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f917c580 0x55d6f9193270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 -- 172.21.15.45:0/1509440737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f9187570 msgr2=0x55d6f918f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f9187570 0x55d6f918f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 -- 172.21.15.45:0/1509440737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f917c580 msgr2=0x55d6f9193270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f917c580 0x55d6f9193270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.801+0000 7f3675668640 1 -- 172.21.15.45:0/1509440737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f8ebf7d0 con 0x55d6f917cea0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f3675668640 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f366009b600 tx=0x7f3660000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3805a23513da233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f365b7fe640 1 -- 172.21.15.45:0/1509440737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660014020 con 0x55d6f917cea0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f365b7fe640 1 -- 172.21.15.45:0/1509440737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366009bd70 con 0x55d6f917cea0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f365b7fe640 1 -- 172.21.15.45:0/1509440737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660020070 con 0x55d6f917cea0 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 -- 172.21.15.45:0/1509440737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 msgr2=0x55d6f9186a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f366009b600 tx=0x7f3660000bc0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 -- 172.21.15.45:0/1509440737 shutdown_connections 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f9187570 0x55d6f918f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917cea0 0x55d6f9186a00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 --2- 172.21.15.45:0/1509440737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f917c580 0x55d6f9193270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 -- 172.21.15.45:0/1509440737 >> 172.21.15.45:0/1509440737 conn(0x55d6f91788a0 msgr2=0x55d6f91772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 -- 172.21.15.45:0/1509440737 shutdown_connections 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.802+0000 7f367add9740 1 -- 172.21.15.45:0/1509440737 wait complete. 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 -- start start 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 Processor -- start 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 0x55d6f918dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f918e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36701b2110 con 0x55d6f917c580 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36701b1a40 con 0x55d6f9187570 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36701aec60 con 0x55d6f917cea0 2024-04-25T00:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f3675e69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367add9740 10 client.0 _ll_register_callbacks cb 0x7f366000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f3675e69640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46754/0 (socket says 172.21.15.45:46754) 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f3675e69640 1 -- 172.21.15.45:0/2417972441 learned_addr learned my addr 172.21.15.45:0/2417972441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f367666a640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f918e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.803+0000 7f3675668640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 0x55d6f918dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 -- 172.21.15.45:0/2417972441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 msgr2=0x55d6f918dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 0x55d6f918dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 -- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 msgr2=0x55d6f918e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f918e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 -- 172.21.15.45:0/2417972441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f918e7a0 con 0x55d6f917c580 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675668640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 0x55d6f918dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f367666a640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f918e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f3675e69640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f366c005250 tx=0x7f366c0051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dde7ac87b2477bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c035040 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f366c0319c0 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f366009bd70 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f366c05a050 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f367add9740 10 client.17109 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6f91fe900 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366009bd70 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f366c003bf0 con 0x55d6f917c580 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.804+0000 7f367add9740 10 client.17109 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 20 client.17109 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f91fec00 0x55d6f9200ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 4 client.17109 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6f90e8f00 con 0x55d6f91fec00 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f3675668640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f91fec00 0x55d6f9200ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f3675668640 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f91fec00 0x55d6f9200ff0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c722e5096b4d6e00 server_cookie=71d51c53f51ae94a in_seq=0 out_seq=0 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f36597fa640 10 client.17109 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f36597fa640 1 -- 172.21.15.45:0/2417972441 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6f90e8f00 con 0x55d6f91fec00 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f36597fa640 10 client.17109 handle_command_reply: tid=0 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 1 client.17109 shutdown 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 msgr2=0x55d6f918f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.805+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f366c005250 tx=0x7f366c0051e0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 shutdown_connections 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d6f9187570 0x55d6f918e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6f917cea0 0x55d6f918dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d6f91fec00 0x55d6f9200ff0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 --2- 172.21.15.45:0/2417972441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6f917c580 0x55d6f918f900 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 >> 172.21.15.45:0/2417972441 conn(0x55d6f91788a0 msgr2=0x55d6f9177b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36597fa640 0 client.17109 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 shutdown_connections 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 1 -- 172.21.15.45:0/2417972441 wait complete. 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f367add9740 20 client.17109 trim_cache size 0 max 16384 2024-04-25T00:26:55.813 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:26:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 msgr2=0x7f365c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 0x7f365c0a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3668005c40 tx=0x7f3668009200 comp rx=0 tx=0).stop 2024-04-25T00:26:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 msgr2=0x7f367019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f366000d950 tx=0x7f366000de10 comp rx=0 tx=0).stop 2024-04-25T00:26:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 shutdown_connections 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f365c0a2d00 0x7f365c0a51c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3670109930 0x7f367019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36701042e0 0x7f367019a800 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 --2- 172.21.15.45:0/3408966272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3670103920 0x7f367019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 >> 172.21.15.45:0/3408966272 conn(0x7f36700ff3b0 msgr2=0x7f36700ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 shutdown_connections 2024-04-25T00:26:55.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:55.806+0000 7f36780cd640 1 -- 172.21.15.45:0/3408966272 wait complete. 2024-04-25T00:26:56.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:55 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4213967927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:56.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:55 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4213967927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:56.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:55 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4213967927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:56.818 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:26:56.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:26:56.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:26:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 -- 172.21.15.45:0/4059246740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc107720 msgr2=0x7f00fc107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 --2- 172.21.15.45:0/4059246740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc107720 0x7f00fc107b00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009930 tx=0x7f00ec02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 -- 172.21.15.45:0/4059246740 shutdown_connections 2024-04-25T00:26:56.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 --2- 172.21.15.45:0/4059246740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc1020d0 0x7f00fc10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 --2- 172.21.15.45:0/4059246740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 0x7f00fc101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 --2- 172.21.15.45:0/4059246740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc107720 0x7f00fc107b00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 -- 172.21.15.45:0/4059246740 >> 172.21.15.45:0/4059246740 conn(0x7f00fc0fd180 msgr2=0x7f00fc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 -- 172.21.15.45:0/4059246740 shutdown_connections 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.938+0000 7f010124d640 1 -- 172.21.15.45:0/4059246740 wait complete. 2024-04-25T00:26:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 Processor -- start 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 -- start start 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 0x7f00fc073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc107720 0x7f00fc0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc111b30 con 0x7f00fc1020d0 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1119b0 con 0x7f00fc101710 2024-04-25T00:26:56.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f010124d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc111cb0 con 0x7f00fc107720 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.939+0000 7f00fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46768/0 (socket says 172.21.15.45:46768) 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 -- 172.21.15.45:0/3869756299 learned_addr learned my addr 172.21.15.45:0/3869756299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fb59e640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc107720 0x7f00fc0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fad9d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 0x7f00fc073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 -- 172.21.15.45:0/3869756299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc107720 msgr2=0x7f00fc0744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc107720 0x7f00fc0744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 -- 172.21.15.45:0/3869756299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 msgr2=0x7f00fc073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 0x7f00fc073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 -- 172.21.15.45:0/3869756299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1a6570 con 0x7f00fc1020d0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00fa59c640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d950 tx=0x7f00f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00e3fff640 1 -- 172.21.15.45:0/3869756299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0017070 con 0x7f00fc1020d0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc1a6860 con 0x7f00fc1020d0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00e3fff640 1 -- 172.21.15.45:0/3869756299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f0009c30 con 0x7f00fc1020d0 2024-04-25T00:26:56.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f00e3fff640 1 -- 172.21.15.45:0/3869756299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000f6b0 con 0x7f00fc1020d0 2024-04-25T00:26:56.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.940+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1a6e00 con 0x7f00fc1020d0 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f00e3fff640 1 -- 172.21.15.45:0/3869756299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f00f00109d0 con 0x7f00fc1020d0 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 Processor -- start 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 -- start start 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d29d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d5d1f42c0 con 0x561d5d290290 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d5d19c910 con 0x561d5d29b010 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f0103bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d5d1f0280 con 0x561d5d291980 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f00e3fff640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 0x7f00e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46776/0 (socket says 172.21.15.45:46776) 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.943+0000 7f00fad9d640 1 -- 172.21.15.45:0/2891890327 learned_addr learned my addr 172.21.15.45:0/2891890327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fb59e640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 0x7f00e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fa59c640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d29d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fb59e640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00e3fff640 1 -- 172.21.15.45:0/3869756299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f00f00cf420 con 0x7f00fc1020d0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 -- 172.21.15.45:0/2891890327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 msgr2=0x561d5d28fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 -- 172.21.15.45:0/2891890327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 msgr2=0x561d5d29d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d29d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 -- 172.21.15.45:0/2891890327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5d1adfb0 con 0x561d5d290290 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fb59e640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fb59e640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 0x7f00e40a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00f40098b0 tx=0x7f00f4009840 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00fad9d640 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f00ec02a940 tx=0x7f00ec005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a997cf5434fca56e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00e17fa640 1 -- 172.21.15.45:0/2891890327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec03f070 con 0x561d5d290290 2024-04-25T00:26:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00e17fa640 1 -- 172.21.15.45:0/2891890327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec004030 con 0x561d5d290290 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.944+0000 7f00e17fa640 1 -- 172.21.15.45:0/2891890327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec035070 con 0x561d5d290290 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- 172.21.15.45:0/2891890327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 msgr2=0x561d5d290710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f00ec02a940 tx=0x7f00ec005e80 comp rx=0 tx=0).stop 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- 172.21.15.45:0/2891890327 shutdown_connections 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d29d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d290710 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 --2- 172.21.15.45:0/2891890327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- 172.21.15.45:0/2891890327 >> 172.21.15.45:0/2891890327 conn(0x561d5d258950 msgr2=0x561d5d28c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- 172.21.15.45:0/2891890327 shutdown_connections 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- 172.21.15.45:0/2891890327 wait complete. 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 -- start start 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.945+0000 7f0103bfd740 1 Processor -- start 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d298450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d29b010 0x561d5d298990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d5d29de40 con 0x561d5d290290 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d5d1afe90 con 0x561d5d291980 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d5d29dfc0 con 0x561d5d29b010 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f0103bfd740 10 client.0 _ll_register_callbacks cb 0x561d5d1eb970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46788/0 (socket says 172.21.15.45:46788) 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 -- 172.21.15.45:0/1190404036 learned_addr learned my addr 172.21.15.45:0/1190404036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fa59c640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d298450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fb59e640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d29b010 0x561d5d298990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 -- 172.21.15.45:0/1190404036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d29b010 msgr2=0x561d5d298990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d29b010 0x561d5d298990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 -- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 msgr2=0x561d5d298450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d298450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 -- 172.21.15.45:0/1190404036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5cfd07e0 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fa59c640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d298450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0095c0 tx=0x7f00ec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c3476f4c04228a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.946+0000 7f00deff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec035070 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f00ec0317b0 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec004030 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00ec055050 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 10 client.17121 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d5d030f70 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec004030 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00ec0047e0 con 0x561d5d290290 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 10 client.17121 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 20 client.17121 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d2aaf10 0x561d5d2ad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 4 client.17121 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.947+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d5d1c85a0 con 0x561d5d2aaf10 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.948+0000 7f00fa59c640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d2aaf10 0x561d5d2ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.948+0000 7f00fa59c640 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d2aaf10 0x561d5d2ad300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c845bf0714ed2af1 server_cookie=7487d0f62f324a88 in_seq=0 out_seq=0 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.948+0000 7f00deff5640 10 client.17121 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f00deff5640 1 -- 172.21.15.45:0/1190404036 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d5d1c85a0 con 0x561d5d2aaf10 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f00deff5640 10 client.17121 handle_command_reply: tid=0 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 client.17121 shutdown 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 msgr2=0x561d5d297f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0095c0 tx=0x7f00ec005bd0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 shutdown_connections 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d29b010 0x561d5d298990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d298450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d2aaf10 0x561d5d2ad300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1190404036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d297f10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 >> 172.21.15.45:0/1190404036 conn(0x561d5d258950 msgr2=0x561d5d2925f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f00deff5640 0 client.17121 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 shutdown_connections 2024-04-25T00:26:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 1 -- 172.21.15.45:0/1190404036 wait complete. 2024-04-25T00:26:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.949+0000 7f0103bfd740 20 client.17121 trim_cache size 0 max 16384 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 Processor -- start 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 -- start start 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d2b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b0d30 con 0x561d5d290290 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b2870 con 0x561d5d29b010 2024-04-25T00:26:56.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f0103bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b1400 con 0x561d5d291980 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46794/0 (socket says 172.21.15.45:46794) 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 -- 172.21.15.45:0/1257509350 learned_addr learned my addr 172.21.15.45:0/1257509350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fa59c640 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 -- 172.21.15.45:0/1257509350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 msgr2=0x561d5d28b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fb59e640 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d2b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 -- 172.21.15.45:0/1257509350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 msgr2=0x561d5d2b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d2b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.967+0000 7f00fad9d640 1 -- 172.21.15.45:0/1257509350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5cec8170 con 0x561d5d290290 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f00fad9d640 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00ec004b80 tx=0x7f00ec005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f3c64f6ef2558c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f00e0ff9640 1 -- 172.21.15.45:0/1257509350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec035070 con 0x561d5d290290 2024-04-25T00:26:56.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f00e0ff9640 1 -- 172.21.15.45:0/1257509350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec04c070 con 0x561d5d290290 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f00e0ff9640 1 -- 172.21.15.45:0/1257509350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec03f040 con 0x561d5d290290 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 -- 172.21.15.45:0/1257509350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 msgr2=0x561d5d2a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00ec004b80 tx=0x7f00ec005e80 comp rx=0 tx=0).stop 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 -- 172.21.15.45:0/1257509350 shutdown_connections 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d29b010 0x561d5d2b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d291980 0x561d5d28b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 --2- 172.21.15.45:0/1257509350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d290290 0x561d5d2a6d40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 -- 172.21.15.45:0/1257509350 >> 172.21.15.45:0/1257509350 conn(0x561d5d258950 msgr2=0x561d5d28b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 -- 172.21.15.45:0/1257509350 shutdown_connections 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.968+0000 7f0103bfd740 1 -- 172.21.15.45:0/1257509350 wait complete. 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 -- start start 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 Processor -- start 2024-04-25T00:26:56.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d290290 0x561d5d2a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d2a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b21a0 con 0x561d5d29b010 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1b1ad0 con 0x561d5d291980 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1aed10 con 0x561d5d290290 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f0103bfd740 10 client.0 _ll_register_callbacks cb 0x7f00f000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46800/0 (socket says 172.21.15.45:46800) 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 -- 172.21.15.45:0/126285281 learned_addr learned my addr 172.21.15.45:0/126285281 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fad9d640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d290290 0x561d5d2a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fa59c640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d2a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 -- 172.21.15.45:0/126285281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d290290 msgr2=0x561d5d2a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d290290 0x561d5d2a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 -- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 msgr2=0x561d5d2a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d2a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fb59e640 1 -- 172.21.15.45:0/126285281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5d2949d0 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.969+0000 7f00fa59c640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d2a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00fb59e640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009fd0 tx=0x7f00f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3906fac08e55baf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004d20 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f00f4004030 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec04c070 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00f4004230 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 10 client.17127 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d5d295d70 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec04c070 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00f4027030 con 0x561d5d29b010 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 10 client.17127 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 20 client.17127 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:56.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d312520 0x561d5d314940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:56.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 4 client.17127 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:26:56.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.970+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d5d201800 con 0x561d5d312520 2024-04-25T00:26:56.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.971+0000 7f00fad9d640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d312520 0x561d5d314940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:56.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.971+0000 7f00deff5640 10 client.17127 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:56.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.971+0000 7f00fad9d640 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d312520 0x561d5d314940 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a1714935a6819e server_cookie=2193835430d89805 in_seq=0 out_seq=0 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f00deff5640 1 -- 172.21.15.45:0/126285281 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d5d201800 con 0x561d5d312520 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f00deff5640 10 client.17127 handle_command_reply: tid=0 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 client.17127 shutdown 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 msgr2=0x561d5d2941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009fd0 tx=0x7f00f4004ab0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 shutdown_connections 2024-04-25T00:26:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561d5d312520 0x561d5d314940 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d5d29b010 0x561d5d2941a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d5d291980 0x561d5d2a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f00deff5640 0 client.17127 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 --2- 172.21.15.45:0/126285281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d5d290290 0x561d5d2a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 >> 172.21.15.45:0/126285281 conn(0x561d5d258950 msgr2=0x561d5d28b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.972+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 shutdown_connections 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f0103bfd740 1 -- 172.21.15.45:0/126285281 wait complete. 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f0103bfd740 20 client.17127 trim_cache size 0 max 16384 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "40750fa9-2ade-4e6c-9b76-f0aac1194cd5", 2024-04-25T00:26:56.979 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:26:56.980 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 msgr2=0x7f00e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 0x7f00e40a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00f40098b0 tx=0x7f00f4009840 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 msgr2=0x7f00fc073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d950 tx=0x7f00f000de10 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 shutdown_connections 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00e40a2d00 0x7f00e40a51c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00fc107720 0x7f00fc0744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00fc1020d0 0x7f00fc073f80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 --2- 172.21.15.45:0/3869756299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00fc101710 0x7f00fc073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 >> 172.21.15.45:0/3869756299 conn(0x7f00fc0fd180 msgr2=0x7f00fc0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 shutdown_connections 2024-04-25T00:26:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:56.973+0000 7f010124d640 1 -- 172.21.15.45:0/3869756299 wait complete. 2024-04-25T00:26:56.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:56.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:57.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:56 smithi045 ceph-mon[29194]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:57.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:56 smithi122 ceph-mon[32959]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 -- 172.21.15.45:0/2858512128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac06a780 msgr2=0x7fc7ac10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/2858512128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac06a780 0x7fc7ac10e5f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009cd0 tx=0x7fc7a402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 -- 172.21.15.45:0/2858512128 shutdown_connections 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/2858512128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac06a780 0x7fc7ac10e5f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/2858512128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/2858512128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac107730 0x7fc7ac107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 -- 172.21.15.45:0/2858512128 >> 172.21.15.45:0/2858512128 conn(0x7fc7ac0776a0 msgr2=0x7fc7ac077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 -- 172.21.15.45:0/2858512128 shutdown_connections 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.088+0000 7fc7b10bd640 1 -- 172.21.15.45:0/2858512128 wait complete. 2024-04-25T00:26:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.089+0000 7fc7b10bd640 1 Processor -- start 2024-04-25T00:26:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.089+0000 7fc7b10bd640 1 -- start start 2024-04-25T00:26:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac06a780 0x7fc7ac198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac107730 0x7fc7ac19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac111b40 con 0x7fc7ac069dc0 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1119c0 con 0x7fc7ac06a780 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac111cc0 con 0x7fc7ac107730 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46820/0 (socket says 172.21.15.45:46820) 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 -- 172.21.15.45:0/304449260 learned_addr learned my addr 172.21.15.45:0/304449260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac06a780 0x7fc7ac198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac107730 0x7fc7ac19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 -- 172.21.15.45:0/304449260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac107730 msgr2=0x7fc7ac19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac107730 0x7fc7ac19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 -- 172.21.15.45:0/304449260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac06a780 msgr2=0x7fc7ac198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac06a780 0x7fc7ac198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 -- 172.21.15.45:0/304449260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac19d0b0 con 0x7fc7ac069dc0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d950 tx=0x7fc79c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc793fff640 1 -- 172.21.15.45:0/304449260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c017070 con 0x7fc7ac069dc0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac19d3a0 con 0x7fc7ac069dc0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc793fff640 1 -- 172.21.15.45:0/304449260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c009c30 con 0x7fc7ac069dc0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.090+0000 7fc793fff640 1 -- 172.21.15.45:0/304449260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00f6b0 con 0x7fc7ac069dc0 2024-04-25T00:26:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.091+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1a4c30 con 0x7fc7ac069dc0 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc793fff640 1 -- 172.21.15.45:0/304449260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc79c0109d0 con 0x7fc7ac069dc0 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 Processor -- start 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 -- start start 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc793fff640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 0x7fc7940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ed94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ed9a30 0x557b32edbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557b32e2f6b0 con 0x557b32eca530 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557b32dd7880 con 0x557b32ecae50 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.093+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b32e2b670 con 0x557b32ed9a30 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 0x7fc7940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ed94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46828/0 (socket says 172.21.15.45:46828) 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3031166013 learned_addr learned my addr 172.21.15.45:0/3031166013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc793fff640 1 -- 172.21.15.45:0/304449260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc79c0cf340 con 0x7fc7ac069dc0 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ed9a30 0x557b32edbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3031166013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ed9a30 msgr2=0x557b32edbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ed9a30 0x557b32edbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3031166013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 msgr2=0x557b32ed94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ed94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3031166013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b32c0d7d0 con 0x557b32eca530 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ed94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 0x7fc7940a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005cc0 tx=0x7fc7a4031630 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc79c09b200 tx=0x7fc79c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcee38a083d94ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7917fa640 1 -- 172.21.15.45:0/3031166013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c020040 con 0x557b32eca530 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7917fa640 1 -- 172.21.15.45:0/3031166013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c009de0 con 0x557b32eca530 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.094+0000 7fc7917fa640 1 -- 172.21.15.45:0/3031166013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0a4170 con 0x557b32eca530 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3031166013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 msgr2=0x557b32eca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc79c09b200 tx=0x7fc79c005880 comp rx=0 tx=0).stop 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3031166013 shutdown_connections 2024-04-25T00:26:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ed9a30 0x557b32edbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ed94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3031166013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32eca910 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3031166013 >> 172.21.15.45:0/3031166013 conn(0x557b32ec6950 msgr2=0x557b32ec6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3031166013 shutdown_connections 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3031166013 wait complete. 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 -- start start 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.095+0000 7fc7b3dd2740 1 Processor -- start 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ecae50 0x557b32ed6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557b32edc730 con 0x557b32eca530 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557b32deaf50 con 0x557b32ed9a30 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b32edc8b0 con 0x557b32ecae50 2024-04-25T00:26:57.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 10 client.0 _ll_register_callbacks cb 0x557b32e26000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46840/0 (socket says 172.21.15.45:46840) 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3318762985 learned_addr learned my addr 172.21.15.45:0/3318762985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ecae50 0x557b32ed6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3318762985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ecae50 msgr2=0x557b32ed6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ecae50 0x557b32ed6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 msgr2=0x557b32ed73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 -- 172.21.15.45:0/3318762985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b32b433f0 con 0x557b32eca530 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ccd30 tx=0x7fc79c0cc6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1918ff850c39142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc78eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c020040 con 0x557b32eca530 2024-04-25T00:26:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc79c097b70 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.096+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc79c009de0 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc79c0a8b80 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 10 client.17139 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b32b030f0 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c009de0 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc79c097250 con 0x557b32eca530 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 10 client.17139 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 20 client.17139 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32ee9ab0 0x557b32eebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 4 client.17139 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:57.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b32e3cbf0 con 0x557b32ee9ab0 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.097+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32ee9ab0 0x557b32eebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.098+0000 7fc78eff5640 10 client.17139 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.098+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32ee9ab0 0x557b32eebea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7fccca76bd09f04 server_cookie=ad693fc47c10bb5e in_seq=0 out_seq=0 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.098+0000 7fc78eff5640 1 -- 172.21.15.45:0/3318762985 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b32e3cbf0 con 0x557b32ee9ab0 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.098+0000 7fc78eff5640 10 client.17139 handle_command_reply: tid=0 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 client.17139 shutdown 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 msgr2=0x557b32ed6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ccd30 tx=0x7fc79c0cc6d0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 shutdown_connections 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32ecae50 0x557b32ed6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32ee9ab0 0x557b32eebea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3318762985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32eca530 0x557b32ed6970 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc78eff5640 0 client.17139 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 >> 172.21.15.45:0/3318762985 conn(0x557b32ec6950 msgr2=0x557b32ecc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 shutdown_connections 2024-04-25T00:26:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3318762985 wait complete. 2024-04-25T00:26:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.099+0000 7fc7b3dd2740 20 client.17139 trim_cache size 0 max 16384 2024-04-25T00:26:57.122 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:56 smithi071 ceph-mon[32667]: pgmap v152: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:26:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 Processor -- start 2024-04-25T00:26:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 -- start start 2024-04-25T00:26:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ee5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ec6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ac9a0 con 0x557b32ed9a30 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ae4e0 con 0x557b32ecae50 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.116+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ad070 con 0x557b32eca530 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46850/0 (socket says 172.21.15.45:46850) 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 -- 172.21.15.45:0/3323132037 learned_addr learned my addr 172.21.15.45:0/3323132037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ee5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ec6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 -- 172.21.15.45:0/3323132037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 msgr2=0x557b32ee5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ee5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 -- 172.21.15.45:0/3323132037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 msgr2=0x557b32ec6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ec6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 -- 172.21.15.45:0/3323132037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b32eec210 con 0x557b32ed9a30 2024-04-25T00:26:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ec6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009cd0 tx=0x7fc7a4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514656907ca62a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc790ff9640 1 -- 172.21.15.45:0/3323132037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a404d020 con 0x557b32ed9a30 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc790ff9640 1 -- 172.21.15.45:0/3323132037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4031b80 con 0x557b32ed9a30 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.117+0000 7fc790ff9640 1 -- 172.21.15.45:0/3323132037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4049420 con 0x557b32ed9a30 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3323132037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 msgr2=0x557b32ec1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009cd0 tx=0x7fc7a4005c50 comp rx=0 tx=0).stop 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3323132037 shutdown_connections 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ed9a30 0x557b32ec1a30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ecae50 0x557b32ec6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3323132037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ee5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3323132037 >> 172.21.15.45:0/3323132037 conn(0x557b32ec6950 msgr2=0x557b32ed6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3323132037 shutdown_connections 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3323132037 wait complete. 2024-04-25T00:26:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- start start 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 Processor -- start 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ed3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ade10 con 0x557b32ecae50 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ad740 con 0x557b32ed9a30 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1aa980 con 0x557b32eca530 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7b3dd2740 10 client.0 _ll_register_callbacks cb 0x7fc7a0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.118+0000 7fc7aa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46858/0 (socket says 172.21.15.45:46858) 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 -- 172.21.15.45:0/3113042102 learned_addr learned my addr 172.21.15.45:0/3113042102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ed3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7ab59e640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 -- 172.21.15.45:0/3113042102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 msgr2=0x557b32ed3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ed3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 -- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 msgr2=0x557b32ed40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 -- 172.21.15.45:0/3113042102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b32f4ece0 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ed3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7aa59c640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000c00 tx=0x7fc7a00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea4f24763ab5f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc78eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0005220 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc7a00053c0 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4031b80 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7a002f050 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7b3dd2740 10 client.17145 fetch_fsmap learned FSMap version 28 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b32f4ff40 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.119+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4031b80 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc7a0020400 con 0x557b32ecae50 2024-04-25T00:26:57.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 10 client.17145 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 20 client.17145 populate_metadata read hostname 'smithi045' 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32f506f0 0x557b32f52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 4 client.17145 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b32e36f40 con 0x557b32f506f0 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32f506f0 0x557b32f52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc78eff5640 10 client.17145 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7aad9d640 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32f506f0 0x557b32f52b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363b869da3183cf4 server_cookie=6d0d706d667476 in_seq=0 out_seq=0 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc78eff5640 1 -- 172.21.15.45:0/3113042102 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557b32e36f40 con 0x557b32f506f0 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc78eff5640 10 client.17145 handle_command_reply: tid=0 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.120+0000 7fc7b3dd2740 1 client.17145 shutdown 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 msgr2=0x557b32ed3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000c00 tx=0x7fc7a00035f0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 shutdown_connections 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b32ed9a30 0x557b32ed40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557b32f506f0 0x557b32f52b10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557b32ecae50 0x557b32ed3b80 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 --2- 172.21.15.45:0/3113042102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b32eca530 0x557b32ed3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc78eff5640 0 client.17145 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 >> 172.21.15.45:0/3113042102 conn(0x557b32ec6950 msgr2=0x557b32ed71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 shutdown_connections 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 1 -- 172.21.15.45:0/3113042102 wait complete. 2024-04-25T00:26:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b3dd2740 20 client.17145 trim_cache size 0 max 16384 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "40750fa9-2ade-4e6c-9b76-f0aac1194cd5": { 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "40750fa9-2ade-4e6c-9b76-f0aac1194cd5", 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:26:57.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:26:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 msgr2=0x7fc7940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 0x7fc7940a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005cc0 tx=0x7fc7a4031630 comp rx=0 tx=0).stop 2024-04-25T00:26:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 msgr2=0x7fc7ac1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:57.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d950 tx=0x7fc79c00de10 comp rx=0 tx=0).stop 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 shutdown_connections 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc7940a2d00 0x7fc7940a51c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ac107730 0x7fc7ac19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac06a780 0x7fc7ac198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 --2- 172.21.15.45:0/304449260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7ac069dc0 0x7fc7ac1980c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 >> 172.21.15.45:0/304449260 conn(0x7fc7ac0776a0 msgr2=0x7fc7ac0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 shutdown_connections 2024-04-25T00:26:57.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:26:57.121+0000 7fc7b10bd640 1 -- 172.21.15.45:0/304449260 wait complete. 2024-04-25T00:26:57.133 INFO:tasks.cephfs.filesystem:scrub status for tag:40750fa9-2ade-4e6c-9b76-f0aac1194cd5 - {'path': '/', 'tag': '40750fa9-2ade-4e6c-9b76-f0aac1194cd5', 'options': 'recursive,force'} 2024-04-25T00:26:58.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:57 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:26:58.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:57 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:57 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:26:58.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:57 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:57 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:26:58.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:57 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:26:58.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:57 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:57 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:26:58.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:57 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:57 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:26:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:57 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:26:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:57 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:57 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:26:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:57 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:58.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:57 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:26:59.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:26:58 smithi122 ceph-mon[32959]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:26:58 smithi071 ceph-mon[32667]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:26:59.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:26:58 smithi045 ceph-mon[29194]: pgmap v153: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:00 smithi122 ceph-mon[32959]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:00 smithi071 ceph-mon[32667]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:00 smithi045 ceph-mon[29194]: pgmap v154: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:02 smithi122 ceph-mon[32959]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:02 smithi071 ceph-mon[32667]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:03.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:02 smithi045 ceph-mon[29194]: pgmap v155: 129 pgs: 129 active+clean; 259 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:04 smithi122 ceph-mon[32959]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:27:05.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:04 smithi071 ceph-mon[32667]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:27:05.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:04 smithi045 ceph-mon[29194]: pgmap v156: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:27:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:06 smithi122 ceph-mon[32959]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:27:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:06 smithi071 ceph-mon[32667]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:27:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:06 smithi045 ceph-mon[29194]: pgmap v157: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:27:07.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:08.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:07 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:08.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:07 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:08.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:07 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:27:08.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:07 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:08.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:07 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:08.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:07 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:27:08.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:07 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:08.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:07 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:08.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:07 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:27:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:08 smithi122 ceph-mon[32959]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:09.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:08 smithi071 ceph-mon[32667]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:09.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:08 smithi045 ceph-mon[29194]: pgmap v158: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:10 smithi122 ceph-mon[32959]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:27:11.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:10 smithi071 ceph-mon[32667]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:27:11.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:10 smithi045 ceph-mon[29194]: pgmap v159: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:27:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:12 smithi122 ceph-mon[32959]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:13.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:12 smithi071 ceph-mon[32667]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:13.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:12 smithi045 ceph-mon[29194]: pgmap v160: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:15.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:14 smithi122 ceph-mon[32959]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:27:15.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:14 smithi071 ceph-mon[32667]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:27:15.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:14 smithi045 ceph-mon[29194]: pgmap v161: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-25T00:27:17.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:16 smithi122 ceph-mon[32959]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:17.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:16 smithi071 ceph-mon[32667]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:17.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:16 smithi045 ceph-mon[29194]: pgmap v162: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:27:19.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:18 smithi122 ceph-mon[32959]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:27:19.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:18 smithi071 ceph-mon[32667]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:27:19.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:18 smithi045 ceph-mon[29194]: pgmap v163: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:27:21.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:20 smithi122 ceph-mon[32959]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:27:21.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:20 smithi071 ceph-mon[32667]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:27:21.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:20 smithi045 ceph-mon[29194]: pgmap v164: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:27:23.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:22 smithi122 ceph-mon[32959]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:23.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:22 smithi071 ceph-mon[32667]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:23.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:22 smithi045 ceph-mon[29194]: pgmap v165: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:24 smithi122 ceph-mon[32959]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:25.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:24 smithi071 ceph-mon[32667]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:25.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:24 smithi045 ceph-mon[29194]: pgmap v166: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:27.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:26 smithi122 ceph-mon[32959]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:27.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:26 smithi071 ceph-mon[32667]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:27.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:27:27.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:27:27.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:26 smithi045 ceph-mon[29194]: pgmap v167: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 -- 172.21.15.45:0/2913037715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c01077a0 msgr2=0x7ff9c0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/2913037715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c01077a0 0x7ff9c0107b80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009cd0 tx=0x7ff9b002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 -- 172.21.15.45:0/2913037715 shutdown_connections 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/2913037715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0102150 0x7ff9c010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/2913037715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c0101790 0x7ff9c0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/2913037715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c01077a0 0x7ff9c0107b80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 -- 172.21.15.45:0/2913037715 >> 172.21.15.45:0/2913037715 conn(0x7ff9c00fd2a0 msgr2=0x7ff9c00ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 -- 172.21.15.45:0/2913037715 shutdown_connections 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 -- 172.21.15.45:0/2913037715 wait complete. 2024-04-25T00:27:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.252+0000 7ff9c57dc640 1 Processor -- start 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 -- start start 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0101790 0x7ff9c0073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 0x7ff9c00744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111bc0 con 0x7ff9c0102150 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111a40 con 0x7ff9c01077a0 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9c57dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111d40 con 0x7ff9c0101790 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51148/0 (socket says 172.21.15.45:51148) 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 -- 172.21.15.45:0/4170504333 learned_addr learned my addr 172.21.15.45:0/4170504333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9beffd640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0101790 0x7ff9c0073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 0x7ff9c00744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 -- 172.21.15.45:0/4170504333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0101790 msgr2=0x7ff9c0073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0101790 0x7ff9c0073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 -- 172.21.15.45:0/4170504333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 msgr2=0x7ff9c00744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 0x7ff9c00744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 -- 172.21.15.45:0/4170504333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c01a65e0 con 0x7ff9c0102150 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 0x7ff9c00744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.253+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400d950 tx=0x7ff9b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.254+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4170504333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4017030 con 0x7ff9c0102150 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.254+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c01a68d0 con 0x7ff9c0102150 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.254+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4170504333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b4004020 con 0x7ff9c0102150 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.254+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4170504333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40047a0 con 0x7ff9c0102150 2024-04-25T00:27:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.254+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a6e70 con 0x7ff9c0102150 2024-04-25T00:27:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.256+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4170504333 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff9b40041d0 con 0x7ff9c0102150 2024-04-25T00:27:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.256+0000 7ff9c85c7740 1 Processor -- start 2024-04-25T00:27:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.256+0000 7ff9c85c7740 1 -- start start 2024-04-25T00:27:27.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8cf50 0x5617dfd970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617dfcf16b0 con 0x5617dfd97620 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5617dfc99880 con 0x5617dfd8cf50 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5617dfced670 con 0x5617dfd8c630 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 0x7ff9a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51164/0 (socket says 172.21.15.45:51164) 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 -- 172.21.15.45:0/1212326447 learned_addr learned my addr 172.21.15.45:0/1212326447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 0x7ff9a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9beffd640 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8cf50 0x5617dfd970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4170504333 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9b40cf1f0 con 0x7ff9c0102150 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 -- 172.21.15.45:0/1212326447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 msgr2=0x5617dfd8ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd8ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 -- 172.21.15.45:0/1212326447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8cf50 msgr2=0x5617dfd970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8cf50 0x5617dfd970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 -- 172.21.15.45:0/1212326447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dfacf7d0 con 0x5617dfd97620 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005fd0 tx=0x7ff9b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78934f6d3c3bc214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.257+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 0x7ff9a80a51f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009460 tx=0x7ff9b8009a10 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9a57fa640 1 -- 172.21.15.45:0/1212326447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40a3fc0 con 0x5617dfd97620 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9a57fa640 1 -- 172.21.15.45:0/1212326447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b409bce0 con 0x5617dfd97620 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9a57fa640 1 -- 172.21.15.45:0/1212326447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40979c0 con 0x5617dfd97620 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 -- 172.21.15.45:0/1212326447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 msgr2=0x5617dfd99a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005fd0 tx=0x7ff9b4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 -- 172.21.15.45:0/1212326447 shutdown_connections 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd99a10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8cf50 0x5617dfd970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/1212326447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd8ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 -- 172.21.15.45:0/1212326447 >> 172.21.15.45:0/1212326447 conn(0x5617dfd88950 msgr2=0x5617dfd88d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 -- 172.21.15.45:0/1212326447 shutdown_connections 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.258+0000 7ff9c85c7740 1 -- 172.21.15.45:0/1212326447 wait complete. 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 -- start start 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 Processor -- start 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfd94560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd94aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617dfd9a3b0 con 0x5617dfd97620 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5617dfcacf50 con 0x5617dfd8c630 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5617dfd9a530 con 0x5617dfd8cf50 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9c85c7740 10 client.0 _ll_register_callbacks cb 0x5617dfce8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51174/0 (socket says 172.21.15.45:51174) 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2318818976 learned_addr learned my addr 172.21.15.45:0/2318818976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9beffd640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfd94560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd94aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2318818976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 msgr2=0x5617dfd94aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd94aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 msgr2=0x5617dfd94560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfd94560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2318818976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dfa053f0 con 0x5617dfd97620 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.259+0000 7ff9beffd640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfd94560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:27.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8014fe0 tx=0x7ff9b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e03d1f3218aab54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8004230 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9b80043d0 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617dfcfebf0 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9b8011e70 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9c85c7740 10 client.17157 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617df9c50f0 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.260+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5617dfcfebf0 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9b801d070 con 0x5617dfd97620 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 10 client.17157 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 20 client.17157 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfda78e0 0x5617dfda9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 4 client.17157 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617dfcf8f40 con 0x5617dfda78e0 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9beffd640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfda78e0 0x5617dfda9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9a2ff5640 10 client.17157 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.261+0000 7ff9beffd640 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfda78e0 0x5617dfda9d10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0ca574e85582f4 server_cookie=ad35e0e823ac79ec in_seq=0 out_seq=0 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/2318818976 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617dfcf8f40 con 0x5617dfda78e0 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9a2ff5640 10 client.17157 handle_command_reply: tid=0 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 client.17157 shutdown 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 msgr2=0x5617dfd94fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8014fe0 tx=0x7ff9b8005f90 comp rx=0 tx=0).stop 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 shutdown_connections 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfda78e0 0x5617dfda9d10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd94fe0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd94aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9a2ff5640 0 client.17157 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2318818976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfd94560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 >> 172.21.15.45:0/2318818976 conn(0x5617dfd88950 msgr2=0x5617dfd8e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 shutdown_connections 2024-04-25T00:27:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2318818976 wait complete. 2024-04-25T00:27:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.262+0000 7ff9c85c7740 20 client.17157 trim_cache size 0 max 16384 2024-04-25T00:27:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 Processor -- start 2024-04-25T00:27:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 -- start start 2024-04-25T00:27:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfda32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd88d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01b0e60 con 0x5617dfd97620 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01b29a0 con 0x5617dfd8c630 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.280+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01b1530 con 0x5617dfd8cf50 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51178/0 (socket says 172.21.15.45:51178) 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2064217937 learned_addr learned my addr 172.21.15.45:0/2064217937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9beffd640 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfda32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd88d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2064217937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 msgr2=0x5617dfd88d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd88d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2064217937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 msgr2=0x5617dfda32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfda32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 -- 172.21.15.45:0/2064217937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dfd7a090 con 0x5617dfd97620 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8003d00 tx=0x7ff9b8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16104007b176f299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/2064217937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b803b070 con 0x5617dfd97620 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/2064217937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b80044e0 con 0x5617dfd97620 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.281+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/2064217937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8029470 con 0x5617dfd97620 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2064217937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 msgr2=0x5617dfd8ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8003d00 tx=0x7ff9b8014600 comp rx=0 tx=0).stop 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2064217937 shutdown_connections 2024-04-25T00:27:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd97620 0x5617dfd8ed90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8cf50 0x5617dfd88d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/2064217937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd8c630 0x5617dfda32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2064217937 >> 172.21.15.45:0/2064217937 conn(0x5617dfd88950 msgr2=0x5617dfd87260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2064217937 shutdown_connections 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- 172.21.15.45:0/2064217937 wait complete. 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 -- start start 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.282+0000 7ff9c85c7740 1 Processor -- start 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd9f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd97620 0x5617dfd90870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01b22d0 con 0x5617dfd8cf50 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01b1c00 con 0x5617dfd97620 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01aee40 con 0x5617dfd8c630 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 10 client.0 _ll_register_callbacks cb 0x7ff9b8009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51186/0 (socket says 172.21.15.45:51186) 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 -- 172.21.15.45:0/3842228977 learned_addr learned my addr 172.21.15.45:0/3842228977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9beffd640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd9f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd97620 0x5617dfd90870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 -- 172.21.15.45:0/3842228977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 msgr2=0x5617dfd9f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd9f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 -- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd97620 msgr2=0x5617dfd90870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd97620 0x5617dfd90870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 -- 172.21.15.45:0/3842228977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dfd90f60 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9be7fc640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff9b409b050 tx=0x7ff9b409b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec06fb82ccbc3f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b409ba50 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff9b40cc800 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.283+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b80044e0 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9b40cca00 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 10 client.17163 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617dfe0d830 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b80044e0 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9b40b4120 con 0x5617dfd8cf50 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 10 client.17163 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 20 client.17163 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfe0db70 0x5617dfe0ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 4 client.17163 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617dfc766f0 con 0x5617dfe0db70 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.284+0000 7ff9beffd640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfe0db70 0x5617dfe0ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9beffd640 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfe0db70 0x5617dfe0ff60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a51700fa818797 server_cookie=915f2da739e207ee in_seq=0 out_seq=0 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9a2ff5640 10 client.17163 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9a2ff5640 1 -- 172.21.15.45:0/3842228977 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5617dfc766f0 con 0x5617dfe0db70 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9a2ff5640 10 client.17163 handle_command_reply: tid=0 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 client.17163 shutdown 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 msgr2=0x5617dfd9fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff9b409b050 tx=0x7ff9b409b5b0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 shutdown_connections 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5617dfd97620 0x5617dfd90870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5617dfe0db70 0x5617dfe0ff60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617dfd8cf50 0x5617dfd9fcf0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 --2- 172.21.15.45:0/3842228977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5617dfd8c630 0x5617dfd9f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9a2ff5640 0 client.17163 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 >> 172.21.15.45:0/3842228977 conn(0x5617dfd88950 msgr2=0x5617dfd87af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 shutdown_connections 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 1 -- 172.21.15.45:0/3842228977 wait complete. 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c85c7740 20 client.17163 trim_cache size 0 max 16384 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:27:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:27:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:27:27.294 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:27:27.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 msgr2=0x7ff9a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.285+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 0x7ff9a80a51f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009460 tx=0x7ff9b8009a10 comp rx=0 tx=0).stop 2024-04-25T00:27:27.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 msgr2=0x7ff9c0073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400d950 tx=0x7ff9b400de10 comp rx=0 tx=0).stop 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 shutdown_connections 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff9a80a2d30 0x7ff9a80a51f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c01077a0 0x7ff9c00744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c0102150 0x7ff9c0073f80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 --2- 172.21.15.45:0/4170504333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9c0101790 0x7ff9c0073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 >> 172.21.15.45:0/4170504333 conn(0x7ff9c00fd2a0 msgr2=0x7ff9c00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 shutdown_connections 2024-04-25T00:27:27.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.286+0000 7ff9c57dc640 1 -- 172.21.15.45:0/4170504333 wait complete. 2024-04-25T00:27:27.297 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:27:27.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 -- 172.21.15.45:0/2008210520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8364102a20 msgr2=0x7f8364102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 --2- 172.21.15.45:0/2008210520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8364102a20 0x7f8364102e00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8360009900 tx=0x7f836002a480 comp rx=0 tx=0).stop 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 -- 172.21.15.45:0/2008210520 shutdown_connections 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 --2- 172.21.15.45:0/2008210520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f836410bd60 0x7f836410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 --2- 172.21.15.45:0/2008210520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 0x7f836410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 --2- 172.21.15.45:0/2008210520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8364102a20 0x7f8364102e00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 -- 172.21.15.45:0/2008210520 >> 172.21.15.45:0/2008210520 conn(0x7f83640777a0 msgr2=0x7f8364077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 -- 172.21.15.45:0/2008210520 shutdown_connections 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.439+0000 7f836b922640 1 -- 172.21.15.45:0/2008210520 wait complete. 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 Processor -- start 2024-04-25T00:27:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 -- start start 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8364102a20 0x7f8364198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 0x7f83641985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111b30 con 0x7f836410bd60 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83641119b0 con 0x7f8364103340 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f836b922640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8364111cb0 con 0x7f8364102a20 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51204/0 (socket says 172.21.15.45:51204) 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 -- 172.21.15.45:0/1478320604 learned_addr learned my addr 172.21.15.45:0/1478320604 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f83696be640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8364102a20 0x7f8364198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8368ebd640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 0x7f83641985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 -- 172.21.15.45:0/1478320604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8364102a20 msgr2=0x7f8364198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8364102a20 0x7f8364198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 -- 172.21.15.45:0/1478320604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 msgr2=0x7f83641985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 0x7f83641985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.440+0000 7f8369ebf640 1 -- 172.21.15.45:0/1478320604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836419d030 con 0x7f836410bd60 2024-04-25T00:27:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f8369ebf640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e960 tx=0x7f835c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c004d20 con 0x7f836410bd60 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f835c005020 con 0x7f836410bd60 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836419d320 con 0x7f836410bd60 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c0116d0 con 0x7f836410bd60 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.441+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641a4bb0 con 0x7f836410bd60 2024-04-25T00:27:27.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.442+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83640689b0 con 0x7f836410bd60 2024-04-25T00:27:27.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.443+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f835c005560 con 0x7f836410bd60 2024-04-25T00:27:27.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.443+0000 7f835a7fc640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 0x7f83500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.443+0000 7f83696be640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 0x7f83500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.444+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f835c0cfb70 con 0x7f836410bd60 2024-04-25T00:27:27.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.444+0000 7f83696be640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 0x7f83500a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f83600058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.448+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f835c016030 con 0x7f836410bd60 2024-04-25T00:27:27.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.672+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8364106f40 con 0x7f836410bd60 2024-04-25T00:27:27.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.673+0000 7f835a7fc640 1 -- 172.21.15.45:0/1478320604 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f835c09bde0 con 0x7f836410bd60 2024-04-25T00:27:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:27:27.674 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:27:27.675 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:27:27.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 msgr2=0x7f83500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 0x7f83500a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f83600058d0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 msgr2=0x7f836419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e960 tx=0x7f835c00ee20 comp rx=0 tx=0).stop 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 shutdown_connections 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f83500a2d00 0x7f83500a51c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f836410bd60 0x7f836419c900 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8364103340 0x7f83641985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 --2- 172.21.15.45:0/1478320604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8364102a20 0x7f8364198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 >> 172.21.15.45:0/1478320604 conn(0x7f83640777a0 msgr2=0x7f83640fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 shutdown_connections 2024-04-25T00:27:27.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.675+0000 7f836b922640 1 -- 172.21.15.45:0/1478320604 wait complete. 2024-04-25T00:27:27.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 -- 172.21.15.45:0/1910892446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 msgr2=0x7fa56c109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 --2- 172.21.15.45:0/1910892446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c109d70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa568009930 tx=0x7fa56802a420 comp rx=0 tx=0).stop 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 -- 172.21.15.45:0/1910892446 shutdown_connections 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 --2- 172.21.15.45:0/1910892446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c104340 0x7fa56c110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 --2- 172.21.15.45:0/1910892446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c103980 0x7fa56c103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 --2- 172.21.15.45:0/1910892446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c109d70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 -- 172.21.15.45:0/1910892446 >> 172.21.15.45:0/1910892446 conn(0x7fa56c0ff4d0 msgr2=0x7fa56c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 -- 172.21.15.45:0/1910892446 shutdown_connections 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.792+0000 7fa573cf5640 1 -- 172.21.15.45:0/1910892446 wait complete. 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 Processor -- start 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 -- start start 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c103980 0x7fa56c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c104340 0x7fa56c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c113d70 con 0x7fa56c109990 2024-04-25T00:27:27.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c113bf0 con 0x7fa56c103980 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa573cf5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c113ef0 con 0x7fa56c104340 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51226/0 (socket says 172.21.15.45:51226) 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 -- 172.21.15.45:0/108765501 learned_addr learned my addr 172.21.15.45:0/108765501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa571a91640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c103980 0x7fa56c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa571290640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c104340 0x7fa56c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 -- 172.21.15.45:0/108765501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c104340 msgr2=0x7fa56c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c104340 0x7fa56c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 -- 172.21.15.45:0/108765501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c103980 msgr2=0x7fa56c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c103980 0x7fa56c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 -- 172.21.15.45:0/108765501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c19f290 con 0x7fa56c109990 2024-04-25T00:27:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.793+0000 7fa572292640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa56400eb10 tx=0x7fa56400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.794+0000 7fa562ffd640 1 -- 172.21.15.45:0/108765501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564004b20 con 0x7fa56c109990 2024-04-25T00:27:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.794+0000 7fa562ffd640 1 -- 172.21.15.45:0/108765501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa564004cc0 con 0x7fa56c109990 2024-04-25T00:27:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.794+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c19f580 con 0x7fa56c109990 2024-04-25T00:27:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.794+0000 7fa562ffd640 1 -- 172.21.15.45:0/108765501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564010760 con 0x7fa56c109990 2024-04-25T00:27:27.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.794+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1a6e10 con 0x7fa56c109990 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.796+0000 7fa562ffd640 1 -- 172.21.15.45:0/108765501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa564005020 con 0x7fa56c109990 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 Processor -- start 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 -- start start 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x564970359030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5649702b3670 con 0x56497034e580 2024-04-25T00:27:27.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56497025b970 con 0x56497034eea0 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa576a01740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5649702af630 con 0x564970359570 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa562ffd640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 0x7fa5580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.797+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51234/0 (socket says 172.21.15.45:51234) 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 -- 172.21.15.45:0/227017753 learned_addr learned my addr 172.21.15.45:0/227017753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa572292640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 0x7fa5580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571a91640 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x564970359030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa562ffd640 1 -- 172.21.15.45:0/108765501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa5640cfca0 con 0x7fa56c109990 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 -- 172.21.15.45:0/227017753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 msgr2=0x56497035b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 -- 172.21.15.45:0/227017753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 msgr2=0x564970359030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x564970359030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 -- 172.21.15.45:0/227017753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497027f400 con 0x56497034e580 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa572292640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 0x7fa5580a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cd690 tx=0x7fa5640912f0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa571290640 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0135a0 tx=0x7fa55c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570db1252bca2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa557fff640 1 -- 172.21.15.45:0/227017753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c004050 con 0x56497034e580 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa557fff640 1 -- 172.21.15.45:0/227017753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55c0041f0 con 0x56497034e580 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.798+0000 7fa557fff640 1 -- 172.21.15.45:0/227017753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00f7c0 con 0x56497034e580 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- 172.21.15.45:0/227017753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 msgr2=0x56497034e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0135a0 tx=0x7fa55c009850 comp rx=0 tx=0).stop 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- 172.21.15.45:0/227017753 shutdown_connections 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x564970359030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 --2- 172.21.15.45:0/227017753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x56497034e960 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- 172.21.15.45:0/227017753 >> 172.21.15.45:0/227017753 conn(0x56497034a8a0 msgr2=0x56497034ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- 172.21.15.45:0/227017753 shutdown_connections 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- 172.21.15.45:0/227017753 wait complete. 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.799+0000 7fa576a01740 1 -- start start 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 Processor -- start 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034eea0 0x564970356180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56497035c300 con 0x56497034e580 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56497026f050 con 0x564970359570 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56497035c480 con 0x56497034eea0 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa576a01740 10 client.0 _ll_register_callbacks cb 0x56496ffc6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51242/0 (socket says 172.21.15.45:51242) 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 -- 172.21.15.45:0/3910745892 learned_addr learned my addr 172.21.15.45:0/3910745892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571a91640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034eea0 0x564970356180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 -- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034eea0 msgr2=0x564970356180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034eea0 0x564970356180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 -- 172.21.15.45:0/3910745892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 msgr2=0x5649703566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.800+0000 7fa571290640 1 -- 172.21.15.45:0/3910745892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564970240dc0 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa571290640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003e70 tx=0x7fa55c0133e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e7e94cf96f911e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00f580 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa55c00f720 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa55c0041f0 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa55c035050 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 10 client.17181 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56496ffc73f0 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55c0041f0 con 0x56497034e580 2024-04-25T00:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa55c01e070 con 0x56497034e580 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 10 client.17181 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 10 client.17181 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 20 client.17181 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703694b0 0x56497036b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 4 client.17181 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.801+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649702c0bb0 con 0x5649703694b0 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.802+0000 7fa571a91640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703694b0 0x56497036b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.802+0000 7fa571a91640 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703694b0 0x56497036b8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a6c18eb14012a56 server_cookie=26dfbe953b34013e in_seq=0 out_seq=0 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.802+0000 7fa5557fa640 10 client.17181 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa5557fa640 1 -- 172.21.15.45:0/3910745892 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649702c0bb0 con 0x5649703694b0 2024-04-25T00:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa5557fa640 10 client.17181 handle_command_reply: tid=0 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 client.17181 shutdown 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 msgr2=0x564970358e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003e70 tx=0x7fa55c0133e0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 shutdown_connections 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703694b0 0x56497036b8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034eea0 0x564970356180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 --2- 172.21.15.45:0/3910745892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x564970358e90 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa5557fa640 0 client.17181 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 >> 172.21.15.45:0/3910745892 conn(0x56497034a8a0 msgr2=0x564970349b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 shutdown_connections 2024-04-25T00:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 1 -- 172.21.15.45:0/3910745892 wait complete. 2024-04-25T00:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.803+0000 7fa576a01740 20 client.17181 trim_cache size 0 max 16384 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.816+0000 7fa576a01740 1 Processor -- start 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.816+0000 7fa576a01740 1 -- start start 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 0x5649703651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1b0dc0 con 0x56497034eea0 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1b2900 con 0x564970359570 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa576a01740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1b1490 con 0x56497034e580 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51244/0 (socket says 172.21.15.45:51244) 2024-04-25T00:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 -- 172.21.15.45:0/887947525 learned_addr learned my addr 172.21.15.45:0/887947525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571290640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 0x5649703651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571290640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 -- 172.21.15.45:0/887947525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 msgr2=0x5649703651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 0x5649703651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 -- 172.21.15.45:0/887947525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 msgr2=0x5649703589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 -- 172.21.15.45:0/887947525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649700917d0 con 0x56497034eea0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571290640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 0x5649703651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa571a91640 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa56802a940 tx=0x7fa568031900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988c10f2c52ba743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa5577fe640 1 -- 172.21.15.45:0/887947525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56804c070 con 0x56497034eea0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa5577fe640 1 -- 172.21.15.45:0/887947525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa568048420 con 0x56497034eea0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.817+0000 7fa5577fe640 1 -- 172.21.15.45:0/887947525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568035040 con 0x56497034eea0 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- 172.21.15.45:0/887947525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 msgr2=0x564970350560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa56802a940 tx=0x7fa568031900 comp rx=0 tx=0).stop 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- 172.21.15.45:0/887947525 shutdown_connections 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564970359570 0x5649703589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034eea0 0x564970350560 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 --2- 172.21.15.45:0/887947525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56497034e580 0x5649703651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- 172.21.15.45:0/887947525 >> 172.21.15.45:0/887947525 conn(0x56497034a8a0 msgr2=0x5649703492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- 172.21.15.45:0/887947525 shutdown_connections 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- 172.21.15.45:0/887947525 wait complete. 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 -- start start 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.818+0000 7fa576a01740 1 Processor -- start 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x5649703619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1b2230 con 0x56497034e580 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1b1b60 con 0x56497034eea0 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1aeda0 con 0x564970359570 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa576a01740 10 client.0 _ll_register_callbacks cb 0x7fa55c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51252/0 (socket says 172.21.15.45:51252) 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 -- 172.21.15.45:0/2499357653 learned_addr learned my addr 172.21.15.45:0/2499357653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571a91640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x5649703619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 -- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 msgr2=0x56497035ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 -- 172.21.15.45:0/2499357653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 msgr2=0x5649703619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x5649703619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 -- 172.21.15.45:0/2499357653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497035d280 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa571290640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003bc0 tx=0x7fa55c0133e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29445d7be99a2779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.819+0000 7fa5557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c005ab0 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa568048420 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fa55c02c430 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa55c003900 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 10 client.17187 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649703cf280 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa568048420 con 0x56497034e580 2024-04-25T00:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa55c01e040 con 0x56497034e580 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 10 client.17187 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 10 client.17187 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 20 client.17187 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703cf750 0x5649703d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 4 client.17187 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.820+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649702baf00 con 0x5649703cf750 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa571a91640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703cf750 0x5649703d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa571a91640 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703cf750 0x5649703d1b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b10cf89250f594a6 server_cookie=224eec76cdd1a43e in_seq=0 out_seq=0 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa5557fa640 10 client.17187 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa5557fa640 1 -- 172.21.15.45:0/2499357653 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649702baf00 con 0x5649703cf750 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa5557fa640 10 client.17187 handle_command_reply: tid=0 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 client.17187 shutdown 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 msgr2=0x5649703614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa55c003bc0 tx=0x7fa55c0133e0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 shutdown_connections 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5649703cf750 0x5649703d1b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564970359570 0x56497035ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56497034eea0 0x5649703619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 --2- 172.21.15.45:0/2499357653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56497034e580 0x5649703614a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa5557fa640 0 client.17187 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 >> 172.21.15.45:0/2499357653 conn(0x56497034a8a0 msgr2=0x5649703492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 shutdown_connections 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 1 -- 172.21.15.45:0/2499357653 wait complete. 2024-04-25T00:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.821+0000 7fa576a01740 20 client.17187 trim_cache size 0 max 16384 2024-04-25T00:27:27.831 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:27:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 msgr2=0x7fa5580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 0x7fa5580a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cd690 tx=0x7fa5640912f0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 msgr2=0x7fa56c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa56400eb10 tx=0x7fa56400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 shutdown_connections 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa5580a2d00 0x7fa5580a51c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56c109990 0x7fa56c19eb60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa56c104340 0x7fa56c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 --2- 172.21.15.45:0/108765501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa56c103980 0x7fa56c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 >> 172.21.15.45:0/108765501 conn(0x7fa56c0ff4d0 msgr2=0x7fa56c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 shutdown_connections 2024-04-25T00:27:27.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.822+0000 7fa573cf5640 1 -- 172.21.15.45:0/108765501 wait complete. 2024-04-25T00:27:27.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:27:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 -- 172.21.15.45:0/254124372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a0106aa0 msgr2=0x7f93a010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 --2- 172.21.15.45:0/254124372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a0106aa0 0x7f93a010e6b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9398009ca0 tx=0x7f939802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 -- 172.21.15.45:0/254124372 shutdown_connections 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 --2- 172.21.15.45:0/254124372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a0106aa0 0x7f93a010e6b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 --2- 172.21.15.45:0/254124372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0100c90 0x7f93a01063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 --2- 172.21.15.45:0/254124372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a01002e0 0x7f93a01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 -- 172.21.15.45:0/254124372 >> 172.21.15.45:0/254124372 conn(0x7f93a00758d0 msgr2=0x7f93a0075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 -- 172.21.15.45:0/254124372 shutdown_connections 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.936+0000 7f93a7909640 1 -- 172.21.15.45:0/254124372 wait complete. 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 Processor -- start 2024-04-25T00:27:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 -- start start 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0106aa0 0x7f93a01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a019c950 0x7f93a019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0111b90 con 0x7f93a01002e0 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0111a10 con 0x7f93a019c950 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a7909640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0111d10 con 0x7f93a0106aa0 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a56a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a56a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51284/0 (socket says 172.21.15.45:51284) 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a56a5640 1 -- 172.21.15.45:0/439469372 learned_addr learned my addr 172.21.15.45:0/439469372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.937+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0106aa0 0x7f93a01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a019c950 0x7f93a019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 -- 172.21.15.45:0/439469372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0106aa0 msgr2=0x7f93a01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0106aa0 0x7f93a01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 -- 172.21.15.45:0/439469372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a019c950 msgr2=0x7f93a019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a019c950 0x7f93a019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 -- 172.21.15.45:0/439469372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a019d440 con 0x7f93a01002e0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a56a5640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f939c00e9f0 tx=0x7f939c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93967fc640 1 -- 172.21.15.45:0/439469372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c00cd80 con 0x7f93a01002e0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93967fc640 1 -- 172.21.15.45:0/439469372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939c010910 con 0x7f93a01002e0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a019d730 con 0x7f93a01002e0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93967fc640 1 -- 172.21.15.45:0/439469372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c018990 con 0x7f93a01002e0 2024-04-25T00:27:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.938+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01a4f70 con 0x7f93a01002e0 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93967fc640 1 -- 172.21.15.45:0/439469372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f939c010ac0 con 0x7f93a01002e0 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 Processor -- start 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 -- start start 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2ea0 0x5634ac9ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634ac9af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5634ac907670 con 0x5634ac9a2580 2024-04-25T00:27:27.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5634ac8af970 con 0x5634ac9ad570 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93aa613740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634ac903630 con 0x5634ac9a2ea0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93967fc640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 0x7f938c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51290/0 (socket says 172.21.15.45:51290) 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.941+0000 7f93a5ea6640 1 -- 172.21.15.45:0/590478837 learned_addr learned my addr 172.21.15.45:0/590478837 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634ac9af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a56a5640 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2ea0 0x5634ac9ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 0x7f938c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93967fc640 1 -- 172.21.15.45:0/439469372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f939c0cfa80 con 0x7f93a01002e0 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 -- 172.21.15.45:0/590478837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2ea0 msgr2=0x5634ac9ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2ea0 0x5634ac9ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 -- 172.21.15.45:0/590478837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 msgr2=0x5634ac9af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634ac9af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 -- 172.21.15.45:0/590478837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ac8d3400 con 0x5634ac9a2580 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9398009430 tx=0x7f9398009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c202391737de9d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f938b7fe640 1 -- 172.21.15.45:0/590478837 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398009800 con 0x5634ac9a2580 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f938b7fe640 1 -- 172.21.15.45:0/590478837 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939804f070 con 0x5634ac9a2580 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f938b7fe640 1 -- 172.21.15.45:0/590478837 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939802fc70 con 0x5634ac9a2580 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.942+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 0x7f938c0a5240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f939804e000 tx=0x7f939804c000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- 172.21.15.45:0/590478837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 msgr2=0x5634ac9a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9398009430 tx=0x7f9398009a20 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- 172.21.15.45:0/590478837 shutdown_connections 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634ac9af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2ea0 0x5634ac9ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 --2- 172.21.15.45:0/590478837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2580 0x5634ac9a2960 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- 172.21.15.45:0/590478837 >> 172.21.15.45:0/590478837 conn(0x5634ac99e8a0 msgr2=0x5634ac99ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- 172.21.15.45:0/590478837 shutdown_connections 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- 172.21.15.45:0/590478837 wait complete. 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 -- start start 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.943+0000 7f93aa613740 1 Processor -- start 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5634ac9b01e0 con 0x5634ac9ad570 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5634ac8c3050 con 0x5634ac9a2ea0 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634ac9b0360 con 0x5634ac9a2580 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93aa613740 10 client.0 _ll_register_callbacks cb 0x5634ac61abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51306/0 (socket says 172.21.15.45:51306) 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 -- 172.21.15.45:0/1445913841 learned_addr learned my addr 172.21.15.45:0/1445913841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 -- 172.21.15.45:0/1445913841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 msgr2=0x5634ac9acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 -- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 msgr2=0x5634ac9aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9aa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 -- 172.21.15.45:0/1445913841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ac894dc0 con 0x5634ac9ad570 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9398005e30 tx=0x7f939802f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d647dfd8ba24ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.944+0000 7f9388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939805c060 con 0x5634ac9ad570 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f93980305c0 con 0x5634ac9ad570 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939804f070 con 0x5634ac9ad570 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9398063050 con 0x5634ac9ad570 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 10 client.17199 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634ac61b3f0 con 0x5634ac9ad570 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939804f070 con 0x5634ac9ad570 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9398030ca0 con 0x5634ac9ad570 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 10 client.17199 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 10 client.17199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 20 client.17199 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634ac9bd310 0x5634ac9bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 4 client.17199 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.945+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634ac914bb0 con 0x5634ac9bd310 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.946+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634ac9bd310 0x5634ac9bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.946+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634ac9bd310 0x5634ac9bf700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f13cd4f3dcdd58 server_cookie=3bcd2d43477b2c4 in_seq=0 out_seq=0 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.946+0000 7f9388ff9640 10 client.17199 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f9388ff9640 1 -- 172.21.15.45:0/1445913841 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634ac914bb0 con 0x5634ac9bd310 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f9388ff9640 10 client.17199 handle_command_reply: tid=0 2024-04-25T00:27:27.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 client.17199 shutdown 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 msgr2=0x5634ac9aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9398005e30 tx=0x7f939802f4b0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 shutdown_connections 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634ac9bd310 0x5634ac9bf700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9aa6d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f9388ff9640 0 client.17199 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 --2- 172.21.15.45:0/1445913841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 >> 172.21.15.45:0/1445913841 conn(0x5634ac99e8a0 msgr2=0x5634ac99db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 shutdown_connections 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 1 -- 172.21.15.45:0/1445913841 wait complete. 2024-04-25T00:27:27.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.947+0000 7f93aa613740 20 client.17199 trim_cache size 0 max 16384 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 Processor -- start 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 -- start start 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01aeb10 con 0x5634ac9ad570 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b0650 con 0x5634ac9a2ea0 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.960+0000 7f93aa613740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01af1e0 con 0x5634ac9a2580 2024-04-25T00:27:27.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51316/0 (socket says 172.21.15.45:51316) 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 -- 172.21.15.45:0/2744303981 learned_addr learned my addr 172.21.15.45:0/2744303981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a56a5640 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 -- 172.21.15.45:0/2744303981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 msgr2=0x5634ac9ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 -- 172.21.15.45:0/2744303981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 msgr2=0x5634ac9b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 -- 172.21.15.45:0/2744303981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ac9bfa70 con 0x5634ac9ad570 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9398006010 tx=0x7f9398009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5ae6f79679c58e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f938affd640 1 -- 172.21.15.45:0/2744303981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939805c070 con 0x5634ac9ad570 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f938affd640 1 -- 172.21.15.45:0/2744303981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939804f070 con 0x5634ac9ad570 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.961+0000 7f938affd640 1 -- 172.21.15.45:0/2744303981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980319a0 con 0x5634ac9ad570 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- 172.21.15.45:0/2744303981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 msgr2=0x5634ac9b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9398006010 tx=0x7f9398009360 comp rx=0 tx=0).stop 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- 172.21.15.45:0/2744303981 shutdown_connections 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9ad570 0x5634ac9b5b50 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 --2- 172.21.15.45:0/2744303981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- 172.21.15.45:0/2744303981 >> 172.21.15.45:0/2744303981 conn(0x5634ac99e8a0 msgr2=0x5634ac9b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- 172.21.15.45:0/2744303981 shutdown_connections 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- 172.21.15.45:0/2744303981 wait complete. 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 -- start start 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.962+0000 7f93aa613740 1 Processor -- start 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634aca227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01aff80 con 0x5634ac9a2ea0 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01af8b0 con 0x5634ac9ad570 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01acaf0 con 0x5634ac9a2580 2024-04-25T00:27:27.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 10 client.0 _ll_register_callbacks cb 0x7f939000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51320/0 (socket says 172.21.15.45:51320) 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 -- 172.21.15.45:0/1069020324 learned_addr learned my addr 172.21.15.45:0/1069020324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a5ea6640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634aca227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 -- 172.21.15.45:0/1069020324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 msgr2=0x5634ac9b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 -- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 msgr2=0x5634aca227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634aca227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 -- 172.21.15.45:0/1069020324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634ac9a9140 con 0x5634ac9a2ea0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93a4ea4640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9390005fd0 tx=0x7f9390012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5005b400103795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f9388ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:27.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390021030 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f9390010c30 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.963+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939804f070 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f939002a050 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 10 client.17205 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634ac6e57d0 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939804f070 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93900047a0 con 0x5634ac9a2ea0 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 10 client.17205 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 10 client.17205 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 20 client.17205 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634aca24380 0x5634aca267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 4 client.17205 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634ac90ef00 con 0x5634aca24380 2024-04-25T00:27:27.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.964+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634aca24380 0x5634aca267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93a56a5640 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634aca24380 0x5634aca267a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=893966da0a3ba151 server_cookie=716ff73e15e3ee86 in_seq=0 out_seq=0 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f9388ff9640 10 client.17205 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f9388ff9640 1 -- 172.21.15.45:0/1069020324 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634ac90ef00 con 0x5634aca24380 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f9388ff9640 10 client.17205 handle_command_reply: tid=0 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 client.17205 shutdown 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 msgr2=0x5634ac9b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9390005fd0 tx=0x7f9390012b60 comp rx=0 tx=0).stop 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 shutdown_connections 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5634aca24380 0x5634aca267a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634ac9ad570 0x5634aca227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5634ac9a2ea0 0x5634ac9b3bd0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f9388ff9640 0 client.17205 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 --2- 172.21.15.45:0/1069020324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634ac9a2580 0x5634ac9b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 >> 172.21.15.45:0/1069020324 conn(0x5634ac99e8a0 msgr2=0x5634ac9b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 shutdown_connections 2024-04-25T00:27:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 1 -- 172.21.15.45:0/1069020324 wait complete. 2024-04-25T00:27:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.965+0000 7f93aa613740 20 client.17205 trim_cache size 0 max 16384 2024-04-25T00:27:27.972 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 msgr2=0x7f938c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 0x7f938c0a5240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f939804e000 tx=0x7f939804c000 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 msgr2=0x7f93a0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f939c00e9f0 tx=0x7f939c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 shutdown_connections 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f938c0a2d80 0x7f938c0a5240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a019c950 0x7f93a019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93a0106aa0 0x7f93a01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 --2- 172.21.15.45:0/439469372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a01002e0 0x7f93a0198090 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 >> 172.21.15.45:0/439469372 conn(0x7f93a00758d0 msgr2=0x7f93a00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 shutdown_connections 2024-04-25T00:27:27.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:27.966+0000 7f93a7909640 1 -- 172.21.15.45:0/439469372 wait complete. 2024-04-25T00:27:27.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1478320604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:27.977 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:27:28.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1478320604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 -- 172.21.15.45:0/4133054892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 msgr2=0x7fc28c10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 --2- 172.21.15.45:0/4133054892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c10e5e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc284009cd0 tx=0x7fc28402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 -- 172.21.15.45:0/4133054892 shutdown_connections 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 --2- 172.21.15.45:0/4133054892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c10e5e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 --2- 172.21.15.45:0/4133054892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c101710 0x7fc28c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 --2- 172.21.15.45:0/4133054892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c107720 0x7fc28c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 -- 172.21.15.45:0/4133054892 >> 172.21.15.45:0/4133054892 conn(0x7fc28c0fd180 msgr2=0x7fc28c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 -- 172.21.15.45:0/4133054892 shutdown_connections 2024-04-25T00:27:28.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.080+0000 7fc29109e640 1 -- 172.21.15.45:0/4133054892 wait complete. 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 Processor -- start 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 -- start start 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 0x7fc28c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c107720 0x7fc28c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111b30 con 0x7fc28c1020d0 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1119b0 con 0x7fc28c107720 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc29109e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111cb0 con 0x7fc28c101710 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51334/0 (socket says 172.21.15.45:51334) 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 -- 172.21.15.45:0/2115998157 learned_addr learned my addr 172.21.15.45:0/2115998157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 0x7fc28c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc28b59e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c107720 0x7fc28c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 -- 172.21.15.45:0/2115998157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 msgr2=0x7fc28c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 0x7fc28c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 -- 172.21.15.45:0/2115998157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c107720 msgr2=0x7fc28c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c107720 0x7fc28c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc283fff640 1 -- 172.21.15.45:0/2115998157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c19cfe0 con 0x7fc28c1020d0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.081+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 0x7fc28c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc283fff640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d950 tx=0x7fc27400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc288d99640 1 -- 172.21.15.45:0/2115998157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274017070 con 0x7fc28c1020d0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c19d2d0 con 0x7fc28c1020d0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc288d99640 1 -- 172.21.15.45:0/2115998157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc274009c30 con 0x7fc28c1020d0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc288d99640 1 -- 172.21.15.45:0/2115998157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27400f6b0 con 0x7fc28c1020d0 2024-04-25T00:27:28.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.082+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a4b60 con 0x7fc28c1020d0 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc288d99640 1 -- 172.21.15.45:0/2115998157 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc2740109d0 con 0x7fc28c1020d0 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 Processor -- start 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 -- start start 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a5b580 0x55a876a5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 0x55a876a68960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a8769c0670 con 0x55a876a5bea0 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a876968970 con 0x55a876a66570 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc293daf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a8769bc630 con 0x55a876a5b580 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51338/0 (socket says 172.21.15.45:51338) 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc28ad9d640 1 -- 172.21.15.45:0/4181972472 learned_addr learned my addr 172.21.15.45:0/4181972472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.085+0000 7fc288d99640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 0x7fc2780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28b59e640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a5b580 0x55a876a5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc283fff640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 0x55a876a68960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 -- 172.21.15.45:0/4181972472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a5b580 msgr2=0x55a876a5b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a5b580 0x55a876a5b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 -- 172.21.15.45:0/4181972472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 msgr2=0x55a876a68960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 0x55a876a68960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28b59e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 0x7fc2780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 -- 172.21.15.45:0/4181972472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87698c400 con 0x55a876a5bea0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc283fff640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 0x55a876a68960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc288d99640 1 -- 172.21.15.45:0/2115998157 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc2740cf470 con 0x7fc28c1020d0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00cbf0 tx=0x7fc27c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671ee4fb1e1c5393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc2817fa640 1 -- 172.21.15.45:0/4181972472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c005220 con 0x55a876a5bea0 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc28b59e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 0x7fc2780a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc284009430 tx=0x7fc2840095c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc2817fa640 1 -- 172.21.15.45:0/4181972472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c016070 con 0x55a876a5bea0 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.086+0000 7fc2817fa640 1 -- 172.21.15.45:0/4181972472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c010690 con 0x55a876a5bea0 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- 172.21.15.45:0/4181972472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 msgr2=0x55a876a66030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00cbf0 tx=0x7fc27c00e720 comp rx=0 tx=0).stop 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- 172.21.15.45:0/4181972472 shutdown_connections 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a66570 0x55a876a68960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a66030 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 --2- 172.21.15.45:0/4181972472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a5b580 0x55a876a5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- 172.21.15.45:0/4181972472 >> 172.21.15.45:0/4181972472 conn(0x55a876a578a0 msgr2=0x55a876a57c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- 172.21.15.45:0/4181972472 shutdown_connections 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- 172.21.15.45:0/4181972472 wait complete. 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 -- start start 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.087+0000 7fc293daf740 1 Processor -- start 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a876a69360 con 0x55a876a5b580 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a87697c050 con 0x55a876a5bea0 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a876a694e0 con 0x55a876a66570 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 10 client.0 _ll_register_callbacks cb 0x55a8766d3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51350/0 (socket says 172.21.15.45:51350) 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 -- 172.21.15.45:0/1824295102 learned_addr learned my addr 172.21.15.45:0/1824295102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc283fff640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28b59e640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 -- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 msgr2=0x55a876a636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a636b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 -- 172.21.15.45:0/1824295102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 msgr2=0x55a876a63170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a63170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 -- 172.21.15.45:0/1824295102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87694ddc0 con 0x55a876a5b580 2024-04-25T00:27:28.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc283fff640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d70 tx=0x7fc27c010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da80cd571264c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc2727fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c02a060 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc27c010d50 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.088+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c016070 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc27c031050 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 10 client.17217 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8766d43f0 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c016070 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc27c00f4e0 con 0x55a876a5b580 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 10 client.17217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:27:28.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 20 client.17217 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876a76500 0x55a876a788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 4 client.17217 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8769cdbb0 con 0x55a876a76500 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.089+0000 7fc283fff640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876a76500 0x55a876a788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.090+0000 7fc283fff640 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876a76500 0x55a876a788f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cbf6e7344dc6d9f server_cookie=f607e7b1a9dcddf5 in_seq=0 out_seq=0 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.090+0000 7fc2727fc640 10 client.17217 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.090+0000 7fc2727fc640 1 -- 172.21.15.45:0/1824295102 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8769cdbb0 con 0x55a876a76500 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc2727fc640 10 client.17217 handle_command_reply: tid=0 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 client.17217 shutdown 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 msgr2=0x55a876a65e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d70 tx=0x7fc27c010bb0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 shutdown_connections 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876a76500 0x55a876a788f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc2727fc640 0 client.17217 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 --2- 172.21.15.45:0/1824295102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a65e80 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 >> 172.21.15.45:0/1824295102 conn(0x55a876a578a0 msgr2=0x55a876a56b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 shutdown_connections 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 1 -- 172.21.15.45:0/1824295102 wait complete. 2024-04-25T00:27:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.091+0000 7fc293daf740 20 client.17217 trim_cache size 0 max 16384 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 Processor -- start 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 -- start start 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a6eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1aeb00 con 0x55a876a5b580 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.104+0000 7fc293daf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1b0640 con 0x55a876a5bea0 2024-04-25T00:27:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc293daf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1af1d0 con 0x55a876a66570 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51354/0 (socket says 172.21.15.45:51354) 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 -- 172.21.15.45:0/2273573239 learned_addr learned my addr 172.21.15.45:0/2273573239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28b59e640 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 -- 172.21.15.45:0/2273573239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 msgr2=0x55a876a659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 -- 172.21.15.45:0/2273573239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 msgr2=0x55a876a6eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a6eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 -- 172.21.15.45:0/2273573239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87679e7d0 con 0x55a876a5b580 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d40 tx=0x7fc27c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64776bfc2706242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc280ff9640 1 -- 172.21.15.45:0/2273573239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c02a040 con 0x55a876a5b580 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc280ff9640 1 -- 172.21.15.45:0/2273573239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c016070 con 0x55a876a5b580 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.105+0000 7fc280ff9640 1 -- 172.21.15.45:0/2273573239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0257f0 con 0x55a876a5b580 2024-04-25T00:27:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- 172.21.15.45:0/2273573239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 msgr2=0x55a876a72250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d40 tx=0x7fc27c00c490 comp rx=0 tx=0).stop 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- 172.21.15.45:0/2273573239 shutdown_connections 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5bea0 0x55a876a6eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 --2- 172.21.15.45:0/2273573239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5b580 0x55a876a72250 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- 172.21.15.45:0/2273573239 >> 172.21.15.45:0/2273573239 conn(0x55a876a578a0 msgr2=0x55a876a5e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- 172.21.15.45:0/2273573239 shutdown_connections 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- 172.21.15.45:0/2273573239 wait complete. 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 -- start start 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.106+0000 7fc293daf740 1 Processor -- start 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5b580 0x55a876a60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a5f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1aff70 con 0x55a876a5bea0 2024-04-25T00:27:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1af8a0 con 0x55a876a5b580 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1acae0 con 0x55a876a66570 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 10 client.0 _ll_register_callbacks cb 0x7fc27c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51370/0 (socket says 172.21.15.45:51370) 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 -- 172.21.15.45:0/1072373940 learned_addr learned my addr 172.21.15.45:0/1072373940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5b580 0x55a876a60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc28b59e640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a5f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 -- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 msgr2=0x55a876a5f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a5f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 -- 172.21.15.45:0/1072373940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5b580 msgr2=0x55a876a60900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5b580 0x55a876a60900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 -- 172.21.15.45:0/1072373940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a876a5fd10 con 0x55a876a5bea0 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc28b59e640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a5f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc283fff640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc274098000 tx=0x7fc27406b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e3ce015a466638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc2727fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274020040 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fc2740ccae0 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c016070 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2740666c0 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 10 client.17223 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.107+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a876ae1440 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c016070 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc274097ca0 con 0x55a876a5bea0 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 10 client.17223 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 10 client.17223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 20 client.17223 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876ae1b70 0x55a876ae3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 4 client.17223 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8769c7f00 con 0x55a876ae1b70 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876ae1b70 0x55a876ae3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc28ad9d640 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876ae1b70 0x55a876ae3f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b86cb8eef236f69 server_cookie=71db1a8607e9f9c4 in_seq=0 out_seq=0 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.108+0000 7fc2727fc640 10 client.17223 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc2727fc640 1 -- 172.21.15.45:0/1072373940 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8769c7f00 con 0x55a876ae1b70 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc2727fc640 10 client.17223 handle_command_reply: tid=0 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 client.17223 shutdown 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 msgr2=0x55a876a60e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc274098000 tx=0x7fc27406b490 comp rx=0 tx=0).stop 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 shutdown_connections 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55a876ae1b70 0x55a876ae3f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a876a66570 0x55a876a5f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc2727fc640 0 client.17223 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a876a5bea0 0x55a876a60e40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 --2- 172.21.15.45:0/1072373940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a876a5b580 0x55a876a60900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 >> 172.21.15.45:0/1072373940 conn(0x55a876a578a0 msgr2=0x55a876a562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 shutdown_connections 2024-04-25T00:27:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 1 -- 172.21.15.45:0/1072373940 wait complete. 2024-04-25T00:27:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc293daf740 20 client.17223 trim_cache size 0 max 16384 2024-04-25T00:27:28.117 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 msgr2=0x7fc2780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 0x7fc2780a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc284009430 tx=0x7fc2840095c0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 msgr2=0x7fc28c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.109+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d950 tx=0x7fc27400de10 comp rx=0 tx=0).stop 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 shutdown_connections 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2780a2d00 0x7fc2780a51c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c107720 0x7fc28c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28c1020d0 0x7fc28c198530 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 --2- 172.21.15.45:0/2115998157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc28c101710 0x7fc28c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 >> 172.21.15.45:0/2115998157 conn(0x7fc28c0fd180 msgr2=0x7fc28c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 shutdown_connections 2024-04-25T00:27:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.110+0000 7fc29109e640 1 -- 172.21.15.45:0/2115998157 wait complete. 2024-04-25T00:27:28.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1478320604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:27:28.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.224+0000 7fb88b658640 1 -- 172.21.15.45:0/3848536687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100070 msgr2=0x7fb8841004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.224+0000 7fb88b658640 1 --2- 172.21.15.45:0/3848536687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100070 0x7fb8841004d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb874009900 tx=0x7fb87402a410 comp rx=0 tx=0).stop 2024-04-25T00:27:28.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.224+0000 7fb88b658640 1 -- 172.21.15.45:0/3848536687 shutdown_connections 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.224+0000 7fb88b658640 1 --2- 172.21.15.45:0/3848536687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884100a10 0x7fb884110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 --2- 172.21.15.45:0/3848536687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100070 0x7fb8841004d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 --2- 172.21.15.45:0/3848536687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884102530 0x7fb884102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- 172.21.15.45:0/3848536687 >> 172.21.15.45:0/3848536687 conn(0x7fb8840758d0 msgr2=0x7fb884075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- 172.21.15.45:0/3848536687 shutdown_connections 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- 172.21.15.45:0/3848536687 wait complete. 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 Processor -- start 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- start start 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884100070 0x7fb88419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884102530 0x7fb88419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb884113e10 con 0x7fb884100a10 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb884113c90 con 0x7fb884100070 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.225+0000 7fb88b658640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb884113f90 con 0x7fb884102530 2024-04-25T00:27:28.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51380/0 (socket says 172.21.15.45:51380) 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 -- 172.21.15.45:0/210394576 learned_addr learned my addr 172.21.15.45:0/210394576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb889bf5640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884102530 0x7fb88419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb8893f4640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884100070 0x7fb88419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 -- 172.21.15.45:0/210394576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884102530 msgr2=0x7fb88419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884102530 0x7fb88419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 -- 172.21.15.45:0/210394576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884100070 msgr2=0x7fb88419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884100070 0x7fb88419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 -- 172.21.15.45:0/210394576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88419f300 con 0x7fb884100a10 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb888bf3640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb8740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb87a7fc640 1 -- 172.21.15.45:0/210394576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87403f030 con 0x7fb884100a10 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb87a7fc640 1 -- 172.21.15.45:0/210394576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb874003ce0 con 0x7fb884100a10 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88419f590 con 0x7fb884100a10 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb87a7fc640 1 -- 172.21.15.45:0/210394576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874035070 con 0x7fb884100a10 2024-04-25T00:27:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.226+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a6e80 con 0x7fb884100a10 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.228+0000 7fb87a7fc640 1 -- 172.21.15.45:0/210394576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb874042030 con 0x7fb884100a10 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 Processor -- start 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 -- start start 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb87a7fc640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 0x7fb8700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d181b12670 con 0x55d181bad580 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d181aba970 con 0x55d181bb8570 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb88e367740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d181b0e630 con 0x55d181badea0 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb889bf5640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 0x7fb8700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb87a7fc640 1 -- 172.21.15.45:0/210394576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb8740efd90 con 0x7fb884100a10 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51388/0 (socket says 172.21.15.45:51388) 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.229+0000 7fb8893f4640 1 -- 172.21.15.45:0/1340181152 learned_addr learned my addr 172.21.15.45:0/1340181152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb888bf3640 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 -- 172.21.15.45:0/1340181152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 msgr2=0x55d181bb8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 -- 172.21.15.45:0/1340181152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 msgr2=0x55d181bba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 -- 172.21.15.45:0/1340181152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d181ade400 con 0x55d181bad580 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb889bf5640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 0x7fb8700a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005fd0 tx=0x7fb87c009ae0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb8893f4640 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb880009c00 tx=0x7fb8800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef39c1a3b67492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb86f7fe640 1 -- 172.21.15.45:0/1340181152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880012450 con 0x55d181bad580 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb86f7fe640 1 -- 172.21.15.45:0/1340181152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880012ad0 con 0x55d181bad580 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.230+0000 7fb86f7fe640 1 -- 172.21.15.45:0/1340181152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000fab0 con 0x55d181bad580 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- 172.21.15.45:0/1340181152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 msgr2=0x55d181bad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb880009c00 tx=0x7fb8800117b0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- 172.21.15.45:0/1340181152 shutdown_connections 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 --2- 172.21.15.45:0/1340181152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bad960 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- 172.21.15.45:0/1340181152 >> 172.21.15.45:0/1340181152 conn(0x55d181ba98a0 msgr2=0x55d181ba9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- 172.21.15.45:0/1340181152 shutdown_connections 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- 172.21.15.45:0/1340181152 wait complete. 2024-04-25T00:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 -- start start 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.231+0000 7fb88e367740 1 Processor -- start 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bad580 0x55d181bb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bb5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d181bbb360 con 0x55d181badea0 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d181ace050 con 0x55d181bad580 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d181bbb4e0 con 0x55d181bb8570 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb88e367740 10 client.0 _ll_register_callbacks cb 0x55d181825be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51402/0 (socket says 172.21.15.45:51402) 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 -- 172.21.15.45:0/4276477226 learned_addr learned my addr 172.21.15.45:0/4276477226 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb889bf5640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bb5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb8893f4640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bad580 0x55d181bb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 -- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 msgr2=0x55d181bb5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bb5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 -- 172.21.15.45:0/4276477226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bad580 msgr2=0x55d181bb7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bad580 0x55d181bb7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 -- 172.21.15.45:0/4276477226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d181a9fdc0 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb888bf3640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b89e0 tx=0x7fb8740b8330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3276a11ccabd2072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.232+0000 7fb86cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874049040 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb8740b8680 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb880012ad0 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8740bb970 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 10 client.17235 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1818263f0 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880012ad0 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb874097070 con 0x55d181badea0 2024-04-25T00:27:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 10 client.17235 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 10 client.17235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 20 client.17235 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181bc8530 0x55d181bca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 4 client.17235 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.233+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d181b1fbb0 con 0x55d181bc8530 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.234+0000 7fb8893f4640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181bc8530 0x55d181bca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.234+0000 7fb8893f4640 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181bc8530 0x55d181bca9f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4bd983863b6027b server_cookie=46d855f82e8810f4 in_seq=0 out_seq=0 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.234+0000 7fb86cff9640 10 client.17235 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb86cff9640 1 -- 172.21.15.45:0/4276477226 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d181b1fbb0 con 0x55d181bc8530 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb86cff9640 10 client.17235 handle_command_reply: tid=0 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 client.17235 shutdown 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 msgr2=0x55d181bb5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b89e0 tx=0x7fb8740b8330 comp rx=0 tx=0).stop 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 shutdown_connections 2024-04-25T00:27:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181bc8530 0x55d181bca9f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bb5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb86cff9640 0 client.17235 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181badea0 0x55d181bb5140 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 --2- 172.21.15.45:0/4276477226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bad580 0x55d181bb7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 >> 172.21.15.45:0/4276477226 conn(0x55d181ba98a0 msgr2=0x55d181ba8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 shutdown_connections 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 1 -- 172.21.15.45:0/4276477226 wait complete. 2024-04-25T00:27:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.235+0000 7fb88e367740 20 client.17235 trim_cache size 0 max 16384 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.248+0000 7fb88e367740 1 Processor -- start 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.248+0000 7fb88e367740 1 -- start start 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bd0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b0d00 con 0x55d181bad580 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b2840 con 0x55d181bb8570 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb88e367740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b13d0 con 0x55d181badea0 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51404/0 (socket says 172.21.15.45:51404) 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 -- 172.21.15.45:0/2408975807 learned_addr learned my addr 172.21.15.45:0/2408975807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb889bf5640 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bd0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb888bf3640 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 -- 172.21.15.45:0/2408975807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 msgr2=0x55d181bb7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 -- 172.21.15.45:0/2408975807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 msgr2=0x55d181bd0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bd0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 -- 172.21.15.45:0/2408975807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1818f07d0 con 0x55d181bad580 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb880000c00 tx=0x7fb880005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d044e41874a5deb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb86effd640 1 -- 172.21.15.45:0/2408975807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880009630 con 0x55d181bad580 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb86effd640 1 -- 172.21.15.45:0/2408975807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880005520 con 0x55d181bad580 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.249+0000 7fb86effd640 1 -- 172.21.15.45:0/2408975807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800035f0 con 0x55d181bad580 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- 172.21.15.45:0/2408975807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 msgr2=0x55d181bb0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb880000c00 tx=0x7fb880005f70 comp rx=0 tx=0).stop 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- 172.21.15.45:0/2408975807 shutdown_connections 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181bb8570 0x55d181bd0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181badea0 0x55d181bb7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 --2- 172.21.15.45:0/2408975807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bb0860 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- 172.21.15.45:0/2408975807 >> 172.21.15.45:0/2408975807 conn(0x55d181ba98a0 msgr2=0x55d181ba82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- 172.21.15.45:0/2408975807 shutdown_connections 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- 172.21.15.45:0/2408975807 wait complete. 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 -- start start 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.250+0000 7fb88e367740 1 Processor -- start 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 0x55d181bbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bbc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b2170 con 0x55d181bad580 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b1aa0 con 0x55d181badea0 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841aece0 con 0x55d181bb8570 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb88e367740 10 client.0 _ll_register_callbacks cb 0x7fb87403b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51414/0 (socket says 172.21.15.45:51414) 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 -- 172.21.15.45:0/2689126184 learned_addr learned my addr 172.21.15.45:0/2689126184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb888bf3640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 0x55d181bbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 -- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 msgr2=0x55d181bbc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb889bf5640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bbc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bbc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 -- 172.21.15.45:0/2689126184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 msgr2=0x55d181bbbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 0x55d181bbbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 -- 172.21.15.45:0/2689126184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d181bbcdf0 con 0x55d181bad580 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb888bf3640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 0x55d181bbbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb8893f4640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb880005f40 tx=0x7fb880009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05e8f213e96fad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb86cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.251+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800052b0 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fb88000f720 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb880005520 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb88000f920 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 10 client.17241 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d181c2ec70 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880005520 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb88002c070 con 0x55d181bad580 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 10 client.17241 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 10 client.17241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 20 client.17241 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181c2f190 0x55d181c31580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 4 client.17241 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d181b19f00 con 0x55d181c2f190 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.252+0000 7fb888bf3640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181c2f190 0x55d181c31580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb888bf3640 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181c2f190 0x55d181c31580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=970ae291bb189aae server_cookie=61e9d100738b16ad in_seq=0 out_seq=0 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb86cff9640 10 client.17241 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb86cff9640 1 -- 172.21.15.45:0/2689126184 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d181b19f00 con 0x55d181c2f190 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb86cff9640 10 client.17241 handle_command_reply: tid=0 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 client.17241 shutdown 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 msgr2=0x55d181bc0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb880005f40 tx=0x7fb880009b30 comp rx=0 tx=0).stop 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 shutdown_connections 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d181c2f190 0x55d181c31580 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d181bb8570 0x55d181bbc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d181badea0 0x55d181bbbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb86cff9640 0 client.17241 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 --2- 172.21.15.45:0/2689126184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d181bad580 0x55d181bc0c20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 >> 172.21.15.45:0/2689126184 conn(0x55d181ba98a0 msgr2=0x55d181bc3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 shutdown_connections 2024-04-25T00:27:28.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 1 -- 172.21.15.45:0/2689126184 wait complete. 2024-04-25T00:27:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.253+0000 7fb88e367740 20 client.17241 trim_cache size 0 max 16384 2024-04-25T00:27:28.261 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 msgr2=0x7fb8700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 0x7fb8700a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005fd0 tx=0x7fb87c009ae0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 msgr2=0x7fb88419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb874006010 tx=0x7fb8740042f0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 shutdown_connections 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8700a2d00 0x7fb8700a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb884102530 0x7fb88419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb884100a10 0x7fb88419a850 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 --2- 172.21.15.45:0/210394576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb884100070 0x7fb88419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 >> 172.21.15.45:0/210394576 conn(0x7fb8840758d0 msgr2=0x7fb8840fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 shutdown_connections 2024-04-25T00:27:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.254+0000 7fb88b658640 1 -- 172.21.15.45:0/210394576 wait complete. 2024-04-25T00:27:28.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 -- 172.21.15.45:0/989713120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 msgr2=0x7ff5cc101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 --2- 172.21.15.45:0/989713120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc101b90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009930 tx=0x7ff5bc02a420 comp rx=0 tx=0).stop 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 -- 172.21.15.45:0/989713120 shutdown_connections 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 --2- 172.21.15.45:0/989713120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 --2- 172.21.15.45:0/989713120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc101b90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 --2- 172.21.15.45:0/989713120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc107720 0x7ff5cc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 -- 172.21.15.45:0/989713120 >> 172.21.15.45:0/989713120 conn(0x7ff5cc0fd180 msgr2=0x7ff5cc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 -- 172.21.15.45:0/989713120 shutdown_connections 2024-04-25T00:27:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.370+0000 7ff5d457e640 1 -- 172.21.15.45:0/989713120 wait complete. 2024-04-25T00:27:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 Processor -- start 2024-04-25T00:27:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 -- start start 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc107720 0x7ff5cc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111af0 con 0x7ff5cc101710 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111970 con 0x7ff5cc1020d0 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d457e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111c70 con 0x7ff5cc107720 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51430/0 (socket says 172.21.15.45:51430) 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 -- 172.21.15.45:0/874113763 learned_addr learned my addr 172.21.15.45:0/874113763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc107720 0x7ff5cc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 -- 172.21.15.45:0/874113763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc107720 msgr2=0x7ff5cc19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc107720 0x7ff5cc19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 -- 172.21.15.45:0/874113763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 msgr2=0x7ff5cc1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 -- 172.21.15.45:0/874113763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc19d080 con 0x7ff5cc101710 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.371+0000 7ff5d231a640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800e9f0 tx=0x7ff5c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.372+0000 7ff5c37fe640 1 -- 172.21.15.45:0/874113763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800cd80 con 0x7ff5cc101710 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.372+0000 7ff5c37fe640 1 -- 172.21.15.45:0/874113763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8010910 con 0x7ff5cc101710 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.372+0000 7ff5c37fe640 1 -- 172.21.15.45:0/874113763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8018990 con 0x7ff5cc101710 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.372+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc19d370 con 0x7ff5cc101710 2024-04-25T00:27:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.372+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1a4c00 con 0x7ff5cc101710 2024-04-25T00:27:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.374+0000 7ff5c37fe640 1 -- 172.21.15.45:0/874113763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff5c8010ac0 con 0x7ff5cc101710 2024-04-25T00:27:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.374+0000 7ff5d7369740 1 Processor -- start 2024-04-25T00:27:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 -- start start 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56409c836670 con 0x56409c8d1580 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56409c7de970 con 0x56409c8dc570 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d7369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56409c832630 con 0x56409c8d1ea0 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5c37fe640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 0x7ff5b80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51444/0 (socket says 172.21.15.45:51444) 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/4159984691 learned_addr learned my addr 172.21.15.45:0/4159984691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 0x7ff5b80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d231a640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5c37fe640 1 -- 172.21.15.45:0/874113763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff5c80cfb50 con 0x7ff5cc101710 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/4159984691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 msgr2=0x56409c8dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/4159984691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 msgr2=0x56409c8de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/4159984691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409c802400 con 0x56409c8d1580 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.375+0000 7ff5d231a640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4012a70 tx=0x7ff5c4012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9b02bf6b3aa127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5c0bfe640 1 -- 172.21.15.45:0/4159984691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40124b0 con 0x56409c8d1580 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5c0bfe640 1 -- 172.21.15.45:0/4159984691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4022070 con 0x56409c8d1580 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5c0bfe640 1 -- 172.21.15.45:0/4159984691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4005580 con 0x56409c8d1580 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 0x7ff5b80a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4021000 tx=0x7ff5c401d000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 -- 172.21.15.45:0/4159984691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 msgr2=0x56409c8d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4012a70 tx=0x7ff5c4012f30 comp rx=0 tx=0).stop 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 -- 172.21.15.45:0/4159984691 shutdown_connections 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 --2- 172.21.15.45:0/4159984691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1580 0x56409c8d1960 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.376+0000 7ff5d7369740 1 -- 172.21.15.45:0/4159984691 >> 172.21.15.45:0/4159984691 conn(0x56409c8cd8a0 msgr2=0x56409c8cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- 172.21.15.45:0/4159984691 shutdown_connections 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- 172.21.15.45:0/4159984691 wait complete. 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- start start 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 Processor -- start 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 0x56409c8dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56409c8df360 con 0x56409c8d1ea0 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56409c7f2050 con 0x56409c8dc570 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56409c8df4e0 con 0x56409c8d1580 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d7369740 10 client.0 _ll_register_callbacks cb 0x56409c549be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d1b19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d1b19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51448/0 (socket says 172.21.15.45:51448) 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d1b19640 1 -- 172.21.15.45:0/1556105668 learned_addr learned my addr 172.21.15.45:0/1556105668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d231a640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 0x56409c8dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.377+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 -- 172.21.15.45:0/1556105668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 msgr2=0x56409c8dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 0x56409c8dbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 -- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 msgr2=0x56409c8d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8d96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 -- 172.21.15.45:0/1556105668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409c7c3dc0 con 0x56409c8d1ea0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d231a640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 0x56409c8dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009930 tx=0x7ff5bc0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb222bb094408b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc040070 con 0x56409c8d1ea0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c4022070 con 0x56409c8d1ea0 2024-04-25T00:27:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff5bc004040 con 0x56409c8d1ea0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc04d050 con 0x56409c8d1ea0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d7369740 10 client.17253 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56409c54a3f0 con 0x56409c8d1ea0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4022070 con 0x56409c8d1ea0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5bc0048f0 con 0x56409c8d1ea0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d7369740 10 client.17253 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.378+0000 7ff5d7369740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d7369740 20 client.17253 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c8ec5f0 0x56409c8eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d7369740 4 client.17253 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56409c843bb0 con 0x56409c8ec5f0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d231a640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c8ec5f0 0x56409c8eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5d231a640 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c8ec5f0 0x56409c8eea90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1761fbad661ce24a server_cookie=6f15c025bd9e7587 in_seq=0 out_seq=0 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.379+0000 7ff5b5ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/1556105668 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56409c843bb0 con 0x56409c8ec5f0 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5b5ffb640 10 client.17253 handle_command_reply: tid=0 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 client.17253 shutdown 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 msgr2=0x56409c8d9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009930 tx=0x7ff5bc0315b0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 shutdown_connections 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8dc570 0x56409c8d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c8ec5f0 0x56409c8eea90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8d9190 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 --2- 172.21.15.45:0/1556105668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1580 0x56409c8dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5b5ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 >> 172.21.15.45:0/1556105668 conn(0x56409c8cd8a0 msgr2=0x56409c8d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 shutdown_connections 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 1 -- 172.21.15.45:0/1556105668 wait complete. 2024-04-25T00:27:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.380+0000 7ff5d7369740 20 client.17253 trim_cache size 0 max 16384 2024-04-25T00:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 Processor -- start 2024-04-25T00:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 -- start start 2024-04-25T00:27:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1aed00 con 0x56409c8dc570 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b0840 con 0x56409c8d1580 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d7369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1af3d0 con 0x56409c8d1ea0 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d231a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51450/0 (socket says 172.21.15.45:51450) 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/702224723 learned_addr learned my addr 172.21.15.45:0/702224723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/702224723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 msgr2=0x56409c8d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/702224723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 msgr2=0x56409c8e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.394+0000 7ff5d2b1b640 1 -- 172.21.15.45:0/702224723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409c6147d0 con 0x56409c8dc570 2024-04-25T00:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c4010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f667859033e754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5b7fff640 1 -- 172.21.15.45:0/702224723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4032020 con 0x56409c8dc570 2024-04-25T00:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5b7fff640 1 -- 172.21.15.45:0/702224723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4022050 con 0x56409c8dc570 2024-04-25T00:27:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5b7fff640 1 -- 172.21.15.45:0/702224723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c402a040 con 0x56409c8dc570 2024-04-25T00:27:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 -- 172.21.15.45:0/702224723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 msgr2=0x56409c8db9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005fd0 tx=0x7ff5c4010910 comp rx=0 tx=0).stop 2024-04-25T00:27:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 -- 172.21.15.45:0/702224723 shutdown_connections 2024-04-25T00:27:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8dc570 0x56409c8db9d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8d1ea0 0x56409c8d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 --2- 172.21.15.45:0/702224723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 -- 172.21.15.45:0/702224723 >> 172.21.15.45:0/702224723 conn(0x56409c8cd8a0 msgr2=0x56409c8d7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 -- 172.21.15.45:0/702224723 shutdown_connections 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.395+0000 7ff5d7369740 1 -- 172.21.15.45:0/702224723 wait complete. 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 -- start start 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 Processor -- start 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8dc570 0x56409c8df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b0170 con 0x56409c8d1ea0 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1afaa0 con 0x56409c8d1580 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1acce0 con 0x56409c8dc570 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d7369740 10 client.0 _ll_register_callbacks cb 0x7ff5bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51452/0 (socket says 172.21.15.45:51452) 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 -- 172.21.15.45:0/2698301287 learned_addr learned my addr 172.21.15.45:0/2698301287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d231a640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d2b1b640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8dc570 0x56409c8df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 -- 172.21.15.45:0/2698301287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8dc570 msgr2=0x56409c8df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8dc570 0x56409c8df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 -- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 msgr2=0x56409c8e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.396+0000 7ff5d1b19640 1 -- 172.21.15.45:0/2698301287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409c8e0000 con 0x56409c8d1ea0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d1b19640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005dd0 tx=0x7ff5bc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d352e3a6887c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc040030 con 0x56409c8d1ea0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff5bc0048d0 con 0x56409c8d1ea0 2024-04-25T00:27:28.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c4022050 con 0x56409c8d1ea0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc063050 con 0x56409c8d1ea0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 10 client.17259 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56409c952650 con 0x56409c8d1ea0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c4022050 con 0x56409c8d1ea0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5bc03c560 con 0x56409c8d1ea0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 10 client.17259 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 20 client.17259 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c952e00 0x56409c955220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 4 client.17259 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56409c83df00 con 0x56409c952e00 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.397+0000 7ff5d231a640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c952e00 0x56409c955220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5b5ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:28.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d231a640 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c952e00 0x56409c955220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788625f77c3977c server_cookie=f71279195f52dd71 in_seq=0 out_seq=0 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5b5ffb640 1 -- 172.21.15.45:0/2698301287 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56409c83df00 con 0x56409c952e00 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5b5ffb640 10 client.17259 handle_command_reply: tid=0 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 client.17259 shutdown 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 msgr2=0x56409c8e4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005dd0 tx=0x7ff5bc0035f0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 shutdown_connections 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56409c8dc570 0x56409c8df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56409c952e00 0x56409c955220 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56409c8d1ea0 0x56409c8e4940 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5b5ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 --2- 172.21.15.45:0/2698301287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409c8d1580 0x56409c8e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 >> 172.21.15.45:0/2698301287 conn(0x56409c8cd8a0 msgr2=0x56409c8d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 shutdown_connections 2024-04-25T00:27:28.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 1 -- 172.21.15.45:0/2698301287 wait complete. 2024-04-25T00:27:28.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.398+0000 7ff5d7369740 20 client.17259 trim_cache size 0 max 16384 2024-04-25T00:27:28.406 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 msgr2=0x7ff5b80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 0x7ff5b80a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4021000 tx=0x7ff5c401d000 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 msgr2=0x7ff5cc198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800e9f0 tx=0x7ff5c800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 shutdown_connections 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5b80a2d00 0x7ff5b80a51c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5cc107720 0x7ff5cc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc1020d0 0x7ff5cc1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 --2- 172.21.15.45:0/874113763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc101710 0x7ff5cc198090 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 >> 172.21.15.45:0/874113763 conn(0x7ff5cc0fd180 msgr2=0x7ff5cc0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 shutdown_connections 2024-04-25T00:27:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:28.399+0000 7ff5d457e640 1 -- 172.21.15.45:0/874113763 wait complete. 2024-04-25T00:27:29.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:28 smithi122 ceph-mon[32959]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:29.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:28 smithi071 ceph-mon[32667]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:29.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:28 smithi045 ceph-mon[29194]: pgmap v168: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:29.411 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:27:29.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- 172.21.15.45:0/3126983210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87841032e0 msgr2=0x7f878410b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3126983210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87841032e0 0x7f878410b7c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8774009900 tx=0x7f877402a510 comp rx=0 tx=0).stop 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- 172.21.15.45:0/3126983210 shutdown_connections 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3126983210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f878410bd00 0x7f878410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3126983210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87841032e0 0x7f878410b7c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3126983210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841029c0 0x7f8784102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- 172.21.15.45:0/3126983210 >> 172.21.15.45:0/3126983210 conn(0x7f8784077740 msgr2=0x7f8784077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- 172.21.15.45:0/3126983210 shutdown_connections 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- 172.21.15.45:0/3126983210 wait complete. 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 Processor -- start 2024-04-25T00:27:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.531+0000 7f878ae7d640 1 -- start start 2024-04-25T00:27:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87841029c0 0x7f8784100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 0x7f8784100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8784111b00 con 0x7f878410bd00 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8784111980 con 0x7f87841032e0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878ae7d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8784111c80 con 0x7f87841029c0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51470/0 (socket says 172.21.15.45:51470) 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 -- 172.21.15.45:0/3213851750 learned_addr learned my addr 172.21.15.45:0/3213851750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f877bfff640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 0x7f8784100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f8788c19640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87841029c0 0x7f8784100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 -- 172.21.15.45:0/3213851750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87841029c0 msgr2=0x7f8784100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87841029c0 0x7f8784100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 -- 172.21.15.45:0/3213851750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 msgr2=0x7f8784100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 0x7f8784100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 -- 172.21.15.45:0/3213851750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87841a6520 con 0x7f878410bd00 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f877bfff640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 0x7f8784100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f878941a640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f877c00eaa0 tx=0x7f877c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f8779ffb640 1 -- 172.21.15.45:0/3213851750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00cdb0 con 0x7f878410bd00 2024-04-25T00:27:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.532+0000 7f8779ffb640 1 -- 172.21.15.45:0/3213851750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c005020 con 0x7f878410bd00 2024-04-25T00:27:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.533+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87841a6810 con 0x7f878410bd00 2024-04-25T00:27:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.533+0000 7f8779ffb640 1 -- 172.21.15.45:0/3213851750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0116b0 con 0x7f878410bd00 2024-04-25T00:27:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.533+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841a6db0 con 0x7f878410bd00 2024-04-25T00:27:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.535+0000 7f8779ffb640 1 -- 172.21.15.45:0/3213851750 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f877c005560 con 0x7f878410bd00 2024-04-25T00:27:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.535+0000 7f878db8c740 1 Processor -- start 2024-04-25T00:27:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.535+0000 7f878db8c740 1 -- start start 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184c2010 0x55e3184c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e31841b2c0 con 0x55e3184b7290 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3183c3910 con 0x55e3184c2010 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878db8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e318417280 con 0x55e3184b8980 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8779ffb640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 0x7f876c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51476/0 (socket says 172.21.15.45:51476) 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 -- 172.21.15.45:0/313728789 learned_addr learned my addr 172.21.15.45:0/313728789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878941a640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 0x7f876c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f877bfff640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184c2010 0x55e3184c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8779ffb640 1 -- 172.21.15.45:0/3213851750 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f877c0cf9a0 con 0x7f878410bd00 2024-04-25T00:27:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f877bfff640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 -- 172.21.15.45:0/313728789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 msgr2=0x55e3184b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 -- 172.21.15.45:0/313728789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184c2010 msgr2=0x55e3184c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184c2010 0x55e3184c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 -- 172.21.15.45:0/313728789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3183d4fb0 con 0x55e3184b7290 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f877bfff640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f8788c19640 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8780009c50 tx=0x7f8780010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fd1c4f78a9a13e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.536+0000 7f878941a640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 0x7f876c0a5240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cce40 tx=0x7f877c0666e0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f876affd640 1 -- 172.21.15.45:0/313728789 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780004020 con 0x55e3184b7290 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f876affd640 1 -- 172.21.15.45:0/313728789 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87800041c0 con 0x55e3184b7290 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f876affd640 1 -- 172.21.15.45:0/313728789 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800126b0 con 0x55e3184b7290 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 -- 172.21.15.45:0/313728789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 msgr2=0x55e3184b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8780009c50 tx=0x7f8780010740 comp rx=0 tx=0).stop 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 -- 172.21.15.45:0/313728789 shutdown_connections 2024-04-25T00:27:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184c2010 0x55e3184c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184b7710 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 --2- 172.21.15.45:0/313728789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 -- 172.21.15.45:0/313728789 >> 172.21.15.45:0/313728789 conn(0x55e31847f950 msgr2=0x55e3184b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 -- 172.21.15.45:0/313728789 shutdown_connections 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.537+0000 7f878db8c740 1 -- 172.21.15.45:0/313728789 wait complete. 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 -- start start 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 Processor -- start 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b7290 0x55e3184bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 0x55e3184bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e3184c4ea0 con 0x55e3184c2010 2024-04-25T00:27:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3183d6e90 con 0x55e3184b8980 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e3184c5020 con 0x55e3184b7290 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f878db8c740 10 client.0 _ll_register_callbacks cb 0x55e318412970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51480/0 (socket says 172.21.15.45:51480) 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f877bfff640 1 -- 172.21.15.45:0/2492306527 learned_addr learned my addr 172.21.15.45:0/2492306527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.538+0000 7f8788c19640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 0x55e3184bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b7290 0x55e3184bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 -- 172.21.15.45:0/2492306527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b7290 msgr2=0x55e3184bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b7290 0x55e3184bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 -- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 msgr2=0x55e3184bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 0x55e3184bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 -- 172.21.15.45:0/2492306527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3181f77e0 con 0x55e3184c2010 2024-04-25T00:27:29.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8788c19640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 0x55e3184bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f877bfff640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8774031550 tx=0x7f87740314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20f8c1a8d3366b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8763fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774035040 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8774005040 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87800041c0 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8774051050 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f878db8c740 10 client.17271 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e318257f70 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.539+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87800041c0 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f877403a040 con 0x55e3184c2010 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 10 client.17271 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 20 client.17271 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184d1ff0 0x55e3184d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 4 client.17271 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3183ef5a0 con 0x55e3184d1ff0 2024-04-25T00:27:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.540+0000 7f8788c19640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184d1ff0 0x55e3184d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.541+0000 7f8763fff640 10 client.17271 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.541+0000 7f8788c19640 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184d1ff0 0x55e3184d4400 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb228491621486 server_cookie=47e2e429e258c27a in_seq=0 out_seq=0 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.541+0000 7f8763fff640 1 -- 172.21.15.45:0/2492306527 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3183ef5a0 con 0x55e3184d1ff0 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.541+0000 7f8763fff640 10 client.17271 handle_command_reply: tid=0 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.541+0000 7f878db8c740 1 client.17271 shutdown 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 msgr2=0x55e3184bf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8774031550 tx=0x7f87740314e0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 shutdown_connections 2024-04-25T00:27:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184d1ff0 0x55e3184d4400 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184c2010 0x55e3184bf9d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b8980 0x55e3184bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f8763fff640 0 client.17271 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 --2- 172.21.15.45:0/2492306527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b7290 0x55e3184bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 >> 172.21.15.45:0/2492306527 conn(0x55e31847f950 msgr2=0x55e3184b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 shutdown_connections 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 1 -- 172.21.15.45:0/2492306527 wait complete. 2024-04-25T00:27:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.542+0000 7f878db8c740 20 client.17271 trim_cache size 0 max 16384 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 Processor -- start 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 -- start start 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 0x55e3184be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87841b0df0 con 0x55e31847f950 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87841b2930 con 0x55e3184b7290 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f878db8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87841b14c0 con 0x55e3184b8980 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51482/0 (socket says 172.21.15.45:51482) 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 -- 172.21.15.45:0/3102549689 learned_addr learned my addr 172.21.15.45:0/3102549689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 -- 172.21.15.45:0/3102549689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 msgr2=0x55e3184d9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f8788c19640 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 0x55e3184be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 -- 172.21.15.45:0/3102549689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 msgr2=0x55e3184be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 0x55e3184be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 -- 172.21.15.45:0/3102549689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3180ef170 con 0x55e31847f950 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f8788c19640 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 0x55e3184be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.560+0000 7f877bfff640 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8774006010 tx=0x7f8774031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb316adc4605be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f876a7fc640 1 -- 172.21.15.45:0/3102549689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774035070 con 0x55e31847f950 2024-04-25T00:27:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f876a7fc640 1 -- 172.21.15.45:0/3102549689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774003640 con 0x55e31847f950 2024-04-25T00:27:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f876a7fc640 1 -- 172.21.15.45:0/3102549689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774003aa0 con 0x55e31847f950 2024-04-25T00:27:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- 172.21.15.45:0/3102549689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 msgr2=0x55e3184cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8774006010 tx=0x7f8774031420 comp rx=0 tx=0).stop 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- 172.21.15.45:0/3102549689 shutdown_connections 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e3184d9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3184b7290 0x55e3184be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 --2- 172.21.15.45:0/3102549689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e31847f950 0x55e3184cdda0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- 172.21.15.45:0/3102549689 >> 172.21.15.45:0/3102549689 conn(0x55e3184b2cf0 msgr2=0x55e3184bf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- 172.21.15.45:0/3102549689 shutdown_connections 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- 172.21.15.45:0/3102549689 wait complete. 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.561+0000 7f878db8c740 1 -- start start 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 Processor -- start 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31847f950 0x55e3184c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e318537540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87841b2260 con 0x55e3184b7290 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87841b1b90 con 0x55e31847f950 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87841aedd0 con 0x55e3184b8980 2024-04-25T00:27:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f878db8c740 10 client.0 _ll_register_callbacks cb 0x7f8780005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51496/0 (socket says 172.21.15.45:51496) 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 -- 172.21.15.45:0/2252421090 learned_addr learned my addr 172.21.15.45:0/2252421090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f877bfff640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31847f950 0x55e3184c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f877bfff640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e318537540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 -- 172.21.15.45:0/2252421090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 msgr2=0x55e318537540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e318537540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 -- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31847f950 msgr2=0x55e3184c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31847f950 0x55e3184c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.562+0000 7f8788c19640 1 -- 172.21.15.45:0/2252421090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e318537a80 con 0x55e3184b7290 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8788c19640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8780009c50 tx=0x7f8780009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87933d6908fed1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780009d40 con 0x55e3184b7290 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8774003640 con 0x55e3184b7290 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f8780005040 con 0x55e3184b7290 2024-04-25T00:27:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8780036050 con 0x55e3184b7290 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 10 client.17277 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e318538c00 con 0x55e3184b7290 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774003640 con 0x55e3184b7290 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f878000f040 con 0x55e3184b7290 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 10 client.17277 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 20 client.17277 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184c2010 0x55e31853b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 4 client.17277 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.563+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e318428800 con 0x55e3184c2010 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.564+0000 7f877bfff640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184c2010 0x55e31853b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.564+0000 7f8763fff640 10 client.17277 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.564+0000 7f877bfff640 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184c2010 0x55e31853b170 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f85180e77c92425 server_cookie=b2664bdcea66c911 in_seq=0 out_seq=0 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f8763fff640 1 -- 172.21.15.45:0/2252421090 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e318428800 con 0x55e3184c2010 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f8763fff640 10 client.17277 handle_command_reply: tid=0 2024-04-25T00:27:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 client.17277 shutdown 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 msgr2=0x55e3184c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8780009c50 tx=0x7f8780009670 comp rx=0 tx=0).stop 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 shutdown_connections 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3184b8980 0x55e318537540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e3184c2010 0x55e31853b170 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e3184b7290 0x55e3184c65f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f8763fff640 0 client.17277 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 --2- 172.21.15.45:0/2252421090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31847f950 0x55e3184c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 >> 172.21.15.45:0/2252421090 conn(0x55e3184b2cf0 msgr2=0x55e3184bef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 shutdown_connections 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.565+0000 7f878db8c740 1 -- 172.21.15.45:0/2252421090 wait complete. 2024-04-25T00:27:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878db8c740 20 client.17277 trim_cache size 0 max 16384 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f8a51803-1f1d-4c89-8d73-afcc5e2c2ec8", 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 msgr2=0x7f876c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 0x7f876c0a5240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f877c0cce40 tx=0x7f877c0666e0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 msgr2=0x7f8784101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f877c00eaa0 tx=0x7f877c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 shutdown_connections 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f876c0a2d80 0x7f876c0a5240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f878410bd00 0x7f8784101090 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841032e0 0x7f8784100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 --2- 172.21.15.45:0/3213851750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87841029c0 0x7f8784100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 >> 172.21.15.45:0/3213851750 conn(0x7f8784077740 msgr2=0x7f87840fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 shutdown_connections 2024-04-25T00:27:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.566+0000 7f878ae7d640 1 -- 172.21.15.45:0/3213851750 wait complete. 2024-04-25T00:27:29.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:27:29.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:27:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 -- 172.21.15.45:0/3542971560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301029c0 msgr2=0x7f1e30102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 --2- 172.21.15.45:0/3542971560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301029c0 0x7f1e30102da0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009900 tx=0x7f1e2002a510 comp rx=0 tx=0).stop 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 -- 172.21.15.45:0/3542971560 shutdown_connections 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 --2- 172.21.15.45:0/3542971560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 0x7f1e3010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 --2- 172.21.15.45:0/3542971560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301032e0 0x7f1e3010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 --2- 172.21.15.45:0/3542971560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301029c0 0x7f1e30102da0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 -- 172.21.15.45:0/3542971560 >> 172.21.15.45:0/3542971560 conn(0x7f1e30077740 msgr2=0x7f1e30077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 -- 172.21.15.45:0/3542971560 shutdown_connections 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.681+0000 7f1e35577640 1 -- 172.21.15.45:0/3542971560 wait complete. 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 Processor -- start 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 -- start start 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301029c0 0x7f1e30198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 0x7f1e3019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111b00 con 0x7f1e301032e0 2024-04-25T00:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111980 con 0x7f1e3010bd00 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e35577640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111c80 con 0x7f1e301029c0 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.682+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51510/0 (socket says 172.21.15.45:51510) 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/3313921580 learned_addr learned my addr 172.21.15.45:0/3313921580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2effd640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301029c0 0x7f1e30198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 0x7f1e3019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/3313921580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301029c0 msgr2=0x7f1e30198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301029c0 0x7f1e30198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/3313921580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 msgr2=0x7f1e3019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 0x7f1e3019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/3313921580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3019cff0 con 0x7f1e301032e0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800c890 tx=0x7f1e1800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e27fff640 1 -- 172.21.15.45:0/3313921580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18004220 con 0x7f1e301032e0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3019d2e0 con 0x7f1e301032e0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e27fff640 1 -- 172.21.15.45:0/3313921580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e180043c0 con 0x7f1e301032e0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e27fff640 1 -- 172.21.15.45:0/3313921580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18004a00 con 0x7f1e301032e0 2024-04-25T00:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.683+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e301a4b70 con 0x7f1e301032e0 2024-04-25T00:27:29.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.685+0000 7f1e27fff640 1 -- 172.21.15.45:0/3313921580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1e180079b0 con 0x7f1e301032e0 2024-04-25T00:27:29.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 Processor -- start 2024-04-25T00:27:29.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 -- start start 2024-04-25T00:27:29.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e27fff640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 0x7f1e1c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10630 0x559389a10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5593899756b0 con 0x559389a10f50 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55938991d880 con 0x559389a1b620 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e38364740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559389971670 con 0x559389a10630 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 0x7f1e1c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e27fff640 1 -- 172.21.15.45:0/3313921580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1e180cf510 con 0x7f1e301032e0 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51516/0 (socket says 172.21.15.45:51516) 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.686+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/2563696549 learned_addr learned my addr 172.21.15.45:0/2563696549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2effd640 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10630 0x559389a10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/2563696549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10630 msgr2=0x559389a10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10630 0x559389a10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/2563696549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 msgr2=0x559389a1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/2563696549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593897537d0 con 0x559389a10f50 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 0x7f1e1c0a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e280092f0 tx=0x7f1e28014a70 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a3db0 tx=0x7f1e180054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27126616b58fb161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e257fa640 1 -- 172.21.15.45:0/2563696549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1801d070 con 0x559389a10f50 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e257fa640 1 -- 172.21.15.45:0/2563696549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e1809bd70 con 0x559389a10f50 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.687+0000 7f1e257fa640 1 -- 172.21.15.45:0/2563696549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18099a50 con 0x559389a10f50 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- 172.21.15.45:0/2563696549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 msgr2=0x559389a1b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1e180a3db0 tx=0x7f1e180054a0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- 172.21.15.45:0/2563696549 shutdown_connections 2024-04-25T00:27:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a1b0e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 --2- 172.21.15.45:0/2563696549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10630 0x559389a10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- 172.21.15.45:0/2563696549 >> 172.21.15.45:0/2563696549 conn(0x559389a0c950 msgr2=0x559389a0cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- 172.21.15.45:0/2563696549 shutdown_connections 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- 172.21.15.45:0/2563696549 wait complete. 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 -- start start 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.688+0000 7f1e38364740 1 Processor -- start 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a18540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a18a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559389a1e410 con 0x559389a1b620 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559389930f50 con 0x559389a10630 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559389a1e590 con 0x559389a10f50 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 10 client.0 _ll_register_callbacks cb 0x55938996c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51520/0 (socket says 172.21.15.45:51520) 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 -- 172.21.15.45:0/3448388976 learned_addr learned my addr 172.21.15.45:0/3448388976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2effd640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a18540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a18a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 -- 172.21.15.45:0/3448388976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 msgr2=0x559389a18a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a18a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 -- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 msgr2=0x559389a18540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a18540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 -- 172.21.15.45:0/3448388976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593896893f0 con 0x559389a1b620 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a18a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1e28014230 tx=0x7f1e28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da126cc576fff1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e167fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28026070 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1e28004d20 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.689+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1809bd70 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e28030050 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 10 client.17289 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5593896490f0 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e1809bd70 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e2800f040 con 0x559389a1b620 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 10 client.17289 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 20 client.17289 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a2dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 4 client.17289 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559389982bf0 con 0x559389a2b910 2024-04-25T00:27:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.690+0000 7f1e2effd640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a2dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.691+0000 7f1e2effd640 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a2dd40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991332651258a8c1 server_cookie=214d027d8889963c in_seq=0 out_seq=0 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.691+0000 7f1e167fc640 10 client.17289 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.691+0000 7f1e167fc640 1 -- 172.21.15.45:0/3448388976 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559389982bf0 con 0x559389a2b910 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.691+0000 7f1e167fc640 10 client.17289 handle_command_reply: tid=0 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.691+0000 7f1e38364740 1 client.17289 shutdown 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 msgr2=0x559389a18fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1e28014230 tx=0x7f1e28004ab0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 shutdown_connections 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a2dd40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a1b620 0x559389a18fc0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e167fc640 0 client.17289 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a18a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 --2- 172.21.15.45:0/3448388976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a18540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 >> 172.21.15.45:0/3448388976 conn(0x559389a0c950 msgr2=0x559389a0baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 shutdown_connections 2024-04-25T00:27:29.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 1 -- 172.21.15.45:0/3448388976 wait complete. 2024-04-25T00:27:29.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.692+0000 7f1e38364740 20 client.17289 trim_cache size 0 max 16384 2024-04-25T00:27:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 Processor -- start 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 -- start start 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a27300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a1b620 0x559389a33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301aeb40 con 0x559389a10f50 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301b0680 con 0x559389a10630 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e38364740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301af210 con 0x559389a1b620 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51534/0 (socket says 172.21.15.45:51534) 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/4193224169 learned_addr learned my addr 172.21.15.45:0/4193224169 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2f7fe640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a1b620 0x559389a33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2effd640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a27300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/4193224169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a1b620 msgr2=0x559389a33610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a1b620 0x559389a33610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/4193224169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 msgr2=0x559389a27300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a27300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2e7fc640 1 -- 172.21.15.45:0/4193224169 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559389a11a20 con 0x559389a10f50 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.710+0000 7f1e2effd640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a27300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1e180058c0 tx=0x7f1e180057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06ae351313aea90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e24ff9640 1 -- 172.21.15.45:0/4193224169 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1801d040 con 0x559389a10f50 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e24ff9640 1 -- 172.21.15.45:0/4193224169 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e180a42e0 con 0x559389a10f50 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e24ff9640 1 -- 172.21.15.45:0/4193224169 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180ccb20 con 0x559389a10f50 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 -- 172.21.15.45:0/4193224169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 msgr2=0x559389a12490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1e180058c0 tx=0x7f1e180057a0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 -- 172.21.15.45:0/4193224169 shutdown_connections 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a1b620 0x559389a33610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10f50 0x559389a12490 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 --2- 172.21.15.45:0/4193224169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a10630 0x559389a27300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.711+0000 7f1e38364740 1 -- 172.21.15.45:0/4193224169 >> 172.21.15.45:0/4193224169 conn(0x559389a0c950 msgr2=0x559389a11d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- 172.21.15.45:0/4193224169 shutdown_connections 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- 172.21.15.45:0/4193224169 wait complete. 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- start start 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 Processor -- start 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a1f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301affb0 con 0x559389a10630 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301af8e0 con 0x559389a1b620 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301acb20 con 0x559389a10f50 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e38364740 10 client.0 _ll_register_callbacks cb 0x7f1e280140f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51548/0 (socket says 172.21.15.45:51548) 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e2effd640 1 -- 172.21.15.45:0/808657374 learned_addr learned my addr 172.21.15.45:0/808657374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.712+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a1f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 -- 172.21.15.45:0/808657374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 msgr2=0x559389a1f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a1f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 -- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 msgr2=0x559389a1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 -- 172.21.15.45:0/808657374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559389a90e80 con 0x559389a10630 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e2effd640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1e2002aa30 tx=0x7f1e20031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2836fe58440d39fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2004c070 con 0x559389a10630 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f1e200483b0 con 0x559389a10630 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e180a42e0 con 0x559389a10630 2024-04-25T00:27:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e20057050 con 0x559389a10630 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 10 client.17295 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559389a920a0 con 0x559389a10630 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e180a42e0 con 0x559389a10630 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e200047a0 con 0x559389a10630 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 10 client.17295 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 20 client.17295 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 4 client.17295 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.713+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55938997cf40 con 0x559389a2b910 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e167fc640 10 client.17295 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e2e7fc640 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a94810 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd8090f031809d2 server_cookie=498b262876c1133e in_seq=0 out_seq=0 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e167fc640 1 -- 172.21.15.45:0/808657374 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55938997cf40 con 0x559389a2b910 2024-04-25T00:27:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e167fc640 10 client.17295 handle_command_reply: tid=0 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 client.17295 shutdown 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 msgr2=0x559389a1f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1e2002aa30 tx=0x7f1e20031a00 comp rx=0 tx=0).stop 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 shutdown_connections 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559389a1b620 0x559389a1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559389a10f50 0x559389a1f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559389a2b910 0x559389a94810 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 --2- 172.21.15.45:0/808657374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559389a10630 0x559389a1f170 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e167fc640 0 client.17295 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 >> 172.21.15.45:0/808657374 conn(0x559389a0c950 msgr2=0x559389a0baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.714+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 shutdown_connections 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e38364740 1 -- 172.21.15.45:0/808657374 wait complete. 2024-04-25T00:27:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e38364740 20 client.17295 trim_cache size 0 max 16384 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "f8a51803-1f1d-4c89-8d73-afcc5e2c2ec8": { 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "f8a51803-1f1d-4c89-8d73-afcc5e2c2ec8", 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:27:29.722 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:27:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 msgr2=0x7f1e1c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 0x7f1e1c0a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e280092f0 tx=0x7f1e28014a70 comp rx=0 tx=0).stop 2024-04-25T00:27:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 msgr2=0x7f1e30198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800c890 tx=0x7f1e1800cd50 comp rx=0 tx=0).stop 2024-04-25T00:27:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 shutdown_connections 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e1c0a2d00 0x7f1e1c0a51c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e3010bd00 0x7f1e3019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e301032e0 0x7f1e30198540 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 --2- 172.21.15.45:0/3313921580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e301029c0 0x7f1e30198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 >> 172.21.15.45:0/3313921580 conn(0x7f1e30077740 msgr2=0x7f1e300fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 shutdown_connections 2024-04-25T00:27:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:29.715+0000 7f1e35577640 1 -- 172.21.15.45:0/3313921580 wait complete. 2024-04-25T00:27:29.727 INFO:tasks.cephfs.filesystem:scrub status for tag:f8a51803-1f1d-4c89-8d73-afcc5e2c2ec8 - {'path': '/', 'tag': 'f8a51803-1f1d-4c89-8d73-afcc5e2c2ec8', 'options': 'recursive,force'} 2024-04-25T00:27:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:29 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:27:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:29 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:29 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:27:30.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:29 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:30.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:29 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:27:30.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:29 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:30.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:29 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:27:30.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:29 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:29 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:27:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:29 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:29 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:27:30.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:29 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:27:31.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:30 smithi122 ceph-mon[32959]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:31.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:30 smithi071 ceph-mon[32667]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:31.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:30 smithi045 ceph-mon[29194]: pgmap v169: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:32 smithi122 ceph-mon[32959]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:32 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:27:33.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:32 smithi071 ceph-mon[32667]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:33.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:32 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:27:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:32 smithi045 ceph-mon[29194]: pgmap v170: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:33.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:32 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:27:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:34 smithi122 ceph-mon[32959]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:35.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:34 smithi071 ceph-mon[32667]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:34 smithi045 ceph-mon[29194]: pgmap v171: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:27:37.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:36 smithi071 ceph-mon[32667]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:37.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:36 smithi045 ceph-mon[29194]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:36 smithi122 ceph-mon[32959]: pgmap v172: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:38 smithi122 ceph-mon[32959]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:38 smithi071 ceph-mon[32667]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:38 smithi045 ceph-mon[29194]: pgmap v173: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:40 smithi122 ceph-mon[32959]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:40 smithi071 ceph-mon[32667]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:40 smithi045 ceph-mon[29194]: pgmap v174: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:27:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:42 smithi122 ceph-mon[32959]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:42 smithi071 ceph-mon[32667]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:42 smithi045 ceph-mon[29194]: pgmap v175: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:27:45.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:44 smithi122 ceph-mon[32959]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:27:45.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:44 smithi071 ceph-mon[32667]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:27:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:44 smithi045 ceph-mon[29194]: pgmap v176: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:27:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:46 smithi122 ceph-mon[32959]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:46 smithi071 ceph-mon[32667]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:46 smithi045 ceph-mon[29194]: pgmap v177: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:27:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:48 smithi122 ceph-mon[32959]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:48 smithi071 ceph-mon[32667]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:48 smithi045 ceph-mon[29194]: pgmap v178: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:51.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:50 smithi122 ceph-mon[32959]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:50 smithi071 ceph-mon[32667]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:50 smithi045 ceph-mon[29194]: pgmap v179: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:27:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:52 smithi122 ceph-mon[32959]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:53.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:52 smithi071 ceph-mon[32667]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:52 smithi045 ceph-mon[29194]: pgmap v180: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:55.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:54 smithi122 ceph-mon[32959]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:27:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:54 smithi071 ceph-mon[32667]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:27:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:54 smithi045 ceph-mon[29194]: pgmap v181: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:27:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:56 smithi122 ceph-mon[32959]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:57.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:56 smithi071 ceph-mon[32667]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:56 smithi045 ceph-mon[29194]: pgmap v182: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:27:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:27:58 smithi122 ceph-mon[32959]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:27:58 smithi071 ceph-mon[32667]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:27:58 smithi045 ceph-mon[29194]: pgmap v183: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:27:59.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:27:59.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 -- 172.21.15.45:0/333901635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188069dc0 msgr2=0x7f118806a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 --2- 172.21.15.45:0/333901635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188069dc0 0x7f118806a240 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1170009930 tx=0x7f117002a420 comp rx=0 tx=0).stop 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 -- 172.21.15.45:0/333901635 shutdown_connections 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 --2- 172.21.15.45:0/333901635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f118806a780 0x7f118810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 --2- 172.21.15.45:0/333901635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188069dc0 0x7f118806a240 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 --2- 172.21.15.45:0/333901635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1188107730 0x7f1188107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 -- 172.21.15.45:0/333901635 >> 172.21.15.45:0/333901635 conn(0x7f11880776a0 msgr2=0x7f1188077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 -- 172.21.15.45:0/333901635 shutdown_connections 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.845+0000 7f118cf22640 1 -- 172.21.15.45:0/333901635 wait complete. 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 Processor -- start 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 -- start start 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1188069dc0 0x7f1188198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118806a780 0x7f1188198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1188111b90 con 0x7f1188107730 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1188111a10 con 0x7f118806a780 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.846+0000 7f118cf22640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1188111d10 con 0x7f1188069dc0 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57876/0 (socket says 172.21.15.45:57876) 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 -- 172.21.15.45:0/346912142 learned_addr learned my addr 172.21.15.45:0/346912142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f117ffff640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118806a780 0x7f1188198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f118659c640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1188069dc0 0x7f1188198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 -- 172.21.15.45:0/346912142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1188069dc0 msgr2=0x7f1188198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1188069dc0 0x7f1188198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 -- 172.21.15.45:0/346912142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118806a780 msgr2=0x7f1188198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118806a780 0x7f1188198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 -- 172.21.15.45:0/346912142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118819d100 con 0x7f1188107730 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f1186d9d640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f118000eaa0 tx=0x7f118000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f117f7fe640 1 -- 172.21.15.45:0/346912142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118000cdb0 con 0x7f1188107730 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118819d3f0 con 0x7f1188107730 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f117f7fe640 1 -- 172.21.15.45:0/346912142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1180005020 con 0x7f1188107730 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f117f7fe640 1 -- 172.21.15.45:0/346912142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800116d0 con 0x7f1188107730 2024-04-25T00:27:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.847+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11881a4c80 con 0x7f1188107730 2024-04-25T00:27:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.850+0000 7f117f7fe640 1 -- 172.21.15.45:0/346912142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f11800054d0 con 0x7f1188107730 2024-04-25T00:27:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.850+0000 7f118fc35740 1 Processor -- start 2024-04-25T00:27:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.850+0000 7f118fc35740 1 -- start start 2024-04-25T00:27:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094089910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f117f7fe640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 0x7f11740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d093fee6b0 con 0x55d094098a30 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d093f96880 con 0x55d094089e50 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118fc35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d093fea670 con 0x55d094089530 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118659c640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 0x7f11740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57882/0 (socket says 172.21.15.45:57882) 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 -- 172.21.15.45:0/1700898810 learned_addr learned my addr 172.21.15.45:0/1700898810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f117f7fe640 1 -- 172.21.15.45:0/346912142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f11800cf920 con 0x7f1188107730 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f117ffff640 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094089910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f117ffff640 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 -- 172.21.15.45:0/1700898810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 msgr2=0x55d094089910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094089910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 -- 172.21.15.45:0/1700898810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 msgr2=0x55d0940984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 -- 172.21.15.45:0/1700898810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d093dcc7d0 con 0x55d094098a30 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1180098560 tx=0x7f1180000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a61b0f47dcc9daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.851+0000 7f118659c640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 0x7f11740a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f117800bb60 tx=0x7f117800b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f117cff9640 1 -- 172.21.15.45:0/1700898810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118001f070 con 0x55d094098a30 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f117cff9640 1 -- 172.21.15.45:0/1700898810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f118009bb90 con 0x55d094098a30 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f117cff9640 1 -- 172.21.15.45:0/1700898810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800cd0f0 con 0x55d094098a30 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 -- 172.21.15.45:0/1700898810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 msgr2=0x55d09409ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1180098560 tx=0x7f1180000f40 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 -- 172.21.15.45:0/1700898810 shutdown_connections 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d09409ae20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 --2- 172.21.15.45:0/1700898810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094089910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 -- 172.21.15.45:0/1700898810 >> 172.21.15.45:0/1700898810 conn(0x55d094085950 msgr2=0x55d094085d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 -- 172.21.15.45:0/1700898810 shutdown_connections 2024-04-25T00:27:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.852+0000 7f118fc35740 1 -- 172.21.15.45:0/1700898810 wait complete. 2024-04-25T00:27:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 -- start start 2024-04-25T00:27:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 Processor -- start 2024-04-25T00:27:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 0x55d0940959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089e50 0x55d094095f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d09409b7f0 con 0x55d094098a30 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d093fa9f50 con 0x55d094089530 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d09409b970 con 0x55d094089e50 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f118fc35740 10 client.0 _ll_register_callbacks cb 0x55d093fe5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57886/0 (socket says 172.21.15.45:57886) 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f1186d9d640 1 -- 172.21.15.45:0/1707561707 learned_addr learned my addr 172.21.15.45:0/1707561707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.853+0000 7f117ffff640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089e50 0x55d094095f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118659c640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 0x55d0940959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 -- 172.21.15.45:0/1707561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089e50 msgr2=0x55d094095f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089e50 0x55d094095f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 -- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 msgr2=0x55d0940959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 0x55d0940959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 -- 172.21.15.45:0/1707561707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d093d023f0 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118659c640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 0x55d0940959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f1186d9d640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f11800a4220 tx=0x7f11800a0010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b122398c375a095a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118001f040 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f11800a42c0 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f118009bb90 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11800a44e0 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118fc35740 10 client.17307 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d093cc20f0 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f118009bb90 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f118000f040 con 0x55d094098a30 2024-04-25T00:27:59.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118fc35740 10 client.17307 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.854+0000 7f118fc35740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118fc35740 20 client.17307 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d0940a8bf0 0x55d0940aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118fc35740 4 client.17307 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d093ffbbf0 con 0x55d0940a8bf0 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118659c640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d0940a8bf0 0x55d0940aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f116dffb640 10 client.17307 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.855+0000 7f118659c640 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d0940a8bf0 0x55d0940aafe0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bb5d8bb50a21f6 server_cookie=848b4ce2989e7479 in_seq=0 out_seq=0 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f116dffb640 1 -- 172.21.15.45:0/1707561707 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d093ffbbf0 con 0x55d0940a8bf0 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f116dffb640 10 client.17307 handle_command_reply: tid=0 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 client.17307 shutdown 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 msgr2=0x55d094096470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f11800a4220 tx=0x7f11800a0010 comp rx=0 tx=0).stop 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 shutdown_connections 2024-04-25T00:27:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d0940a8bf0 0x55d0940aafe0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d094096470 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f116dffb640 0 client.17307 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089e50 0x55d094095f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 --2- 172.21.15.45:0/1707561707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089530 0x55d0940959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 >> 172.21.15.45:0/1707561707 conn(0x55d094085950 msgr2=0x55d09408b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 shutdown_connections 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 1 -- 172.21.15.45:0/1707561707 wait complete. 2024-04-25T00:27:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.856+0000 7f118fc35740 20 client.17307 trim_cache size 0 max 16384 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 Processor -- start 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 -- start start 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094098120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094098a30 0x55d094093190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11881aca00 con 0x55d094089e50 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.874+0000 7f118fc35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11881ae540 con 0x55d094098a30 2024-04-25T00:27:59.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f118fc35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11881ad0d0 con 0x55d094089530 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57890/0 (socket says 172.21.15.45:57890) 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 -- 172.21.15.45:0/2736807462 learned_addr learned my addr 172.21.15.45:0/2736807462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f118659c640 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094098120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f1186d9d640 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094098a30 0x55d094093190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 -- 172.21.15.45:0/2736807462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 msgr2=0x55d094098120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094098120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 -- 172.21.15.45:0/2736807462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094098a30 msgr2=0x55d094093190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094098a30 0x55d094093190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 -- 172.21.15.45:0/2736807462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d094089c80 con 0x55d094089e50 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f117ffff640 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f117002aa10 tx=0x7f1170004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4165023c9758e4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f116ffff640 1 -- 172.21.15.45:0/2736807462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170035040 con 0x55d094089e50 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f116ffff640 1 -- 172.21.15.45:0/2736807462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170005510 con 0x55d094089e50 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.875+0000 7f116ffff640 1 -- 172.21.15.45:0/2736807462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170003850 con 0x55d094089e50 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- 172.21.15.45:0/2736807462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 msgr2=0x55d094092c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f117002aa10 tx=0x7f1170004310 comp rx=0 tx=0).stop 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- 172.21.15.45:0/2736807462 shutdown_connections 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094098a30 0x55d094093190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094089e50 0x55d094092c50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- 172.21.15.45:0/2736807462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d094098120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- 172.21.15.45:0/2736807462 >> 172.21.15.45:0/2736807462 conn(0x55d094085950 msgr2=0x55d094094a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- 172.21.15.45:0/2736807462 shutdown_connections 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- 172.21.15.45:0/2736807462 wait complete. 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 -- start start 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 Processor -- start 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d0940ae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.876+0000 7f118fc35740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11881ade70 con 0x55d094098a30 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11881ad7a0 con 0x55d094089e50 2024-04-25T00:27:59.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11881aa9e0 con 0x55d094089530 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 10 client.0 _ll_register_callbacks cb 0x7f118000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57896/0 (socket says 172.21.15.45:57896) 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 -- 172.21.15.45:0/859012730 learned_addr learned my addr 172.21.15.45:0/859012730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118659c640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d0940ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f117ffff640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 -- 172.21.15.45:0/859012730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 msgr2=0x55d0940ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d0940ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 -- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 msgr2=0x55d0940ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 -- 172.21.15.45:0/859012730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0940ac520 con 0x55d094098a30 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f1186d9d640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f118009b2b0 tx=0x7f11800b4e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86025b9df192aaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f116dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118001f070 con 0x55d094098a30 2024-04-25T00:27:59.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f118009ad50 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1170005510 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f118009bb90 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 10 client.17313 fetch_fsmap learned FSMap version 28 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.877+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d09410ee40 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170005510 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f11800ba120 con 0x55d094098a30 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 10 client.17313 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 20 client.17313 populate_metadata read hostname 'smithi045' 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d09410f370 0x55d094111760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 4 client.17313 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d093ff5f40 con 0x55d09410f370 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118659c640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d09410f370 0x55d094111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f116dffb640 10 client.17313 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118659c640 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d09410f370 0x55d094111760 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99fab1c44b7159ca server_cookie=4e4f62c4c27e7139 in_seq=0 out_seq=0 2024-04-25T00:27:59.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f116dffb640 1 -- 172.21.15.45:0/859012730 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d093ff5f40 con 0x55d09410f370 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f116dffb640 10 client.17313 handle_command_reply: tid=0 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.878+0000 7f118fc35740 1 client.17313 shutdown 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 msgr2=0x55d0940abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f118009b2b0 tx=0x7f11800b4e80 comp rx=0 tx=0).stop 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 shutdown_connections 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d09410f370 0x55d094111760 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d094098a30 0x55d0940abcf0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f116dffb640 0 client.17313 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d094089e50 0x55d0940ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 --2- 172.21.15.45:0/859012730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d094089530 0x55d0940ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 >> 172.21.15.45:0/859012730 conn(0x55d094085950 msgr2=0x55d094095990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 shutdown_connections 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 1 -- 172.21.15.45:0/859012730 wait complete. 2024-04-25T00:27:59.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118fc35740 20 client.17313 trim_cache size 0 max 16384 2024-04-25T00:27:59.886 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:27:59.886 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:27:59.886 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:27:59.886 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 msgr2=0x7f11740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 0x7f11740a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f117800bb60 tx=0x7f117800b390 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 msgr2=0x7f118819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f118000eaa0 tx=0x7f118000ef60 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 shutdown_connections 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f11740a2d00 0x7f11740a51c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1188107730 0x7f118819c9d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118806a780 0x7f1188198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 --2- 172.21.15.45:0/346912142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1188069dc0 0x7f1188198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 >> 172.21.15.45:0/346912142 conn(0x7f11880776a0 msgr2=0x7f11880fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 shutdown_connections 2024-04-25T00:27:59.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.879+0000 7f118cf22640 1 -- 172.21.15.45:0/346912142 wait complete. 2024-04-25T00:27:59.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:27:59.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 -- 172.21.15.45:0/2050421347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc105680 msgr2=0x7fd9cc105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/2050421347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc105680 0x7fd9cc105a60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009930 tx=0x7fd9bc02a4d0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 -- 172.21.15.45:0/2050421347 shutdown_connections 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/2050421347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc101b60 0x7fd9cc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/2050421347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc1011a0 0x7fd9cc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/2050421347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc105680 0x7fd9cc105a60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 -- 172.21.15.45:0/2050421347 >> 172.21.15.45:0/2050421347 conn(0x7fd9cc075920 msgr2=0x7fd9cc075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:59.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 -- 172.21.15.45:0/2050421347 shutdown_connections 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.993+0000 7fd9d1b09640 1 -- 172.21.15.45:0/2050421347 wait complete. 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 Processor -- start 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 -- start start 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc1011a0 0x7fd9cc196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 0x7fd9cc19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc110a00 con 0x7fd9cc101b60 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc110880 con 0x7fd9cc105680 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9d1b09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc110b80 con 0x7fd9cc1011a0 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57924/0 (socket says 172.21.15.45:57924) 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 -- 172.21.15.45:0/1246872486 learned_addr learned my addr 172.21.15.45:0/1246872486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9cb7fe640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc1011a0 0x7fd9cc196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9cbfff640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 0x7fd9cc19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 -- 172.21.15.45:0/1246872486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc1011a0 msgr2=0x7fd9cc196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc1011a0 0x7fd9cc196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 -- 172.21.15.45:0/1246872486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 msgr2=0x7fd9cc19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.994+0000 7fd9caffd640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 0x7fd9cc19b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9caffd640 1 -- 172.21.15.45:0/1246872486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc19be50 con 0x7fd9cc101b60 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9cbfff640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 0x7fd9cc19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9caffd640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c920 tx=0x7fd9c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0007c70 con 0x7fd9cc101b60 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c0007e10 con 0x7fd9cc101b60 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc19c140 con 0x7fd9cc101b60 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0018890 con 0x7fd9cc101b60 2024-04-25T00:27:59.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.995+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1a39d0 con 0x7fd9cc101b60 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.996+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc0689b0 con 0x7fd9cc101b60 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.997+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9c0018a70 con 0x7fd9cc101b60 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.998+0000 7fd9c8ff9640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 0x7fd9b40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.998+0000 7fd9cb7fe640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 0x7fd9b40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.998+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd9c00cf900 con 0x7fd9cc101b60 2024-04-25T00:27:59.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:27:59.998+0000 7fd9cb7fe640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 0x7fd9b40a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc031550 tx=0x7fd9bc0058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.003+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9c0014030 con 0x7fd9cc101b60 2024-04-25T00:28:00.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.226+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9cc02d170 con 0x7fd9cc101b60 2024-04-25T00:28:00.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.227+0000 7fd9c8ff9640 1 -- 172.21.15.45:0/1246872486 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fd9c009c1c0 con 0x7fd9cc101b60 2024-04-25T00:28:00.227 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:28:00.228 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:28:00.228 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 msgr2=0x7fd9b40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 0x7fd9b40a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc031550 tx=0x7fd9bc0058d0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 msgr2=0x7fd9cc1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c920 tx=0x7fd9c000cde0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 shutdown_connections 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd9b40a2d00 0x7fd9b40a51c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9cc105680 0x7fd9cc19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9cc101b60 0x7fd9cc1973a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 --2- 172.21.15.45:0/1246872486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9cc1011a0 0x7fd9cc196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 >> 172.21.15.45:0/1246872486 conn(0x7fd9cc075920 msgr2=0x7fd9cc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 shutdown_connections 2024-04-25T00:28:00.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.229+0000 7fd9d1b09640 1 -- 172.21.15.45:0/1246872486 wait complete. 2024-04-25T00:28:00.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 -- 172.21.15.45:0/1741995407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480107720 msgr2=0x7f2480107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 --2- 172.21.15.45:0/1741995407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480107720 0x7f2480107b00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2470009930 tx=0x7f247002a420 comp rx=0 tx=0).stop 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 -- 172.21.15.45:0/1741995407 shutdown_connections 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 --2- 172.21.15.45:0/1741995407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24801020d0 0x7f248010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 --2- 172.21.15.45:0/1741995407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480101710 0x7f2480101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 --2- 172.21.15.45:0/1741995407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2480107720 0x7f2480107b00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 -- 172.21.15.45:0/1741995407 >> 172.21.15.45:0/1741995407 conn(0x7f24800fd180 msgr2=0x7f24800ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 -- 172.21.15.45:0/1741995407 shutdown_connections 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.352+0000 7f24856d4640 1 -- 172.21.15.45:0/1741995407 wait complete. 2024-04-25T00:28:00.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 Processor -- start 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 -- start start 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2480101710 0x7f2480198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480107720 0x7f248019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2480111af0 con 0x7f24801020d0 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2480111970 con 0x7f2480107720 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24856d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2480111c70 con 0x7f2480101710 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f247effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2480101710 0x7f2480198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57930/0 (socket says 172.21.15.45:57930) 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 -- 172.21.15.45:0/2700014865 learned_addr learned my addr 172.21.15.45:0/2700014865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f247f7fe640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480107720 0x7f248019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 -- 172.21.15.45:0/2700014865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2480101710 msgr2=0x7f2480198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2480101710 0x7f2480198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 -- 172.21.15.45:0/2700014865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480107720 msgr2=0x7f248019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480107720 0x7f248019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.353+0000 7f24767fc640 1 -- 172.21.15.45:0/2700014865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248019d010 con 0x7f24801020d0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f24767fc640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f246800e9f0 tx=0x7f246800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f247cff9640 1 -- 172.21.15.45:0/2700014865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800cd80 con 0x7f24801020d0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f247cff9640 1 -- 172.21.15.45:0/2700014865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2468010910 con 0x7f24801020d0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248019d300 con 0x7f24801020d0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f247cff9640 1 -- 172.21.15.45:0/2700014865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468018990 con 0x7f24801020d0 2024-04-25T00:28:00.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.354+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801a4b90 con 0x7f24801020d0 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.356+0000 7f247cff9640 1 -- 172.21.15.45:0/2700014865 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2468010ac0 con 0x7f24801020d0 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.356+0000 7f24883e9740 1 Processor -- start 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.356+0000 7f24883e9740 1 -- start start 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x561061890960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5610617f5670 con 0x56106189b570 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f247cff9640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 0x7f246c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56106179d970 con 0x561061890580 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24883e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5610617f1630 con 0x561061890ea0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f247f7fe640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 0x7f246c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57944/0 (socket says 172.21.15.45:57944) 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 -- 172.21.15.45:0/2682963411 learned_addr learned my addr 172.21.15.45:0/2682963411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f247cff9640 1 -- 172.21.15.45:0/2700014865 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f24680cfb50 con 0x7f24801020d0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f247f7fe640 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x561061890960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f247effd640 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 -- 172.21.15.45:0/2682963411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 msgr2=0x56106189b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 -- 172.21.15.45:0/2682963411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 msgr2=0x561061890960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x561061890960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.357+0000 7f24767fc640 1 -- 172.21.15.45:0/2682963411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610617c1400 con 0x56106189b570 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24767fc640 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f246809bca0 tx=0x7f246806dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ad5b2241c23e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f247f7fe640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 0x7f246c0a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2478005c40 tx=0x7f24780094f0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24757fa640 1 -- 172.21.15.45:0/2682963411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468026040 con 0x56106189b570 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24757fa640 1 -- 172.21.15.45:0/2682963411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24680a00c0 con 0x56106189b570 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24757fa640 1 -- 172.21.15.45:0/2682963411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680bb0d0 con 0x56106189b570 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24883e9740 1 -- 172.21.15.45:0/2682963411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 msgr2=0x56106189d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24883e9740 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f246809bca0 tx=0x7f246806dd90 comp rx=0 tx=0).stop 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24883e9740 1 -- 172.21.15.45:0/2682963411 shutdown_connections 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.358+0000 7f24883e9740 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56106189b570 0x56106189d960 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 --2- 172.21.15.45:0/2682963411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x561061890960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- 172.21.15.45:0/2682963411 >> 172.21.15.45:0/2682963411 conn(0x56106188c8a0 msgr2=0x56106188cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- 172.21.15.45:0/2682963411 shutdown_connections 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- 172.21.15.45:0/2682963411 wait complete. 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- start start 2024-04-25T00:28:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 Processor -- start 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106189af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061898840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56106189e2a0 con 0x561061890ea0 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5610617b1050 con 0x561061890580 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56106189e420 con 0x56106189b570 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24883e9740 10 client.0 _ll_register_callbacks cb 0x561061508be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57946/0 (socket says 172.21.15.45:57946) 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f24767fc640 1 -- 172.21.15.45:0/680544233 learned_addr learned my addr 172.21.15.45:0/680544233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.359+0000 7f247effd640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106189af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f247f7fe640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061898840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 -- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 msgr2=0x561061898840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061898840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 -- 172.21.15.45:0/680544233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 msgr2=0x56106189af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106189af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 -- 172.21.15.45:0/680544233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561061782dc0 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f247f7fe640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061898840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24767fc640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f246800c6a0 tx=0x7f2468098620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c922dba6faa0325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680bb030 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f24680bb1d0 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24680a00c0 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2468018b30 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 10 client.17331 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610615093f0 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24680a00c0 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2468097a60 con 0x561061890ea0 2024-04-25T00:28:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 10 client.17331 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 10 client.17331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 20 client.17331 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.360+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5610618ab3f0 0x5610618ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.361+0000 7f24883e9740 4 client.17331 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.361+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561061802bb0 con 0x5610618ab3f0 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.361+0000 7f247effd640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5610618ab3f0 0x5610618ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.361+0000 7f24667fc640 10 client.17331 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.361+0000 7f247effd640 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5610618ab3f0 0x5610618ad7e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77cc9f6d56bce148 server_cookie=4f1fa7cd1205c004 in_seq=0 out_seq=0 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24667fc640 1 -- 172.21.15.45:0/680544233 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561061802bb0 con 0x5610618ab3f0 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24667fc640 10 client.17331 handle_command_reply: tid=0 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 client.17331 shutdown 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 msgr2=0x561061898230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f246800c6a0 tx=0x7f2468098620 comp rx=0 tx=0).stop 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 shutdown_connections 2024-04-25T00:28:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5610618ab3f0 0x5610618ad7e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061898840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24667fc640 0 client.17331 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x561061898230 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 --2- 172.21.15.45:0/680544233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106189af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 >> 172.21.15.45:0/680544233 conn(0x56106188c8a0 msgr2=0x5610618926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 shutdown_connections 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 1 -- 172.21.15.45:0/680544233 wait complete. 2024-04-25T00:28:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.362+0000 7f24883e9740 20 client.17331 trim_cache size 0 max 16384 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.375+0000 7f24883e9740 1 Processor -- start 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.375+0000 7f24883e9740 1 -- start start 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56106189b570 0x5610618b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24801aebb0 con 0x561061890580 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24801b06f0 con 0x56106189b570 2024-04-25T00:28:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f24883e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24801af280 con 0x561061890ea0 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57950/0 (socket says 172.21.15.45:57950) 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 -- 172.21.15.45:0/1906715401 learned_addr learned my addr 172.21.15.45:0/1906715401 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247f7fe640 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56106189b570 0x5610618b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 -- 172.21.15.45:0/1906715401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 msgr2=0x56106189aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 -- 172.21.15.45:0/1906715401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56106189b570 msgr2=0x5610618b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56106189b570 0x5610618b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 -- 172.21.15.45:0/1906715401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610615d37d0 con 0x561061890580 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f247effd640 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2470009900 tx=0x7f24700058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f24f9ac6c74d98e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f2474ff9640 1 -- 172.21.15.45:0/1906715401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470049070 con 0x561061890580 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.376+0000 7f2474ff9640 1 -- 172.21.15.45:0/1906715401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470045bc0 con 0x561061890580 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f2474ff9640 1 -- 172.21.15.45:0/1906715401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470035040 con 0x561061890580 2024-04-25T00:28:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- 172.21.15.45:0/1906715401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 msgr2=0x5610618a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2470009900 tx=0x7f24700058d0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- 172.21.15.45:0/1906715401 shutdown_connections 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56106189b570 0x5610618b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561061890ea0 0x56106189aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 --2- 172.21.15.45:0/1906715401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890580 0x5610618a7160 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- 172.21.15.45:0/1906715401 >> 172.21.15.45:0/1906715401 conn(0x56106188c8a0 msgr2=0x56106188bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- 172.21.15.45:0/1906715401 shutdown_connections 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- 172.21.15.45:0/1906715401 wait complete. 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.377+0000 7f24883e9740 1 -- start start 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 Processor -- start 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106190e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061908c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24801b0020 con 0x561061890ea0 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24801af950 con 0x561061890580 2024-04-25T00:28:00.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24801acb90 con 0x56106189b570 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24883e9740 10 client.0 _ll_register_callbacks cb 0x7f24780125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57958/0 (socket says 172.21.15.45:57958) 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 -- 172.21.15.45:0/705153046 learned_addr learned my addr 172.21.15.45:0/705153046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f247f7fe640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061908c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f247effd640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106190e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 -- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 msgr2=0x561061908c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061908c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 -- 172.21.15.45:0/705153046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 msgr2=0x56106190e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106190e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 -- 172.21.15.45:0/705153046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561061909330 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f247effd640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106190e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24767fc640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f24680986e0 tx=0x7f2468009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8663098c0685bbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.378+0000 7f24667fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246809ae30 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f246809b360 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2470045bc0 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f246809b850 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 10 client.17337 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561061916030 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470045bc0 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f24680bafe0 con 0x561061890ea0 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 10 client.17337 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 10 client.17337 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 20 client.17337 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561061916590 0x5610619189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 4 client.17337 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610617fcf00 con 0x561061916590 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.379+0000 7f247effd640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561061916590 0x5610619189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f247effd640 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561061916590 0x5610619189b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1baa58062a75c24 server_cookie=e7569a66300e92ea in_seq=0 out_seq=0 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24667fc640 10 client.17337 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24667fc640 1 -- 172.21.15.45:0/705153046 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610617fcf00 con 0x561061916590 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24667fc640 10 client.17337 handle_command_reply: tid=0 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 client.17337 shutdown 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 msgr2=0x56106190ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f24680986e0 tx=0x7f2468009700 comp rx=0 tx=0).stop 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 shutdown_connections 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x561061916590 0x5610619189b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56106189b570 0x561061908c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561061890ea0 0x56106190ed70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 --2- 172.21.15.45:0/705153046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561061890580 0x56106190e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24667fc640 0 client.17337 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 >> 172.21.15.45:0/705153046 conn(0x56106188c8a0 msgr2=0x5610618a6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 shutdown_connections 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 1 -- 172.21.15.45:0/705153046 wait complete. 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.380+0000 7f24883e9740 20 client.17337 trim_cache size 0 max 16384 2024-04-25T00:28:00.388 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 msgr2=0x7f246c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 0x7f246c0a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2478005c40 tx=0x7f24780094f0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 msgr2=0x7f24801985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f246800e9f0 tx=0x7f246800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 shutdown_connections 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f246c0a2d00 0x7f246c0a51c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2480107720 0x7f248019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24801020d0 0x7f24801985b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 --2- 172.21.15.45:0/2700014865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2480101710 0x7f2480198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 >> 172.21.15.45:0/2700014865 conn(0x7f24800fd180 msgr2=0x7f24800fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 shutdown_connections 2024-04-25T00:28:00.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.381+0000 7f24856d4640 1 -- 172.21.15.45:0/2700014865 wait complete. 2024-04-25T00:28:00.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:28:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.495+0000 7feacd212640 1 -- 172.21.15.45:0/107578195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 msgr2=0x7feac810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.495+0000 7feacd212640 1 --2- 172.21.15.45:0/107578195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac810e1a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7feac0009ca0 tx=0x7feac002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 -- 172.21.15.45:0/107578195 shutdown_connections 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 --2- 172.21.15.45:0/107578195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac810e1a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 --2- 172.21.15.45:0/107578195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 0x7feac810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 --2- 172.21.15.45:0/107578195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 0x7feac8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 -- 172.21.15.45:0/107578195 >> 172.21.15.45:0/107578195 conn(0x7feac8077740 msgr2=0x7feac8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 -- 172.21.15.45:0/107578195 shutdown_connections 2024-04-25T00:28:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 -- 172.21.15.45:0/107578195 wait complete. 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.496+0000 7feacd212640 1 Processor -- start 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 -- start start 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 0x7feac8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 0x7feac8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111b00 con 0x7feac810bd00 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111980 con 0x7feac81032e0 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feacd212640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111c80 con 0x7feac81029c0 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57972/0 (socket says 172.21.15.45:57972) 2024-04-25T00:28:00.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 -- 172.21.15.45:0/2444012259 learned_addr learned my addr 172.21.15.45:0/2444012259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac659c640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 0x7feac8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac6d9d640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 0x7feac8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 -- 172.21.15.45:0/2444012259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 msgr2=0x7feac8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 0x7feac8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 -- 172.21.15.45:0/2444012259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 msgr2=0x7feac8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 0x7feac8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 -- 172.21.15.45:0/2444012259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac819cfc0 con 0x7feac810bd00 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feac759e640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7feac0005cc0 tx=0x7feac0003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.497+0000 7feaaffff640 1 -- 172.21.15.45:0/2444012259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0046070 con 0x7feac810bd00 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.498+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac819d250 con 0x7feac810bd00 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.498+0000 7feaaffff640 1 -- 172.21.15.45:0/2444012259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac002fb20 con 0x7feac810bd00 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.498+0000 7feaaffff640 1 -- 172.21.15.45:0/2444012259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0033070 con 0x7feac810bd00 2024-04-25T00:28:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.498+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a4b40 con 0x7feac810bd00 2024-04-25T00:28:00.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.500+0000 7feaaffff640 1 -- 172.21.15.45:0/2444012259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feac0049070 con 0x7feac810bd00 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 Processor -- start 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 -- start start 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae99c60 0x55be0aea8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aeaad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55be0adfe670 con 0x55be0ae992b0 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55be0ada6970 con 0x55be0aea8980 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feacfff6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be0adfa630 con 0x55be0ae99c60 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57982/0 (socket says 172.21.15.45:57982) 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 -- 172.21.15.45:0/2991460077 learned_addr learned my addr 172.21.15.45:0/2991460077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac6d9d640 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae99c60 0x55be0aea8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feaaffff640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 0x7feab00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 -- 172.21.15.45:0/2991460077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae99c60 msgr2=0x55be0aea8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae99c60 0x55be0aea8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 -- 172.21.15.45:0/2991460077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 msgr2=0x55be0aeaad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aeaad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 -- 172.21.15.45:0/2991460077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0adca400 con 0x55be0ae992b0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac759e640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 0x7feab00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feaaffff640 1 -- 172.21.15.45:0/2444012259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feac00f0210 con 0x7feac810bd00 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.501+0000 7feac659c640 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7feabc0093e0 tx=0x7feabc013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c472658d530bdbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feaad7fa640 1 -- 172.21.15.45:0/2991460077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004050 con 0x55be0ae992b0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feaad7fa640 1 -- 172.21.15.45:0/2991460077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc017030 con 0x55be0ae992b0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feaad7fa640 1 -- 172.21.15.45:0/2991460077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc010430 con 0x55be0ae992b0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 -- 172.21.15.45:0/2991460077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 msgr2=0x55be0ae99690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feac759e640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 0x7feab00a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feac00bc070 tx=0x7feac002be70 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7feabc0093e0 tx=0x7feabc013830 comp rx=0 tx=0).stop 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 -- 172.21.15.45:0/2991460077 shutdown_connections 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aeaad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae99c60 0x55be0aea8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 --2- 172.21.15.45:0/2991460077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae992b0 0x55be0ae99690 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 -- 172.21.15.45:0/2991460077 >> 172.21.15.45:0/2991460077 conn(0x55be0ae958a0 msgr2=0x55be0ae95c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 -- 172.21.15.45:0/2991460077 shutdown_connections 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.502+0000 7feacfff6740 1 -- 172.21.15.45:0/2991460077 wait complete. 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 -- start start 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 Processor -- start 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aea8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aea5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55be0aeab740 con 0x55be0ae99c60 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55be0adba050 con 0x55be0aea8980 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be0aeab8c0 con 0x55be0ae992b0 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feacfff6740 10 client.0 _ll_register_callbacks cb 0x55be0ab11be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57988/0 (socket says 172.21.15.45:57988) 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 -- 172.21.15.45:0/628659990 learned_addr learned my addr 172.21.15.45:0/628659990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac659c640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aea5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac659c640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aea8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 -- 172.21.15.45:0/628659990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 msgr2=0x55be0aea8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aea8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 -- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 msgr2=0x55be0aea5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aea5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.503+0000 7feac6d9d640 1 -- 172.21.15.45:0/628659990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0ad8bdc0 con 0x55be0ae99c60 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feac6d9d640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7feab8013710 tx=0x7feab8013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c061da87af5e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8020060 con 0x55be0ae99c60 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feab8011d50 con 0x55be0ae99c60 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc017030 con 0x55be0ae99c60 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feab8027050 con 0x55be0ae99c60 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 10 client.17349 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be0ab123f0 con 0x55be0ae99c60 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc017030 con 0x55be0ae99c60 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feab801a030 con 0x55be0ae99c60 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 10 client.17349 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 10 client.17349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 20 client.17349 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0aeb8a40 0x55be0aebae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 4 client.17349 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.504+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be0ae0bbb0 con 0x55be0aeb8a40 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.505+0000 7feac659c640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0aeb8a40 0x55be0aebae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.505+0000 7feac659c640 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0aeb8a40 0x55be0aebae60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81d776c6ca154d7 server_cookie=e6da2c58dfc5da03 in_seq=0 out_seq=0 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.505+0000 7feaaaff5640 10 client.17349 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feaaaff5640 1 -- 172.21.15.45:0/628659990 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be0ae0bbb0 con 0x55be0aeb8a40 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feaaaff5640 10 client.17349 handle_command_reply: tid=0 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 client.17349 shutdown 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 msgr2=0x55be0aea5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7feab8013710 tx=0x7feab8013bd0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 shutdown_connections 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0aeb8a40 0x55be0aebae60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0aea8980 0x55be0aea5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aea5570 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feaaaff5640 0 client.17349 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 --2- 172.21.15.45:0/628659990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aea8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 >> 172.21.15.45:0/628659990 conn(0x55be0ae958a0 msgr2=0x55be0ae9b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 shutdown_connections 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 1 -- 172.21.15.45:0/628659990 wait complete. 2024-04-25T00:28:00.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.506+0000 7feacfff6740 20 client.17349 trim_cache size 0 max 16384 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 Processor -- start 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 -- start start 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aeb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae99c60 0x55be0aec0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ac870 con 0x55be0aea8980 2024-04-25T00:28:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ae3b0 con 0x55be0ae99c60 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.519+0000 7feacfff6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feac81acf40 con 0x55be0ae992b0 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58000/0 (socket says 172.21.15.45:58000) 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 -- 172.21.15.45:0/3257839546 learned_addr learned my addr 172.21.15.45:0/3257839546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aeb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac6d9d640 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae99c60 0x55be0aec0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 -- 172.21.15.45:0/3257839546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 msgr2=0x55be0aeb4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aeb4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 -- 172.21.15.45:0/3257839546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae99c60 msgr2=0x55be0aec0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae99c60 0x55be0aec0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 -- 172.21.15.45:0/3257839546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be0abdc7d0 con 0x55be0aea8980 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feac659c640 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7feabc003c40 tx=0x7feabc012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e2572752092da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feaacff9640 1 -- 172.21.15.45:0/3257839546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc003730 con 0x55be0aea8980 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feaacff9640 1 -- 172.21.15.45:0/3257839546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc017040 con 0x55be0aea8980 2024-04-25T00:28:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.520+0000 7feaacff9640 1 -- 172.21.15.45:0/3257839546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc02b440 con 0x55be0aea8980 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- 172.21.15.45:0/3257839546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 msgr2=0x55be0aec0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7feabc003c40 tx=0x7feabc012040 comp rx=0 tx=0).stop 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- 172.21.15.45:0/3257839546 shutdown_connections 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0aea8980 0x55be0aec0c50 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae99c60 0x55be0aec0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- 172.21.15.45:0/3257839546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0ae992b0 0x55be0aeb4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- 172.21.15.45:0/3257839546 >> 172.21.15.45:0/3257839546 conn(0x55be0ae958a0 msgr2=0x55be0aea7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- 172.21.15.45:0/3257839546 shutdown_connections 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- 172.21.15.45:0/3257839546 wait complete. 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- start start 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 Processor -- start 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae992b0 0x55be0aeade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 0x55be0af16f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feac81adce0 con 0x55be0ae99c60 2024-04-25T00:28:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feac81ad610 con 0x55be0ae992b0 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feac81aa850 con 0x55be0aea8980 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feacfff6740 10 client.0 _ll_register_callbacks cb 0x7feab80096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feac6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58002/0 (socket says 172.21.15.45:58002) 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.521+0000 7feac6d9d640 1 -- 172.21.15.45:0/3701116435 learned_addr learned my addr 172.21.15.45:0/3701116435 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac659c640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae992b0 0x55be0aeade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac659c640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 0x55be0af16f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 -- 172.21.15.45:0/3701116435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 msgr2=0x55be0af16f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 0x55be0af16f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 -- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae992b0 msgr2=0x55be0aeade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae992b0 0x55be0aeade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 -- 172.21.15.45:0/3701116435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be0af17540 con 0x55be0ae99c60 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac659c640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 0x55be0af16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feac6d9d640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feab800e6e0 tx=0x7feab8011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84be33a8336c3a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feaaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab800ece0 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feab8005040 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc017040 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feab8009d10 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feacfff6740 10 client.17355 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be0af1e8f0 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.522+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feabc017040 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feab802a070 con 0x55be0ae99c60 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 10 client.17355 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 10 client.17355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 20 client.17355 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0af1f0f0 0x55be0af21510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 4 client.17355 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be0ae05f00 con 0x55be0af1f0f0 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feac659c640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0af1f0f0 0x55be0af21510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feac659c640 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0af1f0f0 0x55be0af21510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b774a1ebb6a0a42 server_cookie=6a5fa89ab2ef8184 in_seq=0 out_seq=0 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.523+0000 7feaaaff5640 10 client.17355 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feaaaff5640 1 -- 172.21.15.45:0/3701116435 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be0ae05f00 con 0x55be0af1f0f0 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feaaaff5640 10 client.17355 handle_command_reply: tid=0 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 client.17355 shutdown 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 msgr2=0x55be0aeae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feab800e6e0 tx=0x7feab8011850 comp rx=0 tx=0).stop 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 shutdown_connections 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55be0af1f0f0 0x55be0af21510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be0aea8980 0x55be0af16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be0ae99c60 0x55be0aeae340 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 --2- 172.21.15.45:0/3701116435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be0ae992b0 0x55be0aeade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feaaaff5640 0 client.17355 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 >> 172.21.15.45:0/3701116435 conn(0x55be0ae958a0 msgr2=0x55be0ae9ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 shutdown_connections 2024-04-25T00:28:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 1 -- 172.21.15.45:0/3701116435 wait complete. 2024-04-25T00:28:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacfff6740 20 client.17355 trim_cache size 0 max 16384 2024-04-25T00:28:00.531 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 msgr2=0x7feab00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 0x7feab00a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feac00bc070 tx=0x7feac002be70 comp rx=0 tx=0).stop 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 msgr2=0x7feac819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7feac0005cc0 tx=0x7feac0003b80 comp rx=0 tx=0).stop 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 shutdown_connections 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feab00a2d00 0x7feab00a51c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feac810bd00 0x7feac819c890 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feac81032e0 0x7feac8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 --2- 172.21.15.45:0/2444012259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feac81029c0 0x7feac8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.524+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 >> 172.21.15.45:0/2444012259 conn(0x7feac8077740 msgr2=0x7feac80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.525+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 shutdown_connections 2024-04-25T00:28:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.525+0000 7feacd212640 1 -- 172.21.15.45:0/2444012259 wait complete. 2024-04-25T00:28:00.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:28:00.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- 172.21.15.45:0/1924628931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc100610 msgr2=0x7feabc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 --2- 172.21.15.45:0/1924628931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc100610 0x7feabc100a90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009930 tx=0x7feaa802a420 comp rx=0 tx=0).stop 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- 172.21.15.45:0/1924628931 shutdown_connections 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 --2- 172.21.15.45:0/1924628931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 0x7feabc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 --2- 172.21.15.45:0/1924628931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc100610 0x7feabc100a90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 --2- 172.21.15.45:0/1924628931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc1066b0 0x7feabc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- 172.21.15.45:0/1924628931 >> 172.21.15.45:0/1924628931 conn(0x7feabc077740 msgr2=0x7feabc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- 172.21.15.45:0/1924628931 shutdown_connections 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- 172.21.15.45:0/1924628931 wait complete. 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 Processor -- start 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.640+0000 7feac30fc640 1 -- start start 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100610 0x7feabc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 0x7feabc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feabc111b10 con 0x7feabc1066b0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feabc111990 con 0x7feabc100610 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac30fc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feabc111c90 con 0x7feabc100fd0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58024/0 (socket says 172.21.15.45:58024) 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 -- 172.21.15.45:0/6285113 learned_addr learned my addr 172.21.15.45:0/6285113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feabbfff640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 0x7feabc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac0e98640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100610 0x7feabc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 -- 172.21.15.45:0/6285113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 msgr2=0x7feabc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 0x7feabc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 -- 172.21.15.45:0/6285113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100610 msgr2=0x7feabc197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100610 0x7feabc197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 -- 172.21.15.45:0/6285113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc19cf60 con 0x7feabc1066b0 2024-04-25T00:28:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feac1699640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feab400eaa0 tx=0x7feab400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.641+0000 7feab9ffb640 1 -- 172.21.15.45:0/6285113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400cdb0 con 0x7feabc1066b0 2024-04-25T00:28:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.642+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc19d250 con 0x7feabc1066b0 2024-04-25T00:28:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.642+0000 7feab9ffb640 1 -- 172.21.15.45:0/6285113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab4005020 con 0x7feabc1066b0 2024-04-25T00:28:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.642+0000 7feab9ffb640 1 -- 172.21.15.45:0/6285113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab40116b0 con 0x7feabc1066b0 2024-04-25T00:28:00.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.642+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a4ae0 con 0x7feabc1066b0 2024-04-25T00:28:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feab9ffb640 1 -- 172.21.15.45:0/6285113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feab4005560 con 0x7feabc1066b0 2024-04-25T00:28:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 Processor -- start 2024-04-25T00:28:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 -- start start 2024-04-25T00:28:00.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d574225670 con 0x55d5742c0ea0 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d5741cd970 con 0x55d5742cb570 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac5e0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d574221630 con 0x55d5742c0580 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feab9ffb640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 0x7feaac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.644+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58038/0 (socket says 172.21.15.45:58038) 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 -- 172.21.15.45:0/1915530901 learned_addr learned my addr 172.21.15.45:0/1915530901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feabbfff640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feab9ffb640 1 -- 172.21.15.45:0/6285113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feab40cf950 con 0x7feabc1066b0 2024-04-25T00:28:00.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac1699640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 0x7feaac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feabbfff640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 -- 172.21.15.45:0/1915530901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 msgr2=0x55d5742c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 -- 172.21.15.45:0/1915530901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 msgr2=0x55d5742cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 -- 172.21.15.45:0/1915530901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5741f1400 con 0x55d5742c0ea0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feabbfff640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac0e98640 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7feab000bcb0 tx=0x7feab00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32979bda34945235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feac1699640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 0x7feaac0a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feab4009270 tx=0x7feab40cfc50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feaa6ffd640 1 -- 172.21.15.45:0/1915530901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0004020 con 0x55d5742c0ea0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feaa6ffd640 1 -- 172.21.15.45:0/1915530901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab0005500 con 0x55d5742c0ea0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.645+0000 7feaa6ffd640 1 -- 172.21.15.45:0/1915530901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0004d90 con 0x55d5742c0ea0 2024-04-25T00:28:00.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- 172.21.15.45:0/1915530901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 msgr2=0x55d5742cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7feab000bcb0 tx=0x7feab00107a0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- 172.21.15.45:0/1915530901 shutdown_connections 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742cb030 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 --2- 172.21.15.45:0/1915530901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- 172.21.15.45:0/1915530901 >> 172.21.15.45:0/1915530901 conn(0x55d5742bc8a0 msgr2=0x55d5742bcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- 172.21.15.45:0/1915530901 shutdown_connections 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- 172.21.15.45:0/1915530901 wait complete. 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 -- start start 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.646+0000 7feac5e0a740 1 Processor -- start 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742c0580 0x55d5742cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742cb570 0x55d5742c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d5742ce360 con 0x55d5742c0ea0 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d5741e1050 con 0x55d5742c0580 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d5742ce4e0 con 0x55d5742cb570 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac5e0a740 10 client.0 _ll_register_callbacks cb 0x55d573f38be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58040/0 (socket says 172.21.15.45:58040) 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 -- 172.21.15.45:0/623988877 learned_addr learned my addr 172.21.15.45:0/623988877 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feabbfff640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742c0580 0x55d5742cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feabbfff640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742cb570 0x55d5742c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 -- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742cb570 msgr2=0x55d5742c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742cb570 0x55d5742c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 -- 172.21.15.45:0/623988877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742c0580 msgr2=0x55d5742cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742c0580 0x55d5742cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 -- 172.21.15.45:0/623988877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5741b2dc0 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7feac0e98640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c00 tx=0x7feab000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3395ac4880a8583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.647+0000 7fea9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab002b060 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feab0004d00 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab0005500 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feab0032050 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 10 client.17367 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d573f393f0 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab0005500 con 0x55d5742c0ea0 2024-04-25T00:28:00.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feab0011b10 con 0x55d5742c0ea0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 10 client.17367 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 10 client.17367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 20 client.17367 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d5742db4a0 0x55d5742dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 4 client.17367 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d574232bb0 con 0x55d5742db4a0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.648+0000 7feabbfff640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d5742db4a0 0x55d5742dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.649+0000 7fea9bfff640 10 client.17367 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.649+0000 7feabbfff640 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d5742db4a0 0x55d5742dd890 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e60e7c5de9db20c server_cookie=76286897553583bd in_seq=0 out_seq=0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7fea9bfff640 1 -- 172.21.15.45:0/623988877 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d574232bb0 con 0x55d5742db4a0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7fea9bfff640 10 client.17367 handle_command_reply: tid=0 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 client.17367 shutdown 2024-04-25T00:28:00.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 msgr2=0x55d5742c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c00 tx=0x7feab000bfd0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 shutdown_connections 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d5742db4a0 0x55d5742dd890 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742cb570 0x55d5742c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742c8160 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7fea9bfff640 0 client.17367 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 --2- 172.21.15.45:0/623988877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742c0580 0x55d5742cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 >> 172.21.15.45:0/623988877 conn(0x55d5742bc8a0 msgr2=0x55d5742bbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 shutdown_connections 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 1 -- 172.21.15.45:0/623988877 wait complete. 2024-04-25T00:28:00.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.650+0000 7feac5e0a740 20 client.17367 trim_cache size 0 max 16384 2024-04-25T00:28:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 Processor -- start 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 -- start start 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1aeb30 con 0x55d5742c0ea0 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b0670 con 0x55d5742cb570 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.663+0000 7feac5e0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1af200 con 0x55d5742c0580 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58042/0 (socket says 172.21.15.45:58042) 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 -- 172.21.15.45:0/2544523293 learned_addr learned my addr 172.21.15.45:0/2544523293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feabbfff640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feabbfff640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 -- 172.21.15.45:0/2544523293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 msgr2=0x55d5742c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 -- 172.21.15.45:0/2544523293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 msgr2=0x55d5742ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 -- 172.21.15.45:0/2544523293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5740037d0 con 0x55d5742c0ea0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feabbfff640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feac0e98640 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7feab0011010 tx=0x7feab0010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869ba8b91057c618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feaa67fc640 1 -- 172.21.15.45:0/2544523293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab002b030 con 0x55d5742c0ea0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feaa67fc640 1 -- 172.21.15.45:0/2544523293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab001bdd0 con 0x55d5742c0ea0 2024-04-25T00:28:00.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.664+0000 7feaa67fc640 1 -- 172.21.15.45:0/2544523293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0011800 con 0x55d5742c0ea0 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- 172.21.15.45:0/2544523293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 msgr2=0x55d5742bb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7feab0011010 tx=0x7feab0010e30 comp rx=0 tx=0).stop 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- 172.21.15.45:0/2544523293 shutdown_connections 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0ea0 0x55d5742bb9c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 --2- 172.21.15.45:0/2544523293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0580 0x55d5742c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- 172.21.15.45:0/2544523293 >> 172.21.15.45:0/2544523293 conn(0x55d5742bc8a0 msgr2=0x55d5742bb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- 172.21.15.45:0/2544523293 shutdown_connections 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- 172.21.15.45:0/2544523293 wait complete. 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 -- start start 2024-04-25T00:28:00.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.665+0000 7feac5e0a740 1 Processor -- start 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0ea0 0x55d5742ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1affa0 con 0x55d5742c0580 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1af8d0 con 0x55d5742cb570 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1acb10 con 0x55d5742c0ea0 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 10 client.0 _ll_register_callbacks cb 0x7feab000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58052/0 (socket says 172.21.15.45:58052) 2024-04-25T00:28:00.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 -- 172.21.15.45:0/276641621 learned_addr learned my addr 172.21.15.45:0/276641621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac0e98640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0ea0 0x55d5742ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 -- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0ea0 msgr2=0x55d5742ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0ea0 0x55d5742ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 -- 172.21.15.45:0/276641621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 msgr2=0x55d5742cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 -- 172.21.15.45:0/276641621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5742cf780 con 0x55d5742c0580 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feabbfff640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feaa8000c00 tx=0x7feab00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647be8e6f4013717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7fea9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8053030 con 0x55d5742c0580 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7feaa8046070 con 0x55d5742c0580 2024-04-25T00:28:00.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.666+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab001bdd0 con 0x55d5742c0580 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feaa8035040 con 0x55d5742c0580 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 10 client.17373 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5743413f0 con 0x55d5742c0580 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab001bdd0 con 0x55d5742c0580 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feaa804c040 con 0x55d5742c0580 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 10 client.17373 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 10 client.17373 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 20 client.17373 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d574341b40 0x55d574343f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 4 client.17373 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5742c8a30 con 0x55d574341b40 2024-04-25T00:28:00.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.667+0000 7feac0e98640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d574341b40 0x55d574343f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac0e98640 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d574341b40 0x55d574343f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3c614a233d39202 server_cookie=2de5fe50f59fa8cb in_seq=0 out_seq=0 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7fea9bfff640 10 client.17373 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7fea9bfff640 1 -- 172.21.15.45:0/276641621 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5742c8a30 con 0x55d574341b40 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7fea9bfff640 10 client.17373 handle_command_reply: tid=0 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 client.17373 shutdown 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 msgr2=0x55d5742d3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feaa8000c00 tx=0x7feab00045b0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 shutdown_connections 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d574341b40 0x55d574343f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5742cb570 0x55d5742cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5742c0ea0 0x55d5742ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7fea9bfff640 0 client.17373 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 --2- 172.21.15.45:0/276641621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5742c0580 0x55d5742d3810 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 >> 172.21.15.45:0/276641621 conn(0x55d5742bc8a0 msgr2=0x55d5742d6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 shutdown_connections 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 1 -- 172.21.15.45:0/276641621 wait complete. 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.668+0000 7feac5e0a740 20 client.17373 trim_cache size 0 max 16384 2024-04-25T00:28:00.678 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:00.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 msgr2=0x7feaac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 0x7feaac0a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feab4009270 tx=0x7feab40cfc50 comp rx=0 tx=0).stop 2024-04-25T00:28:00.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 msgr2=0x7feabc19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feab400eaa0 tx=0x7feab400ef60 comp rx=0 tx=0).stop 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 shutdown_connections 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feaac0a2d00 0x7feaac0a51c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feabc1066b0 0x7feabc19c830 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feabc100fd0 0x7feabc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 --2- 172.21.15.45:0/6285113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc100610 0x7feabc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 >> 172.21.15.45:0/6285113 conn(0x7feabc077740 msgr2=0x7feabc0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 shutdown_connections 2024-04-25T00:28:00.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.669+0000 7feac30fc640 1 -- 172.21.15.45:0/6285113 wait complete. 2024-04-25T00:28:00.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:28:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.784+0000 7f27c8d61640 1 -- 172.21.15.45:0/4256721679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c41029c0 msgr2=0x7f27c4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.784+0000 7f27c8d61640 1 --2- 172.21.15.45:0/4256721679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c41029c0 0x7f27c4102da0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009ca0 tx=0x7f27b402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 -- 172.21.15.45:0/4256721679 shutdown_connections 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 --2- 172.21.15.45:0/4256721679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c410bd00 0x7f27c410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 --2- 172.21.15.45:0/4256721679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41032e0 0x7f27c410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 --2- 172.21.15.45:0/4256721679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c41029c0 0x7f27c4102da0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 -- 172.21.15.45:0/4256721679 >> 172.21.15.45:0/4256721679 conn(0x7f27c4077740 msgr2=0x7f27c4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 -- 172.21.15.45:0/4256721679 shutdown_connections 2024-04-25T00:28:00.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 -- 172.21.15.45:0/4256721679 wait complete. 2024-04-25T00:28:00.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 Processor -- start 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.785+0000 7f27c8d61640 1 -- start start 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41029c0 0x7f27c4100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c41032e0 0x7f27c4100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111b00 con 0x7f27c410bd00 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111980 con 0x7f27c41029c0 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4111c80 con 0x7f27c41032e0 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58066/0 (socket says 172.21.15.45:58066) 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 -- 172.21.15.45:0/3721495781 learned_addr learned my addr 172.21.15.45:0/3721495781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c259c640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41029c0 0x7f27c4100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c41032e0 0x7f27c4100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 -- 172.21.15.45:0/3721495781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c41032e0 msgr2=0x7f27c4100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c41032e0 0x7f27c4100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 -- 172.21.15.45:0/3721495781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41029c0 msgr2=0x7f27c4100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41029c0 0x7f27c4100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 -- 172.21.15.45:0/3721495781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c41a64e0 con 0x7f27c410bd00 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c2d9d640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c970 tx=0x7f27bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27b37fe640 1 -- 172.21.15.45:0/3721495781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc007bb0 con 0x7f27c410bd00 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27b37fe640 1 -- 172.21.15.45:0/3721495781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27bc007d50 con 0x7f27c410bd00 2024-04-25T00:28:00.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.786+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c41a67d0 con 0x7f27c410bd00 2024-04-25T00:28:00.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.787+0000 7f27b37fe640 1 -- 172.21.15.45:0/3721495781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc018800 con 0x7f27c410bd00 2024-04-25T00:28:00.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.787+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a6cd0 con 0x7f27c410bd00 2024-04-25T00:28:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27b37fe640 1 -- 172.21.15.45:0/3721495781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f27bc00fa50 con 0x7f27c410bd00 2024-04-25T00:28:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 Processor -- start 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 -- start start 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fceb580 0x56105fceb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56105fc50670 con 0x56105fcf6570 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56105fbf8970 con 0x56105fceb580 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27cba6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56105fc4c630 con 0x56105fcebea0 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27b37fe640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 0x7f27ac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.789+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58070/0 (socket says 172.21.15.45:58070) 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c2d9d640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 0x7f27ac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2369554540 learned_addr learned my addr 172.21.15.45:0/2369554540 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fceb580 0x56105fceb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c259c640 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27b37fe640 1 -- 172.21.15.45:0/3721495781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f27bc0cf7b0 con 0x7f27c410bd00 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2369554540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 msgr2=0x56105fcf6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2369554540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fceb580 msgr2=0x56105fceb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fceb580 0x56105fceb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2369554540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fc1c400 con 0x56105fcf6570 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f27b80149c0 tx=0x7f27b800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd3f6118c33eb08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27c2d9d640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 0x7f27ac0a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f27bc09b900 tx=0x7f27bc0c6500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27b0ff9640 1 -- 172.21.15.45:0/2369554540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80142d0 con 0x56105fcf6570 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27b0ff9640 1 -- 172.21.15.45:0/2369554540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b800fc40 con 0x56105fcf6570 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.790+0000 7f27b0ff9640 1 -- 172.21.15.45:0/2369554540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8012ba0 con 0x56105fcf6570 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- 172.21.15.45:0/2369554540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 msgr2=0x56105fcf8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f27b80149c0 tx=0x7f27b800c4d0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- 172.21.15.45:0/2369554540 shutdown_connections 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fcf8960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2369554540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fceb580 0x56105fceb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- 172.21.15.45:0/2369554540 >> 172.21.15.45:0/2369554540 conn(0x56105fce78a0 msgr2=0x56105fce7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- 172.21.15.45:0/2369554540 shutdown_connections 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- 172.21.15.45:0/2369554540 wait complete. 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 -- start start 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.791+0000 7f27cba6e740 1 Processor -- start 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcf6570 0x56105fcf3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56105fcf9360 con 0x56105fceb580 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56105fc0c050 con 0x56105fcf6570 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56105fcf94e0 con 0x56105fcebea0 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 10 client.0 _ll_register_callbacks cb 0x56105f963be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58078/0 (socket says 172.21.15.45:58078) 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 -- 172.21.15.45:0/690037857 learned_addr learned my addr 172.21.15.45:0/690037857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c259c640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcf6570 0x56105fcf3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 -- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 msgr2=0x56105fcf31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 -- 172.21.15.45:0/690037857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcf6570 msgr2=0x56105fcf3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcf6570 0x56105fcf3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 -- 172.21.15.45:0/690037857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105fbdddc0 con 0x56105fceb580 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c259c640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f27b80058c0 tx=0x7f27b8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850dbfdc7f299f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80142d0 con 0x56105fceb580 2024-04-25T00:28:00.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f27b800f730 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.792+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b800fc40 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27b8034050 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 10 client.17385 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105f9643f0 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b800fc40 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27b802d060 con 0x56105fceb580 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 10 client.17385 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 10 client.17385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 20 client.17385 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd064b0 0x56105fd088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 4 client.17385 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105fc5dbb0 con 0x56105fd064b0 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.793+0000 7f27c259c640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd064b0 0x56105fd088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.794+0000 7f27c259c640 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd064b0 0x56105fd088a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60128e36c424aa0b server_cookie=7a65365e2c15d89c in_seq=0 out_seq=0 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.794+0000 7f27a9ffb640 10 client.17385 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.794+0000 7f27a9ffb640 1 -- 172.21.15.45:0/690037857 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56105fc5dbb0 con 0x56105fd064b0 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.794+0000 7f27a9ffb640 10 client.17385 handle_command_reply: tid=0 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.794+0000 7f27cba6e740 1 client.17385 shutdown 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 msgr2=0x56105fcf5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f27b80058c0 tx=0x7f27b8005850 comp rx=0 tx=0).stop 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 shutdown_connections 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd064b0 0x56105fd088a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcf6570 0x56105fcf3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcebea0 0x56105fcf31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27a9ffb640 0 client.17385 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 --2- 172.21.15.45:0/690037857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcf5ec0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 >> 172.21.15.45:0/690037857 conn(0x56105fce78a0 msgr2=0x56105fced6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 shutdown_connections 2024-04-25T00:28:00.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 1 -- 172.21.15.45:0/690037857 wait complete. 2024-04-25T00:28:00.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.795+0000 7f27cba6e740 20 client.17385 trim_cache size 0 max 16384 2024-04-25T00:28:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 Processor -- start 2024-04-25T00:28:00.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 -- start start 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fceb580 0x56105fcf2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fd0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b0d30 con 0x56105fcf6570 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b2870 con 0x56105fcebea0 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.808+0000 7f27cba6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b1400 con 0x56105fceb580 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58086/0 (socket says 172.21.15.45:58086) 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 -- 172.21.15.45:0/1619298492 learned_addr learned my addr 172.21.15.45:0/1619298492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fceb580 0x56105fcf2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c259c640 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fd0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 -- 172.21.15.45:0/1619298492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fceb580 msgr2=0x56105fcf2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fceb580 0x56105fcf2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 -- 172.21.15.45:0/1619298492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 msgr2=0x56105fd0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fd0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 -- 172.21.15.45:0/1619298492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fa2e7d0 con 0x56105fcf6570 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f27b8012d80 tx=0x7f27b8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5dad6dced42af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27abfff640 1 -- 172.21.15.45:0/1619298492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8012500 con 0x56105fcf6570 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27abfff640 1 -- 172.21.15.45:0/1619298492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b8014070 con 0x56105fcf6570 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.809+0000 7f27abfff640 1 -- 172.21.15.45:0/1619298492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b802f430 con 0x56105fcf6570 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- 172.21.15.45:0/1619298492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 msgr2=0x56105fd0e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f27b8012d80 tx=0x7f27b8005970 comp rx=0 tx=0).stop 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- 172.21.15.45:0/1619298492 shutdown_connections 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fcf6570 0x56105fd0e570 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fd0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- 172.21.15.45:0/1619298492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fceb580 0x56105fcf2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- 172.21.15.45:0/1619298492 >> 172.21.15.45:0/1619298492 conn(0x56105fce78a0 msgr2=0x56105fd01210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- 172.21.15.45:0/1619298492 shutdown_connections 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- 172.21.15.45:0/1619298492 wait complete. 2024-04-25T00:28:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- start start 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 Processor -- start 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fcfbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcf6570 0x56105fd6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b21a0 con 0x56105fceb580 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b1ad0 con 0x56105fcebea0 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.810+0000 7f27cba6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41aed10 con 0x56105fcf6570 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27cba6e740 10 client.0 _ll_register_callbacks cb 0x7f27b800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58102/0 (socket says 172.21.15.45:58102) 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2719532054 learned_addr learned my addr 172.21.15.45:0/2719532054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c259c640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fcfbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcf6570 msgr2=0x56105fd6a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcf6570 0x56105fd6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2719532054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 msgr2=0x56105fcfbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fcfbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 -- 172.21.15.45:0/2719532054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105fd6ade0 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27c1d9b640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f27b800cf00 tx=0x7f27b800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e041a2c869491dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8023040 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f27b800cb90 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b8014070 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27b800fc40 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27cba6e740 10 client.17391 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.811+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105fd6c0e0 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b8014070 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27b802d070 con 0x56105fceb580 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 10 client.17391 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 10 client.17391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 20 client.17391 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd6c730 0x56105fd6eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 4 client.17391 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105fc57f00 con 0x56105fd6c730 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27c259c640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd6c730 0x56105fd6eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27c259c640 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd6c730 0x56105fd6eb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81fadab4f06fe68b server_cookie=536beef0587c516 in_seq=0 out_seq=0 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.812+0000 7f27a9ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27a9ffb640 1 -- 172.21.15.45:0/2719532054 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105fc57f00 con 0x56105fd6c730 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27a9ffb640 10 client.17391 handle_command_reply: tid=0 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 client.17391 shutdown 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 msgr2=0x56105fcfb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f27b800cf00 tx=0x7f27b800ca40 comp rx=0 tx=0).stop 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 shutdown_connections 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56105fd6c730 0x56105fd6eb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56105fcf6570 0x56105fd6a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56105fcebea0 0x56105fcfbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 --2- 172.21.15.45:0/2719532054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56105fceb580 0x56105fcfb8d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27a9ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 >> 172.21.15.45:0/2719532054 conn(0x56105fce78a0 msgr2=0x56105fd01aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 shutdown_connections 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 1 -- 172.21.15.45:0/2719532054 wait complete. 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27cba6e740 20 client.17391 trim_cache size 0 max 16384 2024-04-25T00:28:00.820 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 msgr2=0x7f27ac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 0x7f27ac0a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f27bc09b900 tx=0x7f27bc0c6500 comp rx=0 tx=0).stop 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 msgr2=0x7f27c4101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c970 tx=0x7f27bc00ce30 comp rx=0 tx=0).stop 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 shutdown_connections 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f27ac0a2d00 0x7f27ac0a51c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c410bd00 0x7f27c4101090 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27c41032e0 0x7f27c4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 --2- 172.21.15.45:0/3721495781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27c41029c0 0x7f27c4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.813+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 >> 172.21.15.45:0/3721495781 conn(0x7f27c4077740 msgr2=0x7f27c40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.814+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 shutdown_connections 2024-04-25T00:28:00.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.814+0000 7f27c8d61640 1 -- 172.21.15.45:0/3721495781 wait complete. 2024-04-25T00:28:00.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 -- 172.21.15.45:0/4139491208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e406a300 msgr2=0x7f20e406a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 --2- 172.21.15.45:0/4139491208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e406a300 0x7f20e406a760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d402a410 comp rx=0 tx=0).stop 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 -- 172.21.15.45:0/4139491208 shutdown_connections 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 --2- 172.21.15.45:0/4139491208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e4104170 0x7f20e41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 --2- 172.21.15.45:0/4139491208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e406a300 0x7f20e406a760 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 --2- 172.21.15.45:0/4139491208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 0x7f20e4069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 -- 172.21.15.45:0/4139491208 >> 172.21.15.45:0/4139491208 conn(0x7f20e40776a0 msgr2=0x7f20e4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 -- 172.21.15.45:0/4139491208 shutdown_connections 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.928+0000 7f20eba12640 1 -- 172.21.15.45:0/4139491208 wait complete. 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 Processor -- start 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 -- start start 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e406a300 0x7f20e419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 0x7f20e41a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41180f0 con 0x7f20e4104170 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4117f70 con 0x7f20e406a300 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20eba12640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4118270 con 0x7f20e4106670 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58122/0 (socket says 172.21.15.45:58122) 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 -- 172.21.15.45:0/3602471132 learned_addr learned my addr 172.21.15.45:0/3602471132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e406a300 0x7f20e419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 0x7f20e41a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 -- 172.21.15.45:0/3602471132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 msgr2=0x7f20e41a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 0x7f20e41a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 -- 172.21.15.45:0/3602471132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e406a300 msgr2=0x7f20e419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e406a300 0x7f20e419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.929+0000 7f20e8fad640 1 -- 172.21.15.45:0/3602471132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e41a35c0 con 0x7f20e4104170 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f20d4006010 tx=0x7f20d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20da7fc640 1 -- 172.21.15.45:0/3602471132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d403f030 con 0x7f20e4104170 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e41a3850 con 0x7f20e4104170 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20da7fc640 1 -- 172.21.15.45:0/3602471132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4003ce0 con 0x7f20e4104170 2024-04-25T00:28:00.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20da7fc640 1 -- 172.21.15.45:0/3602471132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4035070 con 0x7f20e4104170 2024-04-25T00:28:00.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.930+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41ab140 con 0x7f20e4104170 2024-04-25T00:28:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.932+0000 7f20ee7fa740 1 Processor -- start 2024-04-25T00:28:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.932+0000 7f20da7fc640 1 -- 172.21.15.45:0/3602471132 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20d4042030 con 0x7f20e4104170 2024-04-25T00:28:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.932+0000 7f20ee7fa740 1 -- start start 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660580 0x55a6af660960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af66b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6af5c5670 con 0x55a6af66b570 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a6af56d970 con 0x55a6af660ea0 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a6af5c1630 con 0x55a6af660580 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58128/0 (socket says 172.21.15.45:58128) 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 -- 172.21.15.45:0/3860883871 learned_addr learned my addr 172.21.15.45:0/3860883871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660580 0x55a6af660960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af66b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20da7fc640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 0x7f20d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 -- 172.21.15.45:0/3860883871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660580 msgr2=0x55a6af660960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660580 0x55a6af660960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 -- 172.21.15.45:0/3860883871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 msgr2=0x55a6af66b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af66b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 -- 172.21.15.45:0/3860883871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6af591400 con 0x55a6af66b570 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 0x7f20d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ecc70 tx=0x7f20d40b7990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c26828177eaae85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20da7fc640 1 -- 172.21.15.45:0/3602471132 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f20d40ef910 con 0x7f20e4104170 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.933+0000 7f20cf7fe640 1 -- 172.21.15.45:0/3860883871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4049030 con 0x55a6af66b570 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20cf7fe640 1 -- 172.21.15.45:0/3860883871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d40a6030 con 0x55a6af66b570 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20cf7fe640 1 -- 172.21.15.45:0/3860883871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d40bb780 con 0x55a6af66b570 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3860883871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 msgr2=0x55a6af66d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 0x7f20d00a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f20dc005fd0 tx=0x7f20dc004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ecc70 tx=0x7f20d40b7990 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3860883871 shutdown_connections 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af66d960 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af66b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3860883871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660580 0x55a6af660960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3860883871 >> 172.21.15.45:0/3860883871 conn(0x55a6af65c8a0 msgr2=0x55a6af65cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3860883871 shutdown_connections 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.934+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3860883871 wait complete. 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 -- start start 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 Processor -- start 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af6681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af668710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6af5c5670 con 0x55a6af660580 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a6af581050 con 0x55a6af660ea0 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a6af66e120 con 0x55a6af66b570 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20ee7fa740 10 client.0 _ll_register_callbacks cb 0x55a6af2d8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20e97ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20e97ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58138/0 (socket says 172.21.15.45:58138) 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20e97ae640 1 -- 172.21.15.45:0/3851192947 learned_addr learned my addr 172.21.15.45:0/3851192947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.935+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af6681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af668710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 -- 172.21.15.45:0/3851192947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 msgr2=0x55a6af668710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af668710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 -- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 msgr2=0x55a6af6681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af6681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 -- 172.21.15.45:0/3851192947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6af552dc0 con 0x55a6af660580 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af668710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b9f0 tx=0x7f20e000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41bde9cd2fb7652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0003e20 con 0x55a6af660580 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f20e0011440 con 0x55a6af660580 2024-04-25T00:28:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d40a6030 con 0x55a6af660580 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20e0011640 con 0x55a6af660580 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ee7fa740 10 client.17403 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6af2d93f0 con 0x55a6af660580 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d40a6030 con 0x55a6af660580 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.936+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20e001baa0 con 0x55a6af660580 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 10 client.17403 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 20 client.17403 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af67b330 0x55a6af67d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 4 client.17403 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6af5d2bb0 con 0x55a6af67b330 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af67b330 0x55a6af67d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af67b330 0x55a6af67d7a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fa0b2acbffadb5 server_cookie=fa3256f0150f0c7 in_seq=0 out_seq=0 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.937+0000 7f20ccff9640 10 client.17403 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ccff9640 1 -- 172.21.15.45:0/3851192947 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6af5d2bb0 con 0x55a6af67b330 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ccff9640 10 client.17403 handle_command_reply: tid=0 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 client.17403 shutdown 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 msgr2=0x55a6af66aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f20e000b9f0 tx=0x7f20e000beb0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 shutdown_connections 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af668710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660ea0 0x55a6af6681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af67b330 0x55a6af67d7a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3851192947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660580 0x55a6af66aec0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ccff9640 0 client.17403 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 >> 172.21.15.45:0/3851192947 conn(0x55a6af65c8a0 msgr2=0x55a6af65bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 shutdown_connections 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3851192947 wait complete. 2024-04-25T00:28:00.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.938+0000 7f20ee7fa740 20 client.17403 trim_cache size 0 max 16384 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 Processor -- start 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 -- start start 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af66a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660ea0 0x55a6af6733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b5100 con 0x55a6af66b570 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b6c40 con 0x55a6af660580 2024-04-25T00:28:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b57d0 con 0x55a6af660ea0 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58152/0 (socket says 172.21.15.45:58152) 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 -- 172.21.15.45:0/995335849 learned_addr learned my addr 172.21.15.45:0/995335849 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e97ae640 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af66a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e8fad640 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660ea0 0x55a6af6733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 -- 172.21.15.45:0/995335849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660ea0 msgr2=0x55a6af6733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660ea0 0x55a6af6733a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 -- 172.21.15.45:0/995335849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 msgr2=0x55a6af66a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af66a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.952+0000 7f20e9faf640 1 -- 172.21.15.45:0/995335849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6af3a37d0 con 0x55a6af66b570 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20e9faf640 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f20dc012540 tx=0x7f20dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8ecee1332593b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ceffd640 1 -- 172.21.15.45:0/995335849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc020780 con 0x55a6af66b570 2024-04-25T00:28:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ceffd640 1 -- 172.21.15.45:0/995335849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc020e00 con 0x55a6af66b570 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ceffd640 1 -- 172.21.15.45:0/995335849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc010400 con 0x55a6af66b570 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 -- 172.21.15.45:0/995335849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 msgr2=0x55a6af6738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f20dc012540 tx=0x7f20dc016000 comp rx=0 tx=0).stop 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 -- 172.21.15.45:0/995335849 shutdown_connections 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af66b570 0x55a6af6738e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af660ea0 0x55a6af6733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/995335849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af66a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 -- 172.21.15.45:0/995335849 >> 172.21.15.45:0/995335849 conn(0x55a6af65c8a0 msgr2=0x55a6af675e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 -- 172.21.15.45:0/995335849 shutdown_connections 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.953+0000 7f20ee7fa740 1 -- 172.21.15.45:0/995335849 wait complete. 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 -- start start 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 Processor -- start 2024-04-25T00:28:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af673590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af670eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b6570 con 0x55a6af660ea0 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b5ea0 con 0x55a6af660580 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b30e0 con 0x55a6af66b570 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20ee7fa740 10 client.0 _ll_register_callbacks cb 0x7f20e0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20e8fad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58156/0 (socket says 172.21.15.45:58156) 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20e8fad640 1 -- 172.21.15.45:0/3359208593 learned_addr learned my addr 172.21.15.45:0/3359208593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af673590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.954+0000 7f20e9faf640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af670eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 -- 172.21.15.45:0/3359208593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 msgr2=0x55a6af670eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af670eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 -- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 msgr2=0x55a6af673590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af673590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 -- 172.21.15.45:0/3359208593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6af671520 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20e8fad640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f20d4004650 tx=0x7f20d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e75ebac45ddbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4049040 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f20d40f5320 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20dc020e00 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20d403bcb0 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 10 client.17409 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6af6e17e0 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc020e00 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20d404c060 con 0x55a6af660ea0 2024-04-25T00:28:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 10 client.17409 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 20 client.17409 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af6e1dd0 0x55a6af6e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 4 client.17409 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.955+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6af5ccf00 con 0x55a6af6e1dd0 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af6e1dd0 0x55a6af6e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20e97ae640 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af6e1dd0 0x55a6af6e41f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a456bc1c635769da server_cookie=bc2ca531bd7324aa in_seq=0 out_seq=0 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ccff9640 10 client.17409 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ccff9640 1 -- 172.21.15.45:0/3359208593 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6af5ccf00 con 0x55a6af6e1dd0 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ccff9640 10 client.17409 handle_command_reply: tid=0 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 client.17409 shutdown 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 msgr2=0x55a6af670970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f20d4004650 tx=0x7f20d4004270 comp rx=0 tx=0).stop 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 shutdown_connections 2024-04-25T00:28:00.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6af66b570 0x55a6af670eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a6af6e1dd0 0x55a6af6e41f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a6af660ea0 0x55a6af670970 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ccff9640 0 client.17409 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 --2- 172.21.15.45:0/3359208593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a6af660580 0x55a6af673590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.956+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 >> 172.21.15.45:0/3359208593 conn(0x55a6af65c8a0 msgr2=0x55a6af676710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 shutdown_connections 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20ee7fa740 1 -- 172.21.15.45:0/3359208593 wait complete. 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20ee7fa740 20 client.17409 trim_cache size 0 max 16384 2024-04-25T00:28:00.964 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 msgr2=0x7f20d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 0x7f20d00a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f20dc005fd0 tx=0x7f20dc004a40 comp rx=0 tx=0).stop 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 msgr2=0x7f20e419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f20d4006010 tx=0x7f20d40042f0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 shutdown_connections 2024-04-25T00:28:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d00a2d00 0x7f20d00a51c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20e4106670 0x7f20e41a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20e4104170 0x7f20e419eb10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 --2- 172.21.15.45:0/3602471132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20e406a300 0x7f20e419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 >> 172.21.15.45:0/3602471132 conn(0x7f20e40776a0 msgr2=0x7f20e4103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 shutdown_connections 2024-04-25T00:28:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:00.957+0000 7f20eba12640 1 -- 172.21.15.45:0/3602471132 wait complete. 2024-04-25T00:28:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:00 smithi045 ceph-mon[29194]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:01.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:00 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1246872486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:00 smithi122 ceph-mon[32959]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:01.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:00 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1246872486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:00 smithi071 ceph-mon[32667]: pgmap v184: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:01.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:00 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1246872486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:01.969 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:28:01.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:28:01.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:28:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- 172.21.15.45:0/1440083976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 msgr2=0x7fea64110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/1440083976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea64110540 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009ca0 tx=0x7fea5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- 172.21.15.45:0/1440083976 shutdown_connections 2024-04-25T00:28:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/1440083976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea64110540 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/1440083976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/1440083976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- 172.21.15.45:0/1440083976 >> 172.21.15.45:0/1440083976 conn(0x7fea640776a0 msgr2=0x7fea64077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- 172.21.15.45:0/1440083976 shutdown_connections 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- 172.21.15.45:0/1440083976 wait complete. 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 Processor -- start 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.090+0000 7fea6b3d5640 1 -- start start 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea64113d10 con 0x7fea6410e150 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea64113b90 con 0x7fea6406a6e0 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea6b3d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea64113e90 con 0x7fea64069dc0 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58174/0 (socket says 172.21.15.45:58174) 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 -- 172.21.15.45:0/251967008 learned_addr learned my addr 172.21.15.45:0/251967008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69171640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea68970640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 -- 172.21.15.45:0/251967008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 msgr2=0x7fea6419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 -- 172.21.15.45:0/251967008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 msgr2=0x7fea6419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 -- 172.21.15.45:0/251967008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419f180 con 0x7fea6410e150 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea68970640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69171640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.091+0000 7fea69972640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0319f0 tx=0x7fea5c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.092+0000 7fea567fc640 1 -- 172.21.15.45:0/251967008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c003b70 con 0x7fea6410e150 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.092+0000 7fea567fc640 1 -- 172.21.15.45:0/251967008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c003d10 con 0x7fea6410e150 2024-04-25T00:28:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.092+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6419f410 con 0x7fea6410e150 2024-04-25T00:28:02.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.092+0000 7fea567fc640 1 -- 172.21.15.45:0/251967008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c02fcb0 con 0x7fea6410e150 2024-04-25T00:28:02.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.092+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a6d00 con 0x7fea6410e150 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.094+0000 7fea567fc640 1 -- 172.21.15.45:0/251967008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fea5c004050 con 0x7fea6410e150 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.094+0000 7fea6e1be740 1 Processor -- start 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 -- start start 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b59f552c0 con 0x558b59ffc010 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558b59efd910 con 0x558b59ff2980 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea6e1be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b59f51280 con 0x558b59ff1290 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea567fc640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 0x7fea500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea69972640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 0x7fea500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58176/0 (socket says 172.21.15.45:58176) 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 -- 172.21.15.45:0/4292336296 learned_addr learned my addr 172.21.15.45:0/4292336296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea69171640 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea567fc640 1 -- 172.21.15.45:0/251967008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea5c0f0870 con 0x7fea6410e150 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 -- 172.21.15.45:0/4292336296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 msgr2=0x558b59ff1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 -- 172.21.15.45:0/4292336296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 msgr2=0x558b59ff0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.095+0000 7fea68970640 1 -- 172.21.15.45:0/4292336296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b59f0efb0 con 0x558b59ffc010 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.096+0000 7fea69972640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 0x7fea500a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0b94b0 tx=0x7fea5c043e60 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.096+0000 7fea68970640 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fea580127b0 tx=0x7fea580096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8889b371eecbcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.096+0000 7fea4f7fe640 1 -- 172.21.15.45:0/4292336296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58012d30 con 0x558b59ffc010 2024-04-25T00:28:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.096+0000 7fea4f7fe640 1 -- 172.21.15.45:0/4292336296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58010cb0 con 0x558b59ffc010 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.096+0000 7fea4f7fe640 1 -- 172.21.15.45:0/4292336296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5801ebc0 con 0x558b59ffc010 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- 172.21.15.45:0/4292336296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 msgr2=0x558b59ffe400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fea580127b0 tx=0x7fea580096e0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- 172.21.15.45:0/4292336296 shutdown_connections 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b59ffe400 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4292336296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- 172.21.15.45:0/4292336296 >> 172.21.15.45:0/4292336296 conn(0x558b59fb9950 msgr2=0x558b59fed7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- 172.21.15.45:0/4292336296 shutdown_connections 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- 172.21.15.45:0/4292336296 wait complete. 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 -- start start 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.097+0000 7fea6e1be740 1 Processor -- start 2024-04-25T00:28:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 0x558b59ff9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b59ffeea0 con 0x558b59ff2980 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558b59f10e90 con 0x558b59ffc010 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b59fff020 con 0x558b59ff1290 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 10 client.0 _ll_register_callbacks cb 0x558b59f4c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58182/0 (socket says 172.21.15.45:58182) 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 -- 172.21.15.45:0/4050356516 learned_addr learned my addr 172.21.15.45:0/4050356516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea68970640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea68970640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 0x558b59ff9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 -- 172.21.15.45:0/4050356516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 msgr2=0x558b59ff8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 -- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 msgr2=0x558b59ff9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 0x558b59ff9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 -- 172.21.15.45:0/4050356516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b59d317e0 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea68970640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 0x558b59ff9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea69171640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fea60013790 tx=0x7fea60013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4c879875840667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60020060 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea60011e30 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.098+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea58010cb0 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea60027050 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 10 client.17421 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b59d91f70 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58010cb0 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea6001a030 con 0x558b59ff2980 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 10 client.17421 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 20 client.17421 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a00c1c0 0x558b5a00e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 4 client.17421 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b59f295a0 con 0x558b5a00c1c0 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.099+0000 7fea68970640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a00c1c0 0x558b5a00e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.100+0000 7fea4cff9640 10 client.17421 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.100+0000 7fea68970640 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a00c1c0 0x558b5a00e5b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52f715d06bd191b server_cookie=68a1d49b8e94bb05 in_seq=0 out_seq=0 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.100+0000 7fea4cff9640 1 -- 172.21.15.45:0/4050356516 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b59f295a0 con 0x558b5a00c1c0 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.100+0000 7fea4cff9640 10 client.17421 handle_command_reply: tid=0 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 client.17421 shutdown 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 msgr2=0x558b59ff9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fea60013790 tx=0x7fea60013c50 comp rx=0 tx=0).stop 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 shutdown_connections 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ffc010 0x558b59ff9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a00c1c0 0x558b5a00e5b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff2980 0x558b59ff9450 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 --2- 172.21.15.45:0/4050356516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff1290 0x558b59ff8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea4cff9640 0 client.17421 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 >> 172.21.15.45:0/4050356516 conn(0x558b59fb9950 msgr2=0x558b59ff3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 shutdown_connections 2024-04-25T00:28:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 1 -- 172.21.15.45:0/4050356516 wait complete. 2024-04-25T00:28:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.101+0000 7fea6e1be740 20 client.17421 trim_cache size 0 max 16384 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 Processor -- start 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 -- start start 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ffc010 0x558b5a013ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b0cd0 con 0x558b59ff1290 2024-04-25T00:28:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b2810 con 0x558b59ff2980 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea6e1be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b13a0 con 0x558b59ffc010 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58188/0 (socket says 172.21.15.45:58188) 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 -- 172.21.15.45:0/1228768719 learned_addr learned my addr 172.21.15.45:0/1228768719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea69171640 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 -- 172.21.15.45:0/1228768719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ffc010 msgr2=0x558b5a013ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ffc010 0x558b5a013ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 -- 172.21.15.45:0/1228768719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 msgr2=0x558b59ff97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 -- 172.21.15.45:0/1228768719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b59c29170 con 0x558b59ff1290 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea69171640 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.119+0000 7fea68970640 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fea5801e810 tx=0x7fea5801e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941c1cb132d18347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea4effd640 1 -- 172.21.15.45:0/1228768719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58012050 con 0x558b59ff1290 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea4effd640 1 -- 172.21.15.45:0/1228768719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58005520 con 0x558b59ff1290 2024-04-25T00:28:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea4effd640 1 -- 172.21.15.45:0/1228768719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580107a0 con 0x558b59ff1290 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 -- 172.21.15.45:0/1228768719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 msgr2=0x558b5a007dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fea5801e810 tx=0x7fea5801e8e0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 -- 172.21.15.45:0/1228768719 shutdown_connections 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ffc010 0x558b5a013ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff2980 0x558b59ff97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 --2- 172.21.15.45:0/1228768719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ff1290 0x558b5a007dd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 -- 172.21.15.45:0/1228768719 >> 172.21.15.45:0/1228768719 conn(0x558b59fb9950 msgr2=0x558b59fec950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 -- 172.21.15.45:0/1228768719 shutdown_connections 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.120+0000 7fea6e1be740 1 -- 172.21.15.45:0/1228768719 wait complete. 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 -- start start 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 Processor -- start 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff1290 0x558b59ff5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff2980 0x558b59ff6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b2140 con 0x558b59ffc010 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b1a70 con 0x558b59ff1290 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea641aecb0 con 0x558b59ff2980 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea6e1be740 10 client.0 _ll_register_callbacks cb 0x7fea60009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea68970640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58200/0 (socket says 172.21.15.45:58200) 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea68970640 1 -- 172.21.15.45:0/573635892 learned_addr learned my addr 172.21.15.45:0/573635892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea68970640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff1290 0x558b59ff5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.121+0000 7fea69171640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff2980 0x558b59ff6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 -- 172.21.15.45:0/573635892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff2980 msgr2=0x558b59ff6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff2980 0x558b59ff6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 -- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff1290 msgr2=0x558b59ff5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff1290 0x558b59ff5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 -- 172.21.15.45:0/573635892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b59ff6d20 con 0x558b59ffc010 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea68970640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fea5801ed00 tx=0x7fea5801ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0c05b4a066ad6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5801eda0 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fea58024070 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea58005520 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea58005040 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 10 client.17427 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b5a072cf0 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea58005520 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea5802a040 con 0x558b59ffc010 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 10 client.17427 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 20 client.17427 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a0731f0 0x558b5a075610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 4 client.17427 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:28:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.122+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b59f62800 con 0x558b5a0731f0 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.123+0000 7fea69171640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a0731f0 0x558b5a075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.123+0000 7fea69171640 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a0731f0 0x558b5a075610 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c532ba29f9a4f9 server_cookie=7808e60a404ba1e8 in_seq=0 out_seq=0 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.123+0000 7fea4cff9640 10 client.17427 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea4cff9640 1 -- 172.21.15.45:0/573635892 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b59f62800 con 0x558b5a0731f0 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea4cff9640 10 client.17427 handle_command_reply: tid=0 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 client.17427 shutdown 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 msgr2=0x558b5a071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fea5801ed00 tx=0x7fea5801ec00 comp rx=0 tx=0).stop 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 shutdown_connections 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558b5a0731f0 0x558b5a075610 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea4cff9640 0 client.17427 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b59ffc010 0x558b5a071730 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b59ff2980 0x558b59ff6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 --2- 172.21.15.45:0/573635892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b59ff1290 0x558b59ff5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.124+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 >> 172.21.15.45:0/573635892 conn(0x558b59fb9950 msgr2=0x558b59ff2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 shutdown_connections 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6e1be740 1 -- 172.21.15.45:0/573635892 wait complete. 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6e1be740 20 client.17427 trim_cache size 0 max 16384 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fc26e1dd-28ec-48c5-91b4-4782870f6dca", 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:28:02.132 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 msgr2=0x7fea500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 0x7fea500a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0b94b0 tx=0x7fea5c043e60 comp rx=0 tx=0).stop 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 msgr2=0x7fea6419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0319f0 tx=0x7fea5c031a20 comp rx=0 tx=0).stop 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 shutdown_connections 2024-04-25T00:28:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fea500a2d00 0x7fea500a51c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea6410e150 0x7fea6419ea50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea6406a6e0 0x7fea6419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 --2- 172.21.15.45:0/251967008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea64069dc0 0x7fea6419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 >> 172.21.15.45:0/251967008 conn(0x7fea640776a0 msgr2=0x7fea6410f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 shutdown_connections 2024-04-25T00:28:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.125+0000 7fea6b3d5640 1 -- 172.21.15.45:0/251967008 wait complete. 2024-04-25T00:28:02.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:28:02.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:28:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.240+0000 7f298ad42640 1 -- 172.21.15.45:0/1261828587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 msgr2=0x7f298410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.240+0000 7f298ad42640 1 --2- 172.21.15.45:0/1261828587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298410e1a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f297c009ca0 tx=0x7f297c02a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- 172.21.15.45:0/1261828587 shutdown_connections 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- 172.21.15.45:0/1261828587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298410e1a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- 172.21.15.45:0/1261828587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f298410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- 172.21.15.45:0/1261828587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 0x7f2984102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- 172.21.15.45:0/1261828587 >> 172.21.15.45:0/1261828587 conn(0x7f2984077740 msgr2=0x7f2984077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- 172.21.15.45:0/1261828587 shutdown_connections 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- 172.21.15.45:0/1261828587 wait complete. 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 Processor -- start 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- start start 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 0x7f2984197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f2984198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111b00 con 0x7f298410bd00 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111980 con 0x7f29841029c0 2024-04-25T00:28:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.241+0000 7f298ad42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111c80 con 0x7f29841032e0 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58226/0 (socket says 172.21.15.45:58226) 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 -- 172.21.15.45:0/3271157506 learned_addr learned my addr 172.21.15.45:0/3271157506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2983fff640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f2984198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2988ade640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 0x7f2984197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 -- 172.21.15.45:0/3271157506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 msgr2=0x7f2984198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f2984198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 -- 172.21.15.45:0/3271157506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 msgr2=0x7f2984197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 0x7f2984197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 -- 172.21.15.45:0/3271157506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298419cf80 con 0x7f298410bd00 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2983fff640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f2984198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f29892df640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2981ffb640 1 -- 172.21.15.45:0/3271157506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c047070 con 0x7f298410bd00 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2981ffb640 1 -- 172.21.15.45:0/3271157506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f297c02fb20 con 0x7f298410bd00 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f2981ffb640 1 -- 172.21.15.45:0/3271157506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c033050 con 0x7f298410bd00 2024-04-25T00:28:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.242+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298419d210 con 0x7f298410bd00 2024-04-25T00:28:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.243+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a4b00 con 0x7f298410bd00 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f2981ffb640 1 -- 172.21.15.45:0/3271157506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f297c04a070 con 0x7f298410bd00 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 Processor -- start 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 -- start start 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65f50 0x558889e700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558889dca6b0 con 0x558889e65630 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558889d72880 con 0x558889e70620 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f298da50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558889dc6670 con 0x558889e65f50 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f2983fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f2981ffb640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 0x7f296c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f2983fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58242/0 (socket says 172.21.15.45:58242) 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.245+0000 7f2983fff640 1 -- 172.21.15.45:0/1304769335 learned_addr learned my addr 172.21.15.45:0/1304769335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2988ade640 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65f50 0x558889e700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f29892df640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 0x7f296c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 -- 172.21.15.45:0/1304769335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65f50 msgr2=0x558889e700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65f50 0x558889e700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 -- 172.21.15.45:0/1304769335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 msgr2=0x558889e72a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e72a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 -- 172.21.15.45:0/1304769335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558889ba87d0 con 0x558889e65630 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2981ffb640 1 -- 172.21.15.45:0/3271157506 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f297c0f04a0 con 0x7f298410bd00 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f2983fff640 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2978013550 tx=0x7f297800d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288a1bd664424f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f296affd640 1 -- 172.21.15.45:0/1304769335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978019070 con 0x558889e65630 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f29892df640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 0x7f296c0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f297c0c4d40 tx=0x7f297c02a9e0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f296affd640 1 -- 172.21.15.45:0/1304769335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2978004220 con 0x558889e65630 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.246+0000 7f296affd640 1 -- 172.21.15.45:0/1304769335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780117c0 con 0x558889e65630 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- 172.21.15.45:0/1304769335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 msgr2=0x558889e65a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2978013550 tx=0x7f297800d630 comp rx=0 tx=0).stop 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- 172.21.15.45:0/1304769335 shutdown_connections 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e72a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65f50 0x558889e700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 --2- 172.21.15.45:0/1304769335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65630 0x558889e65a10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- 172.21.15.45:0/1304769335 >> 172.21.15.45:0/1304769335 conn(0x558889e61950 msgr2=0x558889e61d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- 172.21.15.45:0/1304769335 shutdown_connections 2024-04-25T00:28:02.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- 172.21.15.45:0/1304769335 wait complete. 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 -- start start 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.247+0000 7f298da50740 1 Processor -- start 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558889e73410 con 0x558889e65f50 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558889d85f50 con 0x558889e70620 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558889e73590 con 0x558889e65630 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f298da50740 10 client.0 _ll_register_callbacks cb 0x558889dc1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58254/0 (socket says 172.21.15.45:58254) 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 -- 172.21.15.45:0/2874820251 learned_addr learned my addr 172.21.15.45:0/2874820251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2983fff640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2983fff640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 -- 172.21.15.45:0/2874820251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 msgr2=0x558889e6d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e6d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 -- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 msgr2=0x558889e6dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e6dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 -- 172.21.15.45:0/2874820251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558889ade3f0 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2983fff640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e6dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2988ade640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2974012700 tx=0x7f2974012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6e3f06801ff3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.248+0000 7f2963fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004260 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f2974004400 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2978004220 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2974025050 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 10 client.17439 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558889a9e0f0 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2978004220 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f297400f440 con 0x558889e65f50 2024-04-25T00:28:02.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 10 client.17439 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 20 client.17439 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889e80980 0x558889e82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 4 client.17439 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.249+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558889dd7bf0 con 0x558889e80980 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.250+0000 7f2983fff640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889e80980 0x558889e82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.250+0000 7f2983fff640 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889e80980 0x558889e82d90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5971d44d9a8caae5 server_cookie=e819a858b893b34b in_seq=0 out_seq=0 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.250+0000 7f2963fff640 10 client.17439 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.250+0000 7f2963fff640 1 -- 172.21.15.45:0/2874820251 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558889dd7bf0 con 0x558889e80980 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f2963fff640 10 client.17439 handle_command_reply: tid=0 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 client.17439 shutdown 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 msgr2=0x558889e6dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2974012700 tx=0x7f2974012bc0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 shutdown_connections 2024-04-25T00:28:02.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e70620 0x558889e6dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889e80980 0x558889e82d90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e6dab0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f2963fff640 0 client.17439 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 --2- 172.21.15.45:0/2874820251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e6d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 >> 172.21.15.45:0/2874820251 conn(0x558889e61950 msgr2=0x558889e67650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 shutdown_connections 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 1 -- 172.21.15.45:0/2874820251 wait complete. 2024-04-25T00:28:02.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.251+0000 7f298da50740 20 client.17439 trim_cache size 0 max 16384 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.268+0000 7f298da50740 1 Processor -- start 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.268+0000 7f298da50740 1 -- start start 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e7c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65f50 0x558889e6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29841aeb20 con 0x558889e70620 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b0660 con 0x558889e65f50 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f298da50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29841af1f0 con 0x558889e65630 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58268/0 (socket says 172.21.15.45:58268) 2024-04-25T00:28:02.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 -- 172.21.15.45:0/925966566 learned_addr learned my addr 172.21.15.45:0/925966566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2988ade640 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65f50 0x558889e6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 -- 172.21.15.45:0/925966566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 msgr2=0x558889e7c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e7c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 -- 172.21.15.45:0/925966566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65f50 msgr2=0x558889e6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65f50 0x558889e6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 -- 172.21.15.45:0/925966566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558889e53090 con 0x558889e70620 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f2983fff640 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2978013550 tx=0x7f2978003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2faf24566abad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f296a7fc640 1 -- 172.21.15.45:0/925966566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978019040 con 0x558889e70620 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f296a7fc640 1 -- 172.21.15.45:0/925966566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2978034070 con 0x558889e70620 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.269+0000 7f296a7fc640 1 -- 172.21.15.45:0/925966566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780035f0 con 0x558889e70620 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- 172.21.15.45:0/925966566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 msgr2=0x558889e67d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2978013550 tx=0x7f2978003e80 comp rx=0 tx=0).stop 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- 172.21.15.45:0/925966566 shutdown_connections 2024-04-25T00:28:02.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e70620 0x558889e67d90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65f50 0x558889e6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 --2- 172.21.15.45:0/925966566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e65630 0x558889e7c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- 172.21.15.45:0/925966566 >> 172.21.15.45:0/925966566 conn(0x558889e61950 msgr2=0x558889e6d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- 172.21.15.45:0/925966566 shutdown_connections 2024-04-25T00:28:02.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- 172.21.15.45:0/925966566 wait complete. 2024-04-25T00:28:02.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 -- start start 2024-04-25T00:28:02.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.270+0000 7f298da50740 1 Processor -- start 2024-04-25T00:28:02.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 0x558889e73ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e70620 0x558889edef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29841aff90 con 0x558889e65f50 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29841af8c0 con 0x558889e65630 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29841acb00 con 0x558889e70620 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 10 client.0 _ll_register_callbacks cb 0x7f29740097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58272/0 (socket says 172.21.15.45:58272) 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 -- 172.21.15.45:0/2659268925 learned_addr learned my addr 172.21.15.45:0/2659268925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2983fff640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 0x558889e73ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 -- 172.21.15.45:0/2659268925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e70620 msgr2=0x558889edef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e70620 0x558889edef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 -- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 msgr2=0x558889e73ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:02.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 0x558889e73ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 -- 172.21.15.45:0/2659268925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558889edf580 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2983fff640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 0x558889e73ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2988ade640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x558889e6eb40 tx=0x7f297400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6851ef98954bfed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2963fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974022040 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f29740054e0 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.271+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2978034070 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f297403c050 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 10 client.17445 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558889ee6940 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2978034070 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2974031020 con 0x558889e65f50 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 10 client.17445 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 20 client.17445 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:02.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889ee7030 0x558889ee9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 4 client.17445 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558889dd1f40 con 0x558889ee7030 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.272+0000 7f2983fff640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889ee7030 0x558889ee9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f2983fff640 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889ee7030 0x558889ee9450 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3377aab49cf26dc4 server_cookie=45712c0dfc4333e0 in_seq=0 out_seq=0 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f2963fff640 10 client.17445 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f2963fff640 1 -- 172.21.15.45:0/2659268925 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558889dd1f40 con 0x558889ee7030 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f2963fff640 10 client.17445 handle_command_reply: tid=0 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f298da50740 1 client.17445 shutdown 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 msgr2=0x558889e74020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x558889e6eb40 tx=0x7f297400f670 comp rx=0 tx=0).stop 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 shutdown_connections 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.273+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558889e70620 0x558889edef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558889ee7030 0x558889ee9450 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558889e65f50 0x558889e74020 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 --2- 172.21.15.45:0/2659268925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558889e65630 0x558889e73ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f2963fff640 0 client.17445 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 >> 172.21.15.45:0/2659268925 conn(0x558889e61950 msgr2=0x558889e6c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 shutdown_connections 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 1 -- 172.21.15.45:0/2659268925 wait complete. 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298da50740 20 client.17445 trim_cache size 0 max 16384 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stdout: "fc26e1dd-28ec-48c5-91b4-4782870f6dca": { 2024-04-25T00:28:02.281 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:28:02.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fc26e1dd-28ec-48c5-91b4-4782870f6dca", 2024-04-25T00:28:02.282 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:28:02.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:28:02.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:28:02.282 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 msgr2=0x7f296c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 0x7f296c0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f297c0c4d40 tx=0x7f297c02a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 msgr2=0x7f298419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f297c006010 tx=0x7f297c003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 shutdown_connections 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f296c0a2d00 0x7f296c0a51c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f298410bd00 0x7f298419c850 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29841032e0 0x7f2984198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 --2- 172.21.15.45:0/3271157506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29841029c0 0x7f2984197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 >> 172.21.15.45:0/3271157506 conn(0x7f2984077740 msgr2=0x7f29840fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 shutdown_connections 2024-04-25T00:28:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:02.274+0000 7f298ad42640 1 -- 172.21.15.45:0/3271157506 wait complete. 2024-04-25T00:28:02.285 INFO:tasks.cephfs.filesystem:scrub status for tag:fc26e1dd-28ec-48c5-91b4-4782870f6dca - {'path': '/', 'tag': 'fc26e1dd-28ec-48c5-91b4-4782870f6dca', 'options': 'recursive,force'} 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:02 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:02 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: pgmap v185: 129 pgs: 129 active+clean; 259 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:02 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:28:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:04 smithi122 ceph-mon[32959]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:28:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:04 smithi071 ceph-mon[32667]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:28:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:04 smithi045 ceph-mon[29194]: pgmap v186: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-25T00:28:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:06 smithi122 ceph-mon[32959]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:06 smithi071 ceph-mon[32667]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:06 smithi045 ceph-mon[29194]: pgmap v187: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:07 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:07 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:07 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:08 smithi122 ceph-mon[32959]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:08 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:08 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:08 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:28:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:08 smithi071 ceph-mon[32667]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:08 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:08 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:08 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:28:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:08 smithi045 ceph-mon[29194]: pgmap v188: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:08 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:08 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:08 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:28:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:10 smithi122 ceph-mon[32959]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:10 smithi071 ceph-mon[32667]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:10 smithi045 ceph-mon[29194]: pgmap v189: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:12 smithi122 ceph-mon[32959]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:12 smithi071 ceph-mon[32667]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:12 smithi045 ceph-mon[29194]: pgmap v190: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T00:28:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:14 smithi122 ceph-mon[32959]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:14 smithi071 ceph-mon[32667]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:14 smithi045 ceph-mon[29194]: pgmap v191: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T00:28:17.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:16 smithi122 ceph-mon[32959]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:16 smithi071 ceph-mon[32667]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:16 smithi045 ceph-mon[29194]: pgmap v192: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T00:28:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:19 smithi122 ceph-mon[32959]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:28:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:19 smithi071 ceph-mon[32667]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:28:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:19 smithi045 ceph-mon[29194]: pgmap v193: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T00:28:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:21 smithi122 ceph-mon[32959]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:28:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:21 smithi071 ceph-mon[32667]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:28:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:21 smithi045 ceph-mon[29194]: pgmap v194: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T00:28:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:23 smithi122 ceph-mon[32959]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:23 smithi071 ceph-mon[32667]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:23 smithi045 ceph-mon[29194]: pgmap v195: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:25 smithi122 ceph-mon[32959]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:25 smithi071 ceph-mon[32667]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:25 smithi045 ceph-mon[29194]: pgmap v196: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:27 smithi122 ceph-mon[32959]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:27.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:27 smithi071 ceph-mon[32667]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:27 smithi045 ceph-mon[29194]: pgmap v197: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:29 smithi122 ceph-mon[32959]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:29 smithi071 ceph-mon[32667]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:29 smithi045 ceph-mon[29194]: pgmap v198: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:31 smithi122 ceph-mon[32959]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:31 smithi071 ceph-mon[32667]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:31 smithi045 ceph-mon[29194]: pgmap v199: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:28:32.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:28:32.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:28:32.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 -- 172.21.15.45:0/3643521927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c06a780 msgr2=0x7fba7c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 --2- 172.21.15.45:0/3643521927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c06a780 0x7fba7c10e5f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fba74009cd0 tx=0x7fba7402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 -- 172.21.15.45:0/3643521927 shutdown_connections 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 --2- 172.21.15.45:0/3643521927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c06a780 0x7fba7c10e5f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 --2- 172.21.15.45:0/3643521927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c069dc0 0x7fba7c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 --2- 172.21.15.45:0/3643521927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c107730 0x7fba7c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 -- 172.21.15.45:0/3643521927 >> 172.21.15.45:0/3643521927 conn(0x7fba7c0776a0 msgr2=0x7fba7c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 -- 172.21.15.45:0/3643521927 shutdown_connections 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.405+0000 7fba82aee640 1 -- 172.21.15.45:0/3643521927 wait complete. 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 Processor -- start 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 -- start start 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 0x7fba7c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 0x7fba7c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c111b40 con 0x7fba7c107730 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1119c0 con 0x7fba7c06a780 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba82aee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c111cc0 con 0x7fba7c069dc0 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba8108b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba8108b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42726/0 (socket says 172.21.15.45:42726) 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba8108b640 1 -- 172.21.15.45:0/3190770256 learned_addr learned my addr 172.21.15.45:0/3190770256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.406+0000 7fba7bfff640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 0x7fba7c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8088a640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 0x7fba7c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 -- 172.21.15.45:0/3190770256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 msgr2=0x7fba7c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 0x7fba7c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 -- 172.21.15.45:0/3190770256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 msgr2=0x7fba7c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 0x7fba7c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 -- 172.21.15.45:0/3190770256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c19cf70 con 0x7fba7c107730 2024-04-25T00:28:32.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba7bfff640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 0x7fba7c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8088a640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 0x7fba7c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba8108b640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fba74005cc0 tx=0x7fba74032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba79ffb640 1 -- 172.21.15.45:0/3190770256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74004040 con 0x7fba7c107730 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba79ffb640 1 -- 172.21.15.45:0/3190770256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba740046c0 con 0x7fba7c107730 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba79ffb640 1 -- 172.21.15.45:0/3190770256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74030e00 con 0x7fba7c107730 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c19d200 con 0x7fba7c107730 2024-04-25T00:28:32.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.407+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1a4af0 con 0x7fba7c107730 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.409+0000 7fba79ffb640 1 -- 172.21.15.45:0/3190770256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba740305a0 con 0x7fba7c107730 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 Processor -- start 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 -- start start 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd49590910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd495a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba79ffb640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 0x7fba680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd494f56b0 con 0x55dd49590e50 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dd4949d880 con 0x55dd4959fa30 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba857fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dd494f1670 con 0x55dd49590530 2024-04-25T00:28:32.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba8088a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba8108b640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 0x7fba680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba8088a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42736/0 (socket says 172.21.15.45:42736) 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba8088a640 1 -- 172.21.15.45:0/3054971462 learned_addr learned my addr 172.21.15.45:0/3054971462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba79ffb640 1 -- 172.21.15.45:0/3190770256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fba7402e030 con 0x7fba7c107730 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba7bfff640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd49590910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.410+0000 7fba7bfff640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd495a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 -- 172.21.15.45:0/3054971462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 msgr2=0x55dd49590910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd49590910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 -- 172.21.15.45:0/3054971462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 msgr2=0x55dd495a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd495a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 -- 172.21.15.45:0/3054971462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd492d37d0 con 0x55dd49590e50 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba7bfff640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd495a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8108b640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 0x7fba680a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba740bd1a0 tx=0x7fba740dd040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba8088a640 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009c80 tx=0x7fba6c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fab0c796ed7a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba72ffd640 1 -- 172.21.15.45:0/3054971462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004030 con 0x55dd49590e50 2024-04-25T00:28:32.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba72ffd640 1 -- 172.21.15.45:0/3054971462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0041d0 con 0x55dd49590e50 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.411+0000 7fba72ffd640 1 -- 172.21.15.45:0/3054971462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c012740 con 0x55dd49590e50 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- 172.21.15.45:0/3054971462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 msgr2=0x55dd4959f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009c80 tx=0x7fba6c010740 comp rx=0 tx=0).stop 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- 172.21.15.45:0/3054971462 shutdown_connections 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd495a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4959f4f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 --2- 172.21.15.45:0/3054971462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd49590910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- 172.21.15.45:0/3054971462 >> 172.21.15.45:0/3054971462 conn(0x55dd4958c950 msgr2=0x55dd4958cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- 172.21.15.45:0/3054971462 shutdown_connections 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- 172.21.15.45:0/3054971462 wait complete. 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 -- start start 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.412+0000 7fba857fa740 1 Processor -- start 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd4959c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd4959cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd495a27f0 con 0x55dd4959fa30 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dd494b0f50 con 0x55dd49590e50 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dd495a2970 con 0x55dd49590530 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba857fa740 10 client.0 _ll_register_callbacks cb 0x55dd494ec000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42738/0 (socket says 172.21.15.45:42738) 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 -- 172.21.15.45:0/1185205799 learned_addr learned my addr 172.21.15.45:0/1185205799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba8088a640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd4959cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd4959c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 -- 172.21.15.45:0/1185205799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 msgr2=0x55dd4959c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd4959c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 -- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 msgr2=0x55dd4959cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd4959cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 -- 172.21.15.45:0/1185205799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd492093f0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba7bfff640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fba64014da0 tx=0x7fba64016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9002828e698b07f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.413+0000 7fba5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64011e50 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fba640140d0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c0041d0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba64028050 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 10 client.17457 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd491c90f0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0041d0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba64003bf0 con 0x55dd4959fa30 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 10 client.17457 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 20 client.17457 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:32.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd495afca0 0x55dd495b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 4 client.17457 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.414+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd49502bf0 con 0x55dd495afca0 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.415+0000 7fba8088a640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd495afca0 0x55dd495b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.415+0000 7fba8088a640 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd495afca0 0x55dd495b20d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d345244f739496ae server_cookie=9a71a02a49d075c0 in_seq=0 out_seq=0 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.415+0000 7fba5bfff640 10 client.17457 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba5bfff640 1 -- 172.21.15.45:0/1185205799 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd49502bf0 con 0x55dd495afca0 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba5bfff640 10 client.17457 handle_command_reply: tid=0 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 client.17457 shutdown 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 msgr2=0x55dd4959d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fba64014da0 tx=0x7fba64016040 comp rx=0 tx=0).stop 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 shutdown_connections 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd495afca0 0x55dd495b20d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd4959d420 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd4959cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba5bfff640 0 client.17457 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 --2- 172.21.15.45:0/1185205799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd4959c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 >> 172.21.15.45:0/1185205799 conn(0x55dd4958c950 msgr2=0x55dd49592780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 shutdown_connections 2024-04-25T00:28:32.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 1 -- 172.21.15.45:0/1185205799 wait complete. 2024-04-25T00:28:32.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.416+0000 7fba857fa740 20 client.17457 trim_cache size 0 max 16384 2024-04-25T00:28:32.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 Processor -- start 2024-04-25T00:28:32.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 -- start start 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd4959bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1ac8c0 con 0x55dd49590e50 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1ae400 con 0x55dd4959fa30 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba857fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1acf90 con 0x55dd49590530 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42752/0 (socket says 172.21.15.45:42752) 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 -- 172.21.15.45:0/90929358 learned_addr learned my addr 172.21.15.45:0/90929358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba7bfff640 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba7bfff640 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd4959bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 -- 172.21.15.45:0/90929358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 msgr2=0x55dd495ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 -- 172.21.15.45:0/90929358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 msgr2=0x55dd4959bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd4959bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 -- 172.21.15.45:0/90929358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4958c780 con 0x55dd49590e50 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.434+0000 7fba8088a640 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011010 tx=0x7fba6c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7531c74cd961c3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba727fc640 1 -- 172.21.15.45:0/90929358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0050f0 con 0x55dd49590e50 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba727fc640 1 -- 172.21.15.45:0/90929358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0035f0 con 0x55dd49590e50 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba727fc640 1 -- 172.21.15.45:0/90929358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c02a2f0 con 0x55dd49590e50 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 -- 172.21.15.45:0/90929358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 msgr2=0x55dd4958cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011010 tx=0x7fba6c009920 comp rx=0 tx=0).stop 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 -- 172.21.15.45:0/90929358 shutdown_connections 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd4959fa30 0x55dd4959bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd49590e50 0x55dd4958cd30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 --2- 172.21.15.45:0/90929358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 -- 172.21.15.45:0/90929358 >> 172.21.15.45:0/90929358 conn(0x55dd4958c950 msgr2=0x55dd4959e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 -- 172.21.15.45:0/90929358 shutdown_connections 2024-04-25T00:28:32.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.435+0000 7fba857fa740 1 -- 172.21.15.45:0/90929358 wait complete. 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 -- start start 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 Processor -- start 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd495a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1add30 con 0x55dd4959fa30 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1ad660 con 0x55dd49590e50 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1aa8a0 con 0x55dd49590530 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba857fa740 10 client.0 _ll_register_callbacks cb 0x7fba6400c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42758/0 (socket says 172.21.15.45:42758) 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 -- 172.21.15.45:0/676340508 learned_addr learned my addr 172.21.15.45:0/676340508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba8088a640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd495a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 -- 172.21.15.45:0/676340508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 msgr2=0x55dd495a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 -- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 msgr2=0x55dd495a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd495a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba7bfff640 1 -- 172.21.15.45:0/676340508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd49615260 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.436+0000 7fba8088a640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd495a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba7bfff640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fba64014ab0 tx=0x7fba64016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3720e4cbc4c4e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64014280 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fba6401a040 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c0035f0 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba6403e050 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 10 client.17463 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd49616570 con 0x55dd4959fa30 2024-04-25T00:28:32.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0035f0 con 0x55dd4959fa30 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba6400fd00 con 0x55dd4959fa30 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 10 client.17463 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 20 client.17463 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd49616cb0 0x55dd496190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 4 client.17463 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.437+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd494fcf40 con 0x55dd49616cb0 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba8088a640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd49616cb0 0x55dd496190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba8088a640 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd49616cb0 0x55dd496190d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59f302015a7a473 server_cookie=9a4a935ac2f8931d in_seq=0 out_seq=0 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba5bfff640 10 client.17463 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba5bfff640 1 -- 172.21.15.45:0/676340508 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd494fcf40 con 0x55dd49616cb0 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba5bfff640 10 client.17463 handle_command_reply: tid=0 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 client.17463 shutdown 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 msgr2=0x55dd495a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fba64014ab0 tx=0x7fba64016040 comp rx=0 tx=0).stop 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 shutdown_connections 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dd49616cb0 0x55dd496190d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd4959fa30 0x55dd495a40f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dd49590e50 0x55dd495a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba5bfff640 0 client.17463 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.438+0000 7fba857fa740 1 --2- 172.21.15.45:0/676340508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd49590530 0x55dd495a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 >> 172.21.15.45:0/676340508 conn(0x55dd4958c950 msgr2=0x55dd4959d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 shutdown_connections 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba857fa740 1 -- 172.21.15.45:0/676340508 wait complete. 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba857fa740 20 client.17463 trim_cache size 0 max 16384 2024-04-25T00:28:32.446 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:28:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:28:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:28:32.447 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:28:32.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 msgr2=0x7fba680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 0x7fba680a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba740bd1a0 tx=0x7fba740dd040 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 msgr2=0x7fba7c19c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fba74005cc0 tx=0x7fba74032f40 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 shutdown_connections 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fba680a2d00 0x7fba680a51c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba7c107730 0x7fba7c19c840 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba7c06a780 0x7fba7c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 --2- 172.21.15.45:0/3190770256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba7c069dc0 0x7fba7c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 >> 172.21.15.45:0/3190770256 conn(0x7fba7c0776a0 msgr2=0x7fba7c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 shutdown_connections 2024-04-25T00:28:32.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.439+0000 7fba82aee640 1 -- 172.21.15.45:0/3190770256 wait complete. 2024-04-25T00:28:32.451 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:28:32.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:28:32.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- 172.21.15.45:0/1826098674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241010f0 msgr2=0x7fd12410d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 --2- 172.21.15.45:0/1826098674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241010f0 0x7fd12410d2d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009ca0 tx=0x7fd11c02a950 comp rx=0 tx=0).stop 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- 172.21.15.45:0/1826098674 shutdown_connections 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 --2- 172.21.15.45:0/1826098674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241010f0 0x7fd12410d2d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 --2- 172.21.15.45:0/1826098674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 0x7fd124100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 --2- 172.21.15.45:0/1826098674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241065a0 0x7fd124106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- 172.21.15.45:0/1826098674 >> 172.21.15.45:0/1826098674 conn(0x7fd124075f80 msgr2=0x7fd124076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- 172.21.15.45:0/1826098674 shutdown_connections 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- 172.21.15.45:0/1826098674 wait complete. 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 Processor -- start 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.553+0000 7fd129ba0640 1 -- start start 2024-04-25T00:28:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 0x7fd124199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 0x7fd12419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241107e0 con 0x7fd1241065a0 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd124110660 con 0x7fd1241010f0 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd124110960 con 0x7fd124100730 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42778/0 (socket says 172.21.15.45:42778) 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 -- 172.21.15.45:0/2897424536 learned_addr learned my addr 172.21.15.45:0/2897424536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd123fff640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 0x7fd12419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd128b9e640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 0x7fd124199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 -- 172.21.15.45:0/2897424536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 msgr2=0x7fd124199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 0x7fd124199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 -- 172.21.15.45:0/2897424536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 msgr2=0x7fd12419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 0x7fd12419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 -- 172.21.15.45:0/2897424536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12419eee0 con 0x7fd1241065a0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd123fff640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 0x7fd12419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd12939f640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd11c02ae70 tx=0x7fd11c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c034040 con 0x7fd1241065a0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c030920 con 0x7fd1241065a0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12419f170 con 0x7fd1241065a0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.554+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c045480 con 0x7fd1241065a0 2024-04-25T00:28:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.555+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1241a6a60 con 0x7fd1241065a0 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.556+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd124068480 con 0x7fd1241065a0 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.557+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd11c037020 con 0x7fd1241065a0 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.558+0000 7fd121ffb640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 0x7fd1140a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.558+0000 7fd128b9e640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 0x7fd1140a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.558+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd11c0f0720 con 0x7fd1241065a0 2024-04-25T00:28:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.558+0000 7fd128b9e640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 0x7fd1140a5240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd118005fd0 tx=0x7fd118005850 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.563+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd11c0bd090 con 0x7fd1241065a0 2024-04-25T00:28:32.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.785+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1241a6c30 con 0x7fd1241065a0 2024-04-25T00:28:32.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.786+0000 7fd121ffb640 1 -- 172.21.15.45:0/2897424536 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fd11c0bc990 con 0x7fd1241065a0 2024-04-25T00:28:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:28:32.787 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:28:32.787 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 msgr2=0x7fd1140a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 0x7fd1140a5240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd118005fd0 tx=0x7fd118005850 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 msgr2=0x7fd12419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd11c02ae70 tx=0x7fd11c02f5b0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 shutdown_connections 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd1140a2d80 0x7fd1140a5240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1241065a0 0x7fd12419e7b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1241010f0 0x7fd12419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 --2- 172.21.15.45:0/2897424536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd124100730 0x7fd124199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 >> 172.21.15.45:0/2897424536 conn(0x7fd124075f80 msgr2=0x7fd124068900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 shutdown_connections 2024-04-25T00:28:32.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.788+0000 7fd129ba0640 1 -- 172.21.15.45:0/2897424536 wait complete. 2024-04-25T00:28:32.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:28:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 -- 172.21.15.45:0/3131404859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0107730 msgr2=0x7f51c0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/3131404859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0107730 0x7f51c0107b10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009930 tx=0x7f51b002a420 comp rx=0 tx=0).stop 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 -- 172.21.15.45:0/3131404859 shutdown_connections 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/3131404859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c006a780 0x7f51c010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/3131404859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c0069dc0 0x7f51c006a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/3131404859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0107730 0x7f51c0107b10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 -- 172.21.15.45:0/3131404859 >> 172.21.15.45:0/3131404859 conn(0x7f51c00776a0 msgr2=0x7f51c0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 -- 172.21.15.45:0/3131404859 shutdown_connections 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.903+0000 7f51c4fc8640 1 -- 172.21.15.45:0/3131404859 wait complete. 2024-04-25T00:28:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 Processor -- start 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 -- start start 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 0x7f51c0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c0107730 0x7f51c019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0111b90 con 0x7f51c0069dc0 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0111a10 con 0x7f51c006a780 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51c4fc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0111d10 con 0x7f51c0107730 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42798/0 (socket says 172.21.15.45:42798) 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.904+0000 7f51be59c640 1 -- 172.21.15.45:0/1877234992 learned_addr learned my addr 172.21.15.45:0/1877234992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51bed9d640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c0107730 0x7f51c019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 0x7f51c0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 -- 172.21.15.45:0/1877234992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c0107730 msgr2=0x7f51c019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c0107730 0x7f51c019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 -- 172.21.15.45:0/1877234992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 msgr2=0x7f51c0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 0x7f51c0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 -- 172.21.15.45:0/1877234992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c019cfb0 con 0x7f51c0069dc0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 0x7f51c0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51be59c640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f51b002a940 tx=0x7f51b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51b77fe640 1 -- 172.21.15.45:0/1877234992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b003f070 con 0x7f51c0069dc0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51b77fe640 1 -- 172.21.15.45:0/1877234992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b0031c90 con 0x7f51c0069dc0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c019d240 con 0x7f51c0069dc0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51b77fe640 1 -- 172.21.15.45:0/1877234992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0035030 con 0x7f51c0069dc0 2024-04-25T00:28:32.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.905+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01a4b30 con 0x7f51c0069dc0 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.907+0000 7f51b77fe640 1 -- 172.21.15.45:0/1877234992 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51b0042070 con 0x7f51c0069dc0 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 Processor -- start 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 -- start start 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433597030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e433599960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51b77fe640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 0x7f51ac0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e4334f1670 con 0x55e43358c580 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e433499970 con 0x55e433597570 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51c7db4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e4334ed630 con 0x55e43358cea0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 0x7f51ac0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42804/0 (socket says 172.21.15.45:42804) 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51bed9d640 1 -- 172.21.15.45:0/3432630142 learned_addr learned my addr 172.21.15.45:0/3432630142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51b77fe640 1 -- 172.21.15.45:0/1877234992 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f51b00eff50 con 0x7f51c0069dc0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51be59c640 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433597030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.908+0000 7f51bed9d640 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e433599960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 -- 172.21.15.45:0/3432630142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 msgr2=0x55e433597030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433597030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 -- 172.21.15.45:0/3432630142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 msgr2=0x55e433599960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e433599960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 -- 172.21.15.45:0/3432630142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4334bd400 con 0x55e43358c580 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 0x7f51ac0a5240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51a8005d00 tx=0x7f51a8009550 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51bed9d640 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f51b80144e0 tx=0x7f51b8009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787660f265d5e168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51b4ff9640 1 -- 172.21.15.45:0/3432630142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8004d20 con 0x55e43358c580 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51b4ff9640 1 -- 172.21.15.45:0/3432630142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b8018070 con 0x55e43358c580 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.909+0000 7f51b4ff9640 1 -- 172.21.15.45:0/3432630142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b80128d0 con 0x55e43358c580 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- 172.21.15.45:0/3432630142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 msgr2=0x55e43358c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f51b80144e0 tx=0x7f51b8009870 comp rx=0 tx=0).stop 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- 172.21.15.45:0/3432630142 shutdown_connections 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e433599960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433597030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- 172.21.15.45:0/3432630142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43358c960 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- 172.21.15.45:0/3432630142 >> 172.21.15.45:0/3432630142 conn(0x55e4335888a0 msgr2=0x55e433588c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- 172.21.15.45:0/3432630142 shutdown_connections 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- 172.21.15.45:0/3432630142 wait complete. 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- start start 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 Processor -- start 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358c580 0x55e433596e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433594110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e43359a360 con 0x55e433597570 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e4334ad050 con 0x55e43358c580 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.910+0000 7f51c7db4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43359a4e0 con 0x55e43358cea0 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51c7db4740 10 client.0 _ll_register_callbacks cb 0x55e433204be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42816/0 (socket says 172.21.15.45:42816) 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 -- 172.21.15.45:0/333883654 learned_addr learned my addr 172.21.15.45:0/333883654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51be59c640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358c580 0x55e433596e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51b7fff640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433594110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 -- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 msgr2=0x55e433594110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433594110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 -- 172.21.15.45:0/333883654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358c580 msgr2=0x55e433596e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358c580 0x55e433596e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 -- 172.21.15.45:0/333883654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43347edc0 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51bed9d640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006010 tx=0x7f51b8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72aba098f901071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b802a060 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f51b8005230 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b8018070 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51b8031050 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51c7db4740 10 client.17481 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.911+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4332053f0 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b8018070 con 0x55e433597570 2024-04-25T00:28:32.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f51b800f040 con 0x55e433597570 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 10 client.17481 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 20 client.17481 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e4335a7500 0x55e4335a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 4 client.17481 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4334febb0 con 0x55e4335a7500 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.912+0000 7f51be59c640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e4335a7500 0x55e4335a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51be59c640 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e4335a7500 0x55e4335a98f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9edcbd6c60bfb55c server_cookie=69ce8923c5e28389 in_seq=0 out_seq=0 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51a5ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51a5ffb640 1 -- 172.21.15.45:0/333883654 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4334febb0 con 0x55e4335a7500 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51a5ffb640 10 client.17481 handle_command_reply: tid=0 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 client.17481 shutdown 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 msgr2=0x55e433594650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006010 tx=0x7f51b8004cf0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 shutdown_connections 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e4335a7500 0x55e4335a98f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e433594650 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e433594110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51a5ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 --2- 172.21.15.45:0/333883654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358c580 0x55e433596e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 >> 172.21.15.45:0/333883654 conn(0x55e4335888a0 msgr2=0x55e43358e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 shutdown_connections 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.913+0000 7f51c7db4740 1 -- 172.21.15.45:0/333883654 wait complete. 2024-04-25T00:28:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.914+0000 7f51c7db4740 20 client.17481 trim_cache size 0 max 16384 2024-04-25T00:28:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 Processor -- start 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 -- start start 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358c580 0x55e4335a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 0x55e433596950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01aeb80 con 0x55e433597570 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01b06c0 con 0x55e43358cea0 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51c7db4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01af250 con 0x55e43358c580 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42818/0 (socket says 172.21.15.45:42818) 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 -- 172.21.15.45:0/914598960 learned_addr learned my addr 172.21.15.45:0/914598960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51b7fff640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 0x55e433596950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51be59c640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358c580 0x55e4335a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 -- 172.21.15.45:0/914598960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358c580 msgr2=0x55e4335a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358c580 0x55e4335a3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 -- 172.21.15.45:0/914598960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 msgr2=0x55e433596950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 0x55e433596950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.927+0000 7f51bed9d640 1 -- 172.21.15.45:0/914598960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4332cf7d0 con 0x55e433597570 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51b7fff640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 0x55e433596950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51bed9d640 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f51b8012520 tx=0x7f51b8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf95521f6a36133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51a7fff640 1 -- 172.21.15.45:0/914598960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b803a070 con 0x55e433597570 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51a7fff640 1 -- 172.21.15.45:0/914598960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b8018040 con 0x55e433597570 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51a7fff640 1 -- 172.21.15.45:0/914598960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b802a070 con 0x55e433597570 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 -- 172.21.15.45:0/914598960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 msgr2=0x55e4335aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f51b8012520 tx=0x7f51b8005f10 comp rx=0 tx=0).stop 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 -- 172.21.15.45:0/914598960 shutdown_connections 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e433597570 0x55e4335aef50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43358cea0 0x55e433596950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 --2- 172.21.15.45:0/914598960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358c580 0x55e4335a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 -- 172.21.15.45:0/914598960 >> 172.21.15.45:0/914598960 conn(0x55e4335888a0 msgr2=0x55e43358f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 -- 172.21.15.45:0/914598960 shutdown_connections 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.928+0000 7f51c7db4740 1 -- 172.21.15.45:0/914598960 wait complete. 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 -- start start 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 Processor -- start 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e43359b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e43359b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01afff0 con 0x55e43358c580 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01af920 con 0x55e433597570 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c01acb60 con 0x55e43358cea0 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51c7db4740 10 client.0 _ll_register_callbacks cb 0x7f51b8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42832/0 (socket says 172.21.15.45:42832) 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 -- 172.21.15.45:0/1204319692 learned_addr learned my addr 172.21.15.45:0/1204319692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51bed9d640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e43359b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e43359b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 -- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 msgr2=0x55e43359b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e43359b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 -- 172.21.15.45:0/1204319692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 msgr2=0x55e43359b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e43359b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51be59c640 1 -- 172.21.15.45:0/1204319692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43360c7c0 con 0x55e43358c580 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.929+0000 7f51bed9d640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e43359b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51be59c640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f51b00b8570 tx=0x7f51b00dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692e2ace8741bf53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0049040 con 0x55e43358c580 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f51b00c0160 con 0x55e43358c580 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b8018040 con 0x55e43358c580 2024-04-25T00:28:32.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51b00bc810 con 0x55e43358c580 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 10 client.17487 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e43360d9e0 con 0x55e43358c580 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b8018040 con 0x55e43358c580 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f51b00ed5c0 con 0x55e43358c580 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 10 client.17487 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 10 client.17487 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 20 client.17487 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e43360e0b0 0x55e4336104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 4 client.17487 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.930+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4334f8f00 con 0x55e43360e0b0 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e43360e0b0 0x55e4336104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51a5ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51b7fff640 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e43360e0b0 0x55e4336104d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e181ed57ac061b3e server_cookie=6f90a0e6422630d0 in_seq=0 out_seq=0 2024-04-25T00:28:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51a5ffb640 1 -- 172.21.15.45:0/1204319692 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4334f8f00 con 0x55e43360e0b0 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51a5ffb640 10 client.17487 handle_command_reply: tid=0 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 client.17487 shutdown 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 msgr2=0x55e43359fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f51b00b8570 tx=0x7f51b00dc040 comp rx=0 tx=0).stop 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 shutdown_connections 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55e43360e0b0 0x55e4336104d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e433597570 0x55e43359b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51a5ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43358cea0 0x55e43359b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 --2- 172.21.15.45:0/1204319692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e43358c580 0x55e43359fcc0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 >> 172.21.15.45:0/1204319692 conn(0x55e4335888a0 msgr2=0x55e4335a2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 shutdown_connections 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 1 -- 172.21.15.45:0/1204319692 wait complete. 2024-04-25T00:28:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.931+0000 7f51c7db4740 20 client.17487 trim_cache size 0 max 16384 2024-04-25T00:28:32.940 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 msgr2=0x7f51ac0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 0x7f51ac0a5240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51a8005d00 tx=0x7f51a8009550 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 msgr2=0x7f51c0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f51b002a940 tx=0x7f51b0003b90 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 shutdown_connections 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f51ac0a2d80 0x7f51ac0a5240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c0107730 0x7f51c019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51c006a780 0x7f51c0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 --2- 172.21.15.45:0/1877234992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51c0069dc0 0x7f51c0198010 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 >> 172.21.15.45:0/1877234992 conn(0x7f51c00776a0 msgr2=0x7f51c00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 shutdown_connections 2024-04-25T00:28:32.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:32.932+0000 7f51c4fc8640 1 -- 172.21.15.45:0/1877234992 wait complete. 2024-04-25T00:28:32.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 -- 172.21.15.45:0/2439485572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 msgr2=0x7f7f6010e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2439485572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6010e1b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009ca0 tx=0x7f7f5802a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 -- 172.21.15.45:0/2439485572 shutdown_connections 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2439485572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6010e1b0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2439485572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 0x7f7f6010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2439485572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 0x7f7f60102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 -- 172.21.15.45:0/2439485572 >> 172.21.15.45:0/2439485572 conn(0x7f7f600769c0 msgr2=0x7f7f60076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 -- 172.21.15.45:0/2439485572 shutdown_connections 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.047+0000 7f7f687d5640 1 -- 172.21.15.45:0/2439485572 wait complete. 2024-04-25T00:28:33.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 Processor -- start 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 -- start start 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 0x7f7f60198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 0x7f7f60198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60111b10 con 0x7f7f6010bd10 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60111990 con 0x7f7f601029d0 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f687d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f60111c90 con 0x7f7f601032f0 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42842/0 (socket says 172.21.15.45:42842) 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 -- 172.21.15.45:0/2335523749 learned_addr learned my addr 172.21.15.45:0/2335523749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66571640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 0x7f7f60198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f65d70640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 0x7f7f60198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 -- 172.21.15.45:0/2335523749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 msgr2=0x7f7f60198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 0x7f7f60198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 -- 172.21.15.45:0/2335523749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 msgr2=0x7f7f60198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 0x7f7f60198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.048+0000 7f7f66d72640 1 -- 172.21.15.45:0/2335523749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6019d000 con 0x7f7f6010bd10 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f66d72640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7f58006010 tx=0x7f7f58003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f577fe640 1 -- 172.21.15.45:0/2335523749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58047070 con 0x7f7f6010bd10 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6019d290 con 0x7f7f6010bd10 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f577fe640 1 -- 172.21.15.45:0/2335523749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5802fb20 con 0x7f7f6010bd10 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f577fe640 1 -- 172.21.15.45:0/2335523749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58033050 con 0x7f7f6010bd10 2024-04-25T00:28:33.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.049+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f601a4b80 con 0x7f7f6010bd10 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.051+0000 7f7f577fe640 1 -- 172.21.15.45:0/2335523749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f5804a070 con 0x7f7f6010bd10 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.051+0000 7f7f6b5be740 1 Processor -- start 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 -- start start 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d655a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d655c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d64b4670 con 0x5577d654f580 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577d645c970 con 0x5577d654fea0 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5577d64b0630 con 0x5577d655a570 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f577fe640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 0x7f7f4c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42848/0 (socket says 172.21.15.45:42848) 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 -- 172.21.15.45:0/4064816641 learned_addr learned my addr 172.21.15.45:0/4064816641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f66d72640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 0x7f7f4c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d655c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f66571640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d655a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f577fe640 1 -- 172.21.15.45:0/2335523749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7f580f0390 con 0x7f7f6010bd10 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 -- 172.21.15.45:0/4064816641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 msgr2=0x5577d655c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d655c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 -- 172.21.15.45:0/4064816641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 msgr2=0x5577d655a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d655a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f65d70640 1 -- 172.21.15.45:0/4064816641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d6480400 con 0x5577d654f580 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.052+0000 7f7f66571640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d655a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f65d70640 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000cfc0 tx=0x7f7f5000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3f5d73eb252946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f66d72640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 0x7f7f4c0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f580ed830 tx=0x7f7f5803a040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f54ff9640 1 -- 172.21.15.45:0/4064816641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50018030 con 0x5577d654f580 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f54ff9640 1 -- 172.21.15.45:0/4064816641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f50012b40 con 0x5577d654f580 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f54ff9640 1 -- 172.21.15.45:0/4064816641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000e990 con 0x5577d654f580 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 -- 172.21.15.45:0/4064816641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 msgr2=0x5577d654f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000cfc0 tx=0x7f7f5000c520 comp rx=0 tx=0).stop 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 -- 172.21.15.45:0/4064816641 shutdown_connections 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d655c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d655a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.053+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/4064816641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d654f960 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- 172.21.15.45:0/4064816641 >> 172.21.15.45:0/4064816641 conn(0x5577d654b8a0 msgr2=0x5577d654bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- 172.21.15.45:0/4064816641 shutdown_connections 2024-04-25T00:28:33.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- 172.21.15.45:0/4064816641 wait complete. 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- start start 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 Processor -- start 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d65571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d6557730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5577d655d360 con 0x5577d654f580 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5577d6470050 con 0x5577d654fea0 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5577d655d4e0 con 0x5577d655a570 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f6b5be740 10 client.0 _ll_register_callbacks cb 0x5577d61c7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42854/0 (socket says 172.21.15.45:42854) 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f65d70640 1 -- 172.21.15.45:0/853196235 learned_addr learned my addr 172.21.15.45:0/853196235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.054+0000 7f7f66571640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d65571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d6557730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 -- 172.21.15.45:0/853196235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 msgr2=0x5577d6557730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d6557730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 -- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 msgr2=0x5577d65571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d65571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 -- 172.21.15.45:0/853196235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d6441dc0 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f66571640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d65571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f65d70640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f7f50015020 tx=0x7f7f500058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19838cc6b110e259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50018040 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7f50005030 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f50012b40 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f50005230 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f6b5be740 10 client.17499 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577d61c83f0 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.055+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f50012b40 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f5000e890 con 0x5577d654f580 2024-04-25T00:28:33.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 10 client.17499 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 10 client.17499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 20 client.17499 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d656a530 0x5577d656c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 4 client.17499 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577d64c1bb0 con 0x5577d656a530 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f66571640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d656a530 0x5577d656c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f66571640 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d656a530 0x5577d656c920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d1e0cc57338efdeb server_cookie=a8facbee2edc7751 in_seq=0 out_seq=0 2024-04-25T00:28:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.056+0000 7f7f49ffb640 10 client.17499 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f49ffb640 1 -- 172.21.15.45:0/853196235 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577d64c1bb0 con 0x5577d656a530 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f49ffb640 10 client.17499 handle_command_reply: tid=0 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 client.17499 shutdown 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 msgr2=0x5577d6559ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f7f50015020 tx=0x7f7f500058a0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 shutdown_connections 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d656a530 0x5577d656c920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d655a570 0x5577d6557730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d654fea0 0x5577d65571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f49ffb640 0 client.17499 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/853196235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6559ee0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 >> 172.21.15.45:0/853196235 conn(0x5577d654b8a0 msgr2=0x5577d654ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 shutdown_connections 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.057+0000 7f7f6b5be740 1 -- 172.21.15.45:0/853196235 wait complete. 2024-04-25T00:28:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.058+0000 7f7f6b5be740 20 client.17499 trim_cache size 0 max 16384 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 Processor -- start 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 -- start start 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d65469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d6559a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601aead0 con 0x5577d654f580 2024-04-25T00:28:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601b0610 con 0x5577d655a570 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601af1a0 con 0x5577d654fea0 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42870/0 (socket says 172.21.15.45:42870) 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f65d70640 1 -- 172.21.15.45:0/1618186985 learned_addr learned my addr 172.21.15.45:0/1618186985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.071+0000 7f7f66571640 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d65469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 -- 172.21.15.45:0/1618186985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 msgr2=0x5577d65469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d65469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 -- 172.21.15.45:0/1618186985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 msgr2=0x5577d6559a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d6559a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 -- 172.21.15.45:0/1618186985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d62927d0 con 0x5577d654f580 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f65d70640 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005ab0 tx=0x7f7f50011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5403b5742f2bb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f4bfff640 1 -- 172.21.15.45:0/1618186985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50018070 con 0x5577d654f580 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f4bfff640 1 -- 172.21.15.45:0/1618186985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f50010c90 con 0x5577d654f580 2024-04-25T00:28:33.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f4bfff640 1 -- 172.21.15.45:0/1618186985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5001d600 con 0x5577d654f580 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f6b5be740 1 -- 172.21.15.45:0/1618186985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 msgr2=0x5577d6566250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.072+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005ab0 tx=0x7f7f50011fb0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- 172.21.15.45:0/1618186985 shutdown_connections 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d6559a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d65469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/1618186985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6566250 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- 172.21.15.45:0/1618186985 >> 172.21.15.45:0/1618186985 conn(0x5577d654b8a0 msgr2=0x5577d6552740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- 172.21.15.45:0/1618186985 shutdown_connections 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- 172.21.15.45:0/1618186985 wait complete. 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- start start 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 Processor -- start 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d655dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d655e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601aff40 con 0x5577d654f580 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601af870 con 0x5577d655a570 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f601acab0 con 0x5577d654fea0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f6b5be740 10 client.0 _ll_register_callbacks cb 0x7f7f5000ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f65d70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42882/0 (socket says 172.21.15.45:42882) 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.073+0000 7f7f65d70640 1 -- 172.21.15.45:0/3996629969 learned_addr learned my addr 172.21.15.45:0/3996629969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 -- 172.21.15.45:0/3996629969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 msgr2=0x5577d655dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f66571640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d655dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d655dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 -- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 msgr2=0x5577d655e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d655e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 -- 172.21.15.45:0/3996629969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d655e780 con 0x5577d654f580 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f65d70640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000ece0 tx=0x7f7f5000cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36acaa4ea9106338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50042060 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f7f50012a80 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f50010c90 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f50018070 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 10 client.17505 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577d65d0630 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f50010c90 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f5001dba0 con 0x5577d654f580 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 10 client.17505 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 10 client.17505 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.074+0000 7f7f6b5be740 20 client.17505 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d65d0ae0 0x5577d65d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 4 client.17505 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577d64bbf00 con 0x5577d65d0ae0 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f66571640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d65d0ae0 0x5577d65d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f66571640 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d65d0ae0 0x5577d65d2f00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3aa7c364cdae2cd server_cookie=474a0bd4ff76ba4b in_seq=0 out_seq=0 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f49ffb640 10 client.17505 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f49ffb640 1 -- 172.21.15.45:0/3996629969 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577d64bbf00 con 0x5577d65d0ae0 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f49ffb640 10 client.17505 handle_command_reply: tid=0 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 client.17505 shutdown 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 msgr2=0x5577d6562850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000ece0 tx=0x7f7f5000cd10 comp rx=0 tx=0).stop 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 shutdown_connections 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5577d65d0ae0 0x5577d65d2f00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.075+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5577d655a570 0x5577d655e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f49ffb640 0 client.17505 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5577d654fea0 0x5577d655dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 1 --2- 172.21.15.45:0/3996629969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5577d654f580 0x5577d6562850 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 >> 172.21.15.45:0/3996629969 conn(0x5577d654b8a0 msgr2=0x5577d6552740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 shutdown_connections 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 1 -- 172.21.15.45:0/3996629969 wait complete. 2024-04-25T00:28:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f6b5be740 20 client.17505 trim_cache size 0 max 16384 2024-04-25T00:28:33.083 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 msgr2=0x7f7f4c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 0x7f7f4c0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f580ed830 tx=0x7f7f5803a040 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 msgr2=0x7f7f6019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7f58006010 tx=0x7f7f58003ba0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 shutdown_connections 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7f4c0a2d00 0x7f7f4c0a51c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f6010bd10 0x7f7f6019c8d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f601032f0 0x7f7f60198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 --2- 172.21.15.45:0/2335523749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f601029d0 0x7f7f60198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 >> 172.21.15.45:0/2335523749 conn(0x7f7f600769c0 msgr2=0x7f7f600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 shutdown_connections 2024-04-25T00:28:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.076+0000 7f7f687d5640 1 -- 172.21.15.45:0/2335523749 wait complete. 2024-04-25T00:28:33.087 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 -- 172.21.15.45:0/4051382935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 msgr2=0x7ff028101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 --2- 172.21.15.45:0/4051382935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028101b90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009930 tx=0x7ff01c02a420 comp rx=0 tx=0).stop 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 -- 172.21.15.45:0/4051382935 shutdown_connections 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 --2- 172.21.15.45:0/4051382935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 0x7ff02810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 --2- 172.21.15.45:0/4051382935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028101b90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 --2- 172.21.15.45:0/4051382935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff028107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 -- 172.21.15.45:0/4051382935 >> 172.21.15.45:0/4051382935 conn(0x7ff0280fd180 msgr2=0x7ff0280ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 -- 172.21.15.45:0/4051382935 shutdown_connections 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 -- 172.21.15.45:0/4051382935 wait complete. 2024-04-25T00:28:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.191+0000 7ff02d258640 1 Processor -- start 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 -- start start 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 0x7ff0281985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff02819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff028111b30 con 0x7ff028101710 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281119b0 con 0x7ff028107720 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02d258640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff028111cb0 con 0x7ff0281020d0 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42904/0 (socket says 172.21.15.45:42904) 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 -- 172.21.15.45:0/1292902257 learned_addr learned my addr 172.21.15.45:0/1292902257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02759e640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff02819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02659c640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 0x7ff0281985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 -- 172.21.15.45:0/1292902257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 msgr2=0x7ff0281985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 0x7ff0281985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 -- 172.21.15.45:0/1292902257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 msgr2=0x7ff02819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff02819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 -- 172.21.15.45:0/1292902257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02819d060 con 0x7ff028101710 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff02759e640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff02819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.192+0000 7ff026d9d640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.193+0000 7ff00ffff640 1 -- 172.21.15.45:0/1292902257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800cd80 con 0x7ff028101710 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.193+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02819d350 con 0x7ff028101710 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.193+0000 7ff00ffff640 1 -- 172.21.15.45:0/1292902257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018010910 con 0x7ff028101710 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.193+0000 7ff00ffff640 1 -- 172.21.15.45:0/1292902257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018018910 con 0x7ff028101710 2024-04-25T00:28:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.193+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281a4be0 con 0x7ff028101710 2024-04-25T00:28:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.195+0000 7ff00ffff640 1 -- 172.21.15.45:0/1292902257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff018010ac0 con 0x7ff028101710 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.195+0000 7ff02ff66740 1 Processor -- start 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.195+0000 7ff02ff66740 1 -- start start 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.195+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa859960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.195+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa866960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02ff66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa7be670 con 0x55fbaa859ea0 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02ff66740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa766970 con 0x55fbaa864570 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02ff66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa7ba630 con 0x55fbaa859580 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff00ffff640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 0x7ff0100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42912/0 (socket says 172.21.15.45:42912) 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 -- 172.21.15.45:0/4236581093 learned_addr learned my addr 172.21.15.45:0/4236581093 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02759e640 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa859960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02659c640 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa866960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02759e640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 0x7ff0100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff00ffff640 1 -- 172.21.15.45:0/1292902257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff0180cfbd0 con 0x7ff028101710 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 -- 172.21.15.45:0/4236581093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 msgr2=0x55fbaa859960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa859960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 -- 172.21.15.45:0/4236581093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 msgr2=0x55fbaa866960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa866960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 -- 172.21.15.45:0/4236581093 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbaa78a400 con 0x55fbaa859ea0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff02759e640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 0x7ff0100a51c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fd0 tx=0x7ff0200094a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff01809ba30 tx=0x7ff01809b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7b502389a807bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.196+0000 7ff00d7fa640 1 -- 172.21.15.45:0/4236581093 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01809bd30 con 0x55fbaa859ea0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff00d7fa640 1 -- 172.21.15.45:0/4236581093 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0180a00e0 con 0x55fbaa859ea0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff00d7fa640 1 -- 172.21.15.45:0/4236581093 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01809b180 con 0x55fbaa859ea0 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 -- 172.21.15.45:0/4236581093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 msgr2=0x55fbaa864030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff01809ba30 tx=0x7ff01809b750 comp rx=0 tx=0).stop 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 -- 172.21.15.45:0/4236581093 shutdown_connections 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa866960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859ea0 0x55fbaa864030 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4236581093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa859960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 -- 172.21.15.45:0/4236581093 >> 172.21.15.45:0/4236581093 conn(0x55fbaa8558a0 msgr2=0x55fbaa855c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 -- 172.21.15.45:0/4236581093 shutdown_connections 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.197+0000 7ff02ff66740 1 -- 172.21.15.45:0/4236581093 wait complete. 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 -- start start 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 Processor -- start 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859580 0x55fbaa863f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa8611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa867360 con 0x55fbaa864570 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa77a050 con 0x55fbaa859580 2024-04-25T00:28:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fbaa8674e0 con 0x55fbaa859ea0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02ff66740 10 client.0 _ll_register_callbacks cb 0x55fbaa4d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42918/0 (socket says 172.21.15.45:42918) 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 -- 172.21.15.45:0/3992535939 learned_addr learned my addr 172.21.15.45:0/3992535939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff026d9d640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859580 0x55fbaa863f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02659c640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa8611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 -- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 msgr2=0x55fbaa8611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa8611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 -- 172.21.15.45:0/3992535939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859580 msgr2=0x55fbaa863f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859580 0x55fbaa863f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.198+0000 7ff02759e640 1 -- 172.21.15.45:0/3992535939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbaa74bdc0 con 0x55fbaa864570 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02759e640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff020012a40 tx=0x7ff020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4411810222149cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020010d60 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff020004b20 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0180a00e0 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff020031050 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 10 client.17517 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbaa4d23f0 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0180a00e0 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff020029400 con 0x55fbaa864570 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 10 client.17517 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 20 client.17517 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa874650 0x55fbaa876ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 4 client.17517 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.199+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbaa7cbbb0 con 0x55fbaa874650 2024-04-25T00:28:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.200+0000 7ff026d9d640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa874650 0x55fbaa876ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.200+0000 7ff00aff5640 10 client.17517 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.200+0000 7ff026d9d640 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa874650 0x55fbaa876ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75f52cd68ed611e server_cookie=5e20070918dd4c65 in_seq=0 out_seq=0 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff00aff5640 1 -- 172.21.15.45:0/3992535939 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbaa7cbbb0 con 0x55fbaa874650 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff00aff5640 10 client.17517 handle_command_reply: tid=0 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 client.17517 shutdown 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 msgr2=0x55fbaa861770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff020012a40 tx=0x7ff020004a40 comp rx=0 tx=0).stop 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 shutdown_connections 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa874650 0x55fbaa876ad0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa861770 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa8611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 --2- 172.21.15.45:0/3992535939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859580 0x55fbaa863f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff00aff5640 0 client.17517 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 >> 172.21.15.45:0/3992535939 conn(0x55fbaa8558a0 msgr2=0x55fbaa85b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 shutdown_connections 2024-04-25T00:28:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 1 -- 172.21.15.45:0/3992535939 wait complete. 2024-04-25T00:28:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.201+0000 7ff02ff66740 20 client.17517 trim_cache size 0 max 16384 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 Processor -- start 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 -- start start 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa870220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859ea0 0x55fbaa87c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281aec30 con 0x55fbaa864570 2024-04-25T00:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b0770 con 0x55fbaa859ea0 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.214+0000 7ff02ff66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281af300 con 0x55fbaa859580 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42922/0 (socket says 172.21.15.45:42922) 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 -- 172.21.15.45:0/4220997598 learned_addr learned my addr 172.21.15.45:0/4220997598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa870220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02659c640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859ea0 0x55fbaa87c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 -- 172.21.15.45:0/4220997598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 msgr2=0x55fbaa870220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa870220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 -- 172.21.15.45:0/4220997598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859ea0 msgr2=0x55fbaa87c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859ea0 0x55fbaa87c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 -- 172.21.15.45:0/4220997598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbaa859cd0 con 0x55fbaa864570 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa870220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff02759e640 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff020009c20 tx=0x7ff020009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0b54731d85512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff00cff9640 1 -- 172.21.15.45:0/4220997598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020039070 con 0x55fbaa864570 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff00cff9640 1 -- 172.21.15.45:0/4220997598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff020010820 con 0x55fbaa864570 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.215+0000 7ff00cff9640 1 -- 172.21.15.45:0/4220997598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020035650 con 0x55fbaa864570 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- 172.21.15.45:0/4220997598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 msgr2=0x55fbaa87c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff020009c20 tx=0x7ff020009280 comp rx=0 tx=0).stop 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- 172.21.15.45:0/4220997598 shutdown_connections 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa864570 0x55fbaa87c930 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa859ea0 0x55fbaa87c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4220997598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859580 0x55fbaa870220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- 172.21.15.45:0/4220997598 >> 172.21.15.45:0/4220997598 conn(0x55fbaa8558a0 msgr2=0x55fbaa854b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- 172.21.15.45:0/4220997598 shutdown_connections 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- 172.21.15.45:0/4220997598 wait complete. 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- start start 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 Processor -- start 2024-04-25T00:28:33.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa86a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa86aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b00a0 con 0x55fbaa859580 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281af9d0 con 0x55fbaa864570 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281acc10 con 0x55fbaa859ea0 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff02ff66740 10 client.0 _ll_register_callbacks cb 0x7ff020009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.216+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42928/0 (socket says 172.21.15.45:42928) 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 -- 172.21.15.45:0/4196156153 learned_addr learned my addr 172.21.15.45:0/4196156153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02759e640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa86aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02659c640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa86a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 -- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 msgr2=0x55fbaa86a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa86a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 -- 172.21.15.45:0/4196156153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 msgr2=0x55fbaa86aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa86aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 -- 172.21.15.45:0/4196156153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbaa59c7d0 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02659c640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa86a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff026d9d640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff018098190 tx=0x7ff01800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c4a3222a5289ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff00aff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01809b870 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff0180cd1e0 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff020010820 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0180a00e0 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02ff66740 10 client.17523 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbaa8dad00 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.217+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff020010820 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff018062060 con 0x55fbaa859580 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 10 client.17523 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 20 client.17523 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa8db040 0x55fbaa8dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 4 client.17523 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbaa7c5f00 con 0x55fbaa8db040 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02659c640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa8db040 0x55fbaa8dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff02659c640 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa8db040 0x55fbaa8dd430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=397d4393b7cc8d0 server_cookie=2e3948ec2ce08aee in_seq=0 out_seq=0 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.218+0000 7ff00aff5640 10 client.17523 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff00aff5640 1 -- 172.21.15.45:0/4196156153 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbaa7c5f00 con 0x55fbaa8db040 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff00aff5640 10 client.17523 handle_command_reply: tid=0 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 client.17523 shutdown 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 msgr2=0x55fbaa86a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff018098190 tx=0x7ff01800e940 comp rx=0 tx=0).stop 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 shutdown_connections 2024-04-25T00:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55fbaa8db040 0x55fbaa8dd430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fbaa864570 0x55fbaa86aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbaa859ea0 0x55fbaa86a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 --2- 172.21.15.45:0/4196156153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fbaa859580 0x55fbaa86a120 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff00aff5640 0 client.17523 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 >> 172.21.15.45:0/4196156153 conn(0x55fbaa8558a0 msgr2=0x55fbaa86f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 shutdown_connections 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 1 -- 172.21.15.45:0/4196156153 wait complete. 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02ff66740 20 client.17523 trim_cache size 0 max 16384 2024-04-25T00:28:33.226 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 msgr2=0x7ff0100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 0x7ff0100a51c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fd0 tx=0x7ff0200094a0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 msgr2=0x7ff028198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 shutdown_connections 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0100a2d00 0x7ff0100a51c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff028107720 0x7ff02819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0281020d0 0x7ff0281985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 --2- 172.21.15.45:0/1292902257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff028101710 0x7ff028198070 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.219+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 >> 172.21.15.45:0/1292902257 conn(0x7ff0280fd180 msgr2=0x7ff0280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.220+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 shutdown_connections 2024-04-25T00:28:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.220+0000 7ff02d258640 1 -- 172.21.15.45:0/1292902257 wait complete. 2024-04-25T00:28:33.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:33 smithi045 ceph-mon[29194]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:33.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:33 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2897424536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:33.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:28:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:33 smithi122 ceph-mon[32959]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:33 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2897424536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:33.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 -- 172.21.15.45:0/3331207524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3501029c0 msgr2=0x7fd350102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 --2- 172.21.15.45:0/3331207524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3501029c0 0x7fd350102da0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd340009ca0 tx=0x7fd34002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 -- 172.21.15.45:0/3331207524 shutdown_connections 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 --2- 172.21.15.45:0/3331207524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd35010bd00 0x7fd35010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 --2- 172.21.15.45:0/3331207524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd35010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 --2- 172.21.15.45:0/3331207524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3501029c0 0x7fd350102da0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.334+0000 7fd3555b6640 1 -- 172.21.15.45:0/3331207524 >> 172.21.15.45:0/3331207524 conn(0x7fd350077740 msgr2=0x7fd350077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- 172.21.15.45:0/3331207524 shutdown_connections 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- 172.21.15.45:0/3331207524 wait complete. 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 Processor -- start 2024-04-25T00:28:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- start start 2024-04-25T00:28:33.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3501029c0 0x7fd350100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd350100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd350111b00 con 0x7fd35010bd00 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd350111980 con 0x7fd3501032e0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.335+0000 7fd3555b6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd350111c80 con 0x7fd3501029c0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42940/0 (socket says 172.21.15.45:42940) 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 -- 172.21.15.45:0/4159476002 learned_addr learned my addr 172.21.15.45:0/4159476002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34effd640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3501029c0 0x7fd350100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd350100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 -- 172.21.15.45:0/4159476002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3501029c0 msgr2=0x7fd350100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3501029c0 0x7fd350100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 -- 172.21.15.45:0/4159476002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 msgr2=0x7fd350100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd350100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 -- 172.21.15.45:0/4159476002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3501a6460 con 0x7fd35010bd00 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd350100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d750 tx=0x7fd34800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd337fff640 1 -- 172.21.15.45:0/4159476002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348017070 con 0x7fd35010bd00 2024-04-25T00:28:33.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3501a6750 con 0x7fd35010bd00 2024-04-25T00:28:33.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd337fff640 1 -- 172.21.15.45:0/4159476002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348005500 con 0x7fd35010bd00 2024-04-25T00:28:33.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd337fff640 1 -- 172.21.15.45:0/4159476002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348004d90 con 0x7fd35010bd00 2024-04-25T00:28:33.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.336+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3501a6c50 con 0x7fd35010bd00 2024-04-25T00:28:33.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.338+0000 7fd337fff640 1 -- 172.21.15.45:0/4159476002 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd348004f30 con 0x7fd35010bd00 2024-04-25T00:28:33.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 Processor -- start 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 -- start start 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd337fff640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 0x7fd3380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5635e8221670 con 0x5635e82bc580 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5635e81c9970 con 0x5635e82bcea0 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd35839b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5635e821d630 con 0x5635e82c7570 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 0x7fd3380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd337fff640 1 -- 172.21.15.45:0/4159476002 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd3480cf180 con 0x7fd35010bd00 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42942/0 (socket says 172.21.15.45:42942) 2024-04-25T00:28:33.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 -- 172.21.15.45:0/905073789 learned_addr learned my addr 172.21.15.45:0/905073789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 -- 172.21.15.45:0/905073789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 msgr2=0x5635e82c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.339+0000 7fd34effd640 1 -- 172.21.15.45:0/905073789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 msgr2=0x5635e82c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd34effd640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd34effd640 1 -- 172.21.15.45:0/905073789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e81ed400 con 0x5635e82bc580 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd34effd640 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd340009ca0 tx=0x7fd3400098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3977ca55692afb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 0x7fd3380a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5635e82bdb40 tx=0x7fd34400a6a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd3357fa640 1 -- 172.21.15.45:0/905073789 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340033050 con 0x5635e82bc580 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd3357fa640 1 -- 172.21.15.45:0/905073789 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd340047070 con 0x5635e82bc580 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd3357fa640 1 -- 172.21.15.45:0/905073789 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34002f840 con 0x5635e82bc580 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd35839b740 1 -- 172.21.15.45:0/905073789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 msgr2=0x5635e82bc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.340+0000 7fd35839b740 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd340009ca0 tx=0x7fd3400098f0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- 172.21.15.45:0/905073789 shutdown_connections 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- 172.21.15.45:0/905073789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82bc960 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- 172.21.15.45:0/905073789 >> 172.21.15.45:0/905073789 conn(0x5635e82b88a0 msgr2=0x5635e82b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- 172.21.15.45:0/905073789 shutdown_connections 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- 172.21.15.45:0/905073789 wait complete. 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- start start 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 Processor -- start 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5635e82ca300 con 0x5635e82bc580 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5635e81dd050 con 0x5635e82bcea0 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5635e82ca480 con 0x5635e82c7570 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd35839b740 10 client.0 _ll_register_callbacks cb 0x5635e7f34be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd34effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd34effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42958/0 (socket says 172.21.15.45:42958) 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd34effd640 1 -- 172.21.15.45:0/937191668 learned_addr learned my addr 172.21.15.45:0/937191668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.341+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 -- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 msgr2=0x5635e82c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 -- 172.21.15.45:0/937191668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 msgr2=0x5635e82c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 -- 172.21.15.45:0/937191668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e81aedc0 con 0x5635e82bc580 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd34effd640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd340000f80 tx=0x7fd34002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88004a937668de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34004f070 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd340047070 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd34002f840 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd34005d050 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd35839b740 10 client.17535 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635e7f353f0 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd340047070 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd340004380 con 0x5635e82bc580 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.342+0000 7fd35839b740 10 client.17535 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd35839b740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd35839b740 20 client.17535 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e82d7360 0x5635e82d9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd35839b740 4 client.17535 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635e822ebb0 con 0x5635e82d7360 2024-04-25T00:28:33.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e82d7360 0x5635e82d9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd332ff5640 10 client.17535 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.343+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e82d7360 0x5635e82d9750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b5182bd9b796fc9 server_cookie=84d5cd824df94160 in_seq=0 out_seq=0 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd332ff5640 1 -- 172.21.15.45:0/937191668 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635e822ebb0 con 0x5635e82d7360 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd332ff5640 10 client.17535 handle_command_reply: tid=0 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 client.17535 shutdown 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 msgr2=0x5635e82c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd340000f80 tx=0x7fd34002f000 comp rx=0 tx=0).stop 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 shutdown_connections 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e82d7360 0x5635e82d9750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82c7570 0x5635e82c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 --2- 172.21.15.45:0/937191668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82bc580 0x5635e82c6ee0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd332ff5640 0 client.17535 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:33.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 >> 172.21.15.45:0/937191668 conn(0x5635e82b88a0 msgr2=0x5635e82be6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 shutdown_connections 2024-04-25T00:28:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.344+0000 7fd35839b740 1 -- 172.21.15.45:0/937191668 wait complete. 2024-04-25T00:28:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.345+0000 7fd35839b740 20 client.17535 trim_cache size 0 max 16384 2024-04-25T00:28:33.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 Processor -- start 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 -- start start 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bc580 0x5635e82d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 0x5635e82bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501b0b50 con 0x5635e82c7570 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501b2690 con 0x5635e82bc580 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd35839b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501b1220 con 0x5635e82bcea0 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42970/0 (socket says 172.21.15.45:42970) 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 -- 172.21.15.45:0/23969764 learned_addr learned my addr 172.21.15.45:0/23969764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 0x5635e82bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34effd640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bc580 0x5635e82d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 -- 172.21.15.45:0/23969764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 msgr2=0x5635e82bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 0x5635e82bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 -- 172.21.15.45:0/23969764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bc580 msgr2=0x5635e82d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bc580 0x5635e82d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 -- 172.21.15.45:0/23969764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e7fff7d0 con 0x5635e82c7570 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 0x5635e82bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.358+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd34809b270 tx=0x7fd34800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8966c6c9cf70eb3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd334ff9640 1 -- 172.21.15.45:0/23969764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348020070 con 0x5635e82c7570 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd334ff9640 1 -- 172.21.15.45:0/23969764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348095e50 con 0x5635e82c7570 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd334ff9640 1 -- 172.21.15.45:0/23969764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348009c60 con 0x5635e82c7570 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- 172.21.15.45:0/23969764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 msgr2=0x5635e82c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd34809b270 tx=0x7fd34800de60 comp rx=0 tx=0).stop 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- 172.21.15.45:0/23969764 shutdown_connections 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82c6a10 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bcea0 0x5635e82bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 --2- 172.21.15.45:0/23969764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bc580 0x5635e82d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- 172.21.15.45:0/23969764 >> 172.21.15.45:0/23969764 conn(0x5635e82b88a0 msgr2=0x5635e82beeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- 172.21.15.45:0/23969764 shutdown_connections 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- 172.21.15.45:0/23969764 wait complete. 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.359+0000 7fd35839b740 1 -- start start 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 Processor -- start 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 0x5635e82cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501b1fc0 con 0x5635e82c7570 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501b18f0 con 0x5635e82bcea0 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501aeb30 con 0x5635e82bc580 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd35839b740 10 client.0 _ll_register_callbacks cb 0x7fd340009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42978/0 (socket says 172.21.15.45:42978) 2024-04-25T00:28:33.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 -- 172.21.15.45:0/3628189041 learned_addr learned my addr 172.21.15.45:0/3628189041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34e7fc640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34effd640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 0x5635e82cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 -- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 msgr2=0x5635e82cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 0x5635e82cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 -- 172.21.15.45:0/3628189041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 msgr2=0x5635e82cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 -- 172.21.15.45:0/3628189041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e82cb740 con 0x5635e82c7570 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34effd640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 0x5635e82cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd34f7fe640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd348097cb0 tx=0x7fd348009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19e5e8b822c0deb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.360+0000 7fd332ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348020050 con 0x5635e82c7570 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fd348005220 con 0x5635e82c7570 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd348095e50 con 0x5635e82c7570 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3480a3f50 con 0x5635e82c7570 2024-04-25T00:28:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 10 client.17541 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635e833d390 con 0x5635e82c7570 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd348095e50 con 0x5635e82c7570 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd34801c9d0 con 0x5635e82c7570 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 10 client.17541 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 20 client.17541 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e833da20 0x5635e833fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 4 client.17541 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635e8228f00 con 0x5635e833da20 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.361+0000 7fd34effd640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e833da20 0x5635e833fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd332ff5640 10 client.17541 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd34effd640 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e833da20 0x5635e833fe40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cea500b1ab106c43 server_cookie=9dcf79bec9485ba1 in_seq=0 out_seq=0 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd332ff5640 1 -- 172.21.15.45:0/3628189041 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635e8228f00 con 0x5635e833da20 2024-04-25T00:28:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd332ff5640 10 client.17541 handle_command_reply: tid=0 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 client.17541 shutdown 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 msgr2=0x5635e82cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd348097cb0 tx=0x7fd348009bd0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 shutdown_connections 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5635e833da20 0x5635e833fe40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5635e82c7570 0x5635e82cb150 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e82bcea0 0x5635e82cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd332ff5640 0 client.17541 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 --2- 172.21.15.45:0/3628189041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635e82bc580 0x5635e82cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 >> 172.21.15.45:0/3628189041 conn(0x5635e82b88a0 msgr2=0x5635e82be530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 shutdown_connections 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 1 -- 172.21.15.45:0/3628189041 wait complete. 2024-04-25T00:28:33.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.362+0000 7fd35839b740 20 client.17541 trim_cache size 0 max 16384 2024-04-25T00:28:33.371 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:33.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 msgr2=0x7fd3380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 0x7fd3380a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5635e82bdb40 tx=0x7fd34400a6a0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 msgr2=0x7fd350101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d750 tx=0x7fd34800dc10 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 shutdown_connections 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd3380a2d00 0x7fd3380a51c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd35010bd00 0x7fd350101090 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3501032e0 0x7fd350100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 --2- 172.21.15.45:0/4159476002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3501029c0 0x7fd350100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 >> 172.21.15.45:0/4159476002 conn(0x7fd350077740 msgr2=0x7fd3500fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 shutdown_connections 2024-04-25T00:28:33.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.363+0000 7fd3555b6640 1 -- 172.21.15.45:0/4159476002 wait complete. 2024-04-25T00:28:33.373 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:33 smithi071 ceph-mon[32667]: pgmap v200: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:33.373 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:33 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2897424536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:28:33.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:28:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/2181358708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb41020d0 msgr2=0x7fdfb410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/2181358708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb41020d0 0x7fdfb410e5e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009cd0 tx=0x7fdfac02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/2181358708 shutdown_connections 2024-04-25T00:28:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/2181358708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb41020d0 0x7fdfb410e5e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/2181358708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb4101710 0x7fdfb4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/2181358708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb4107720 0x7fdfb4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/2181358708 >> 172.21.15.45:0/2181358708 conn(0x7fdfb40fd180 msgr2=0x7fdfb40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/2181358708 shutdown_connections 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.479+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/2181358708 wait complete. 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 Processor -- start 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 -- start start 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 0x7fdfb41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb4107720 0x7fdfb419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111b30 con 0x7fdfb4101710 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41119b0 con 0x7fdfb41020d0 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfbaf9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111cb0 con 0x7fdfb4107720 2024-04-25T00:28:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42988/0 (socket says 172.21.15.45:42988) 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/197663308 learned_addr learned my addr 172.21.15.45:0/197663308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 0x7fdfb41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/197663308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb4107720 msgr2=0x7fdfb419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb4107720 0x7fdfb419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/197663308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 msgr2=0x7fdfb41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 0x7fdfb41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/197663308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb419d060 con 0x7fdfb4101710 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.480+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 0x7fdfb41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800d950 tx=0x7fdfa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfb1ffb640 1 -- 172.21.15.45:0/197663308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8017030 con 0x7fdfb4101710 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb419d350 con 0x7fdfb4101710 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfb1ffb640 1 -- 172.21.15.45:0/197663308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa8004020 con 0x7fdfb4101710 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfb1ffb640 1 -- 172.21.15.45:0/197663308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80047a0 con 0x7fdfb4101710 2024-04-25T00:28:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.481+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb41a4be0 con 0x7fdfb4101710 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.483+0000 7fdfb1ffb640 1 -- 172.21.15.45:0/197663308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdfa80041d0 con 0x7fdfb4101710 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 Processor -- start 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 -- start start 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb1ffb640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 0x7fdfa40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed01960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a26ec66670 con 0x55a26ed0c570 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a26ec0e970 con 0x55a26ed01580 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a26ec62630 con 0x55a26ed01ea0 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb953b640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 0x7fdfa40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb1ffb640 1 -- 172.21.15.45:0/197663308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdfa80cf200 con 0x7fdfb4101710 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43004/0 (socket says 172.21.15.45:43004) 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb8d3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed01960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb953b640 1 -- 172.21.15.45:0/1494626357 learned_addr learned my addr 172.21.15.45:0/1494626357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.484+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/1494626357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 msgr2=0x55a26ed0c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/1494626357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 msgr2=0x55a26ed0e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/1494626357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26ec32400 con 0x55a26ed01580 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb953b640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb953b640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 0x7fdfa40a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009430 tx=0x7fdfac0098d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed01960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80cc5d0 tx=0x7fdfa809afa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450132e38f65b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdf9effd640 1 -- 172.21.15.45:0/1494626357 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80666c0 con 0x55a26ed01580 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdf9effd640 1 -- 172.21.15.45:0/1494626357 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa8067290 con 0x55a26ed01580 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.485+0000 7fdf9effd640 1 -- 172.21.15.45:0/1494626357 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80cc850 con 0x55a26ed01580 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1494626357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 msgr2=0x55a26ed01960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed01960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80cc5d0 tx=0x7fdfa809afa0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1494626357 shutdown_connections 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed0e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1494626357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed01960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1494626357 >> 172.21.15.45:0/1494626357 conn(0x55a26ecfd8a0 msgr2=0x55a26ecfdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1494626357 shutdown_connections 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1494626357 wait complete. 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 -- start start 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.486+0000 7fdfbdd89740 1 Processor -- start 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01ea0 0x55a26ed09220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 0x55a26ed09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a26ed0f360 con 0x55a26ed01580 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a26ec22050 con 0x55a26ed01ea0 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a26ed0f4e0 con 0x55a26ed0c570 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfbdd89740 10 client.0 _ll_register_callbacks cb 0x55a26e979be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43014/0 (socket says 172.21.15.45:43014) 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/2714735332 learned_addr learned my addr 172.21.15.45:0/2714735332 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01ea0 0x55a26ed09220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/2714735332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 msgr2=0x55a26ed09830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb953b640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 0x55a26ed09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 0x55a26ed09830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01ea0 msgr2=0x55a26ed09220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01ea0 0x55a26ed09220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 -- 172.21.15.45:0/2714735332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26ebf3dc0 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb953b640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 0x55a26ed09830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009610 tx=0x7fdfa80998b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a5af1dcc815ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.487+0000 7fdf93fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80cca50 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdfa809b600 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa8067290 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdfa809b3a0 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 10 client.17553 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a26e97a3f0 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa8067290 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdfa8075a70 con 0x55a26ed01580 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 10 client.17553 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 20 client.17553 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed208c0 0x55a26ed22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 4 client.17553 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a26ec73bb0 con 0x55a26ed208c0 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.488+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed208c0 0x55a26ed22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.489+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed208c0 0x55a26ed22cb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2565b91cdd21c165 server_cookie=615d3e97dbfd4a68 in_seq=0 out_seq=0 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.489+0000 7fdf93fff640 10 client.17553 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdf93fff640 1 -- 172.21.15.45:0/2714735332 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a26ec73bb0 con 0x55a26ed208c0 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdf93fff640 10 client.17553 handle_command_reply: tid=0 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 client.17553 shutdown 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 msgr2=0x55a26ed0bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009610 tx=0x7fdfa80998b0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 shutdown_connections 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed0c570 0x55a26ed09830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01ea0 0x55a26ed09220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed208c0 0x55a26ed22cb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2714735332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed01580 0x55a26ed0bf30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdf93fff640 0 client.17553 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 >> 172.21.15.45:0/2714735332 conn(0x55a26ecfd8a0 msgr2=0x55a26ed036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 shutdown_connections 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2714735332 wait complete. 2024-04-25T00:28:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.490+0000 7fdfbdd89740 20 client.17553 trim_cache size 0 max 16384 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.503+0000 7fdfbdd89740 1 Processor -- start 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.503+0000 7fdfbdd89740 1 -- start start 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.503+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.503+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ac9f0 con 0x55a26ed0c570 2024-04-25T00:28:33.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ae530 con 0x55a26ed01580 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ad0c0 con 0x55a26ed01ea0 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43028/0 (socket says 172.21.15.45:43028) 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 -- 172.21.15.45:0/1929389412 learned_addr learned my addr 172.21.15.45:0/1929389412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 -- 172.21.15.45:0/1929389412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 msgr2=0x55a26ed0ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed1bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 -- 172.21.15.45:0/1929389412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 msgr2=0x55a26ed1bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed1bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 -- 172.21.15.45:0/1929389412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26ea447d0 con 0x55a26ed0c570 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed1bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:33.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdfb953b640 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031670 tx=0x7fdfac0093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc45a6ac107a489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdf9e7fc640 1 -- 172.21.15.45:0/1929389412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac04e070 con 0x55a26ed0c570 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdf9e7fc640 1 -- 172.21.15.45:0/1929389412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac036050 con 0x55a26ed0c570 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.504+0000 7fdf9e7fc640 1 -- 172.21.15.45:0/1929389412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac030d10 con 0x55a26ed0c570 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1929389412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 msgr2=0x55a26ed25cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031670 tx=0x7fdfac0093a0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1929389412 shutdown_connections 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed25cd0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed0ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/1929389412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed1bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1929389412 >> 172.21.15.45:0/1929389412 conn(0x55a26ecfd8a0 msgr2=0x55a26ecfcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1929389412 shutdown_connections 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- 172.21.15.45:0/1929389412 wait complete. 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 -- start start 2024-04-25T00:28:33.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.505+0000 7fdfbdd89740 1 Processor -- start 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ade60 con 0x55a26ed0c570 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ad790 con 0x55a26ed01580 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41aa9d0 con 0x55a26ed01ea0 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 10 client.0 _ll_register_callbacks cb 0x7fdfa0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43044/0 (socket says 172.21.15.45:43044) 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 -- 172.21.15.45:0/2109445630 learned_addr learned my addr 172.21.15.45:0/2109445630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb3fff640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 -- 172.21.15.45:0/2109445630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 msgr2=0x55a26ed234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 -- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 msgr2=0x55a26ed26120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed26120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 -- 172.21.15.45:0/2109445630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26ed24240 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfb953b640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031eb0 tx=0x7fdfac0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6204c380c56cf16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdf93fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac05a020 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdfac004710 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.506+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac036050 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdfac065050 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 10 client.17559 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a26ed86aa0 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac036050 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdfac030070 con 0x55a26ed0c570 2024-04-25T00:28:33.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 10 client.17559 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 20 client.17559 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed871c0 0x55a26ed895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 4 client.17559 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a26ec6df00 con 0x55a26ed871c0 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.507+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed871c0 0x55a26ed895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdf93fff640 10 client.17559 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfb8d3a640 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed871c0 0x55a26ed895e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8728721044117ece server_cookie=37c12e467886305e in_seq=0 out_seq=0 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdf93fff640 1 -- 172.21.15.45:0/2109445630 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a26ec6df00 con 0x55a26ed871c0 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdf93fff640 10 client.17559 handle_command_reply: tid=0 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 client.17559 shutdown 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 msgr2=0x55a26ed23a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdfac031eb0 tx=0x7fdfac0042e0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 shutdown_connections 2024-04-25T00:28:33.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55a26ed871c0 0x55a26ed895e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a26ed0c570 0x55a26ed23a10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26ed01ea0 0x55a26ed234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdf93fff640 0 client.17559 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 --2- 172.21.15.45:0/2109445630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a26ed01580 0x55a26ed26120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 >> 172.21.15.45:0/2109445630 conn(0x55a26ecfd8a0 msgr2=0x55a26ed07790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 shutdown_connections 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 1 -- 172.21.15.45:0/2109445630 wait complete. 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.508+0000 7fdfbdd89740 20 client.17559 trim_cache size 0 max 16384 2024-04-25T00:28:33.516 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:28:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 msgr2=0x7fdfa40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 0x7fdfa40a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009430 tx=0x7fdfac0098d0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 msgr2=0x7fdfb4198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800d950 tx=0x7fdfa800de10 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 shutdown_connections 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdfa40a2d00 0x7fdfa40a51c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb4107720 0x7fdfb419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb41020d0 0x7fdfb41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 --2- 172.21.15.45:0/197663308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb4101710 0x7fdfb4198070 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 >> 172.21.15.45:0/197663308 conn(0x7fdfb40fd180 msgr2=0x7fdfb40fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 shutdown_connections 2024-04-25T00:28:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:33.509+0000 7fdfbaf9e640 1 -- 172.21.15.45:0/197663308 wait complete. 2024-04-25T00:28:34.521 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:28:34.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:28:34.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:28:34.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 -- 172.21.15.45:0/544023853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c06a780 msgr2=0x7fdf9c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/544023853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c06a780 0x7fdf9c10e5f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009cd0 tx=0x7fdf9402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 -- 172.21.15.45:0/544023853 shutdown_connections 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/544023853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c06a780 0x7fdf9c10e5f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/544023853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c069dc0 0x7fdf9c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/544023853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c107730 0x7fdf9c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 -- 172.21.15.45:0/544023853 >> 172.21.15.45:0/544023853 conn(0x7fdf9c0776a0 msgr2=0x7fdf9c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 -- 172.21.15.45:0/544023853 shutdown_connections 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.640+0000 7fdfa31b0640 1 -- 172.21.15.45:0/544023853 wait complete. 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 Processor -- start 2024-04-25T00:28:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 -- start start 2024-04-25T00:28:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c069dc0 0x7fdf9c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 0x7fdf9c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c111b40 con 0x7fdf9c107730 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1119c0 con 0x7fdf9c06a780 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.641+0000 7fdfa31b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c111cc0 con 0x7fdf9c069dc0 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36162/0 (socket says 172.21.15.45:36162) 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 -- 172.21.15.45:0/2039726749 learned_addr learned my addr 172.21.15.45:0/2039726749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c069dc0 0x7fdf9c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 0x7fdf9c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 -- 172.21.15.45:0/2039726749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c069dc0 msgr2=0x7fdf9c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c069dc0 0x7fdf9c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 -- 172.21.15.45:0/2039726749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 msgr2=0x7fdf9c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 0x7fdf9c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 -- 172.21.15.45:0/2039726749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c19cf50 con 0x7fdf9c107730 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 0x7fdf9c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa174d640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005cc0 tx=0x7fdf940093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdf99ffb640 1 -- 172.21.15.45:0/2039726749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94003e00 con 0x7fdf9c107730 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdf99ffb640 1 -- 172.21.15.45:0/2039726749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94004520 con 0x7fdf9c107730 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c19d1e0 con 0x7fdf9c107730 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.642+0000 7fdf99ffb640 1 -- 172.21.15.45:0/2039726749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94030db0 con 0x7fdf9c107730 2024-04-25T00:28:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.643+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1a4ad0 con 0x7fdf9c107730 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdf99ffb640 1 -- 172.21.15.45:0/2039726749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdf940046d0 con 0x7fdf9c107730 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 Processor -- start 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 -- start start 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6be2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6bf1420 0x55f3f6bf3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6b462c0 con 0x55f3f6be3980 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6aee910 con 0x55f3f6bf1420 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6b42280 con 0x55f3f6be2290 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36174/0 (socket says 172.21.15.45:36174) 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.645+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2659882083 learned_addr learned my addr 172.21.15.45:0/2659882083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf99ffb640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 0x7fdf8c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6be2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6bf1420 0x55f3f6bf3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdfa174d640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 0x7fdf8c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2659882083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 msgr2=0x55f3f6be2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6be2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2659882083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6bf1420 msgr2=0x55f3f6bf3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6bf1420 0x55f3f6bf3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2659882083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f6afffb0 con 0x55f3f6be3980 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf99ffb640 1 -- 172.21.15.45:0/2039726749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdf9402e030 con 0x7fdf9c107730 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fdf88013610 tx=0x7fdf880097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e59bc871175f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdfa174d640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 0x7fdf8c0a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf940ee330 tx=0x7fdf9408f010 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf86ffd640 1 -- 172.21.15.45:0/2659882083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800f520 con 0x55f3f6be3980 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf86ffd640 1 -- 172.21.15.45:0/2659882083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8800fba0 con 0x55f3f6be3980 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.646+0000 7fdf86ffd640 1 -- 172.21.15.45:0/2659882083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88010aa0 con 0x55f3f6be3980 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2659882083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 msgr2=0x55f3f6be1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fdf88013610 tx=0x7fdf880097d0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2659882083 shutdown_connections 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6bf1420 0x55f3f6bf3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6be2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2659882083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6be1cc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2659882083 >> 172.21.15.45:0/2659882083 conn(0x55f3f6baa950 msgr2=0x55f3f6bde7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2659882083 shutdown_connections 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2659882083 wait complete. 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.647+0000 7fdfa5f9f740 1 -- start start 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 Processor -- start 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6bee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6bf42b0 con 0x55f3f6bf1420 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6b01e90 con 0x55f3f6be3980 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3f6bf4430 con 0x55f3f6be2290 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa5f9f740 10 client.0 _ll_register_callbacks cb 0x55f3f6b3d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36180/0 (socket says 172.21.15.45:36180) 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdf9bfff640 1 -- 172.21.15.45:0/1877682015 learned_addr learned my addr 172.21.15.45:0/1877682015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6bee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.648+0000 7fdf9bfff640 1 -- 172.21.15.45:0/1877682015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 msgr2=0x55f3f6bee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6bee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf9bfff640 1 -- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 msgr2=0x55f3f6bee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf9bfff640 1 -- 172.21.15.45:0/1877682015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f69227e0 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009ee0 tx=0x7fdf88010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92a3b4ca2905662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800f730 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdf8800f8d0 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8800fba0 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf88031050 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdfa5f9f740 10 client.17571 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3f6982f70 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8800fba0 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.649+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf880046c0 con 0x55f3f6bf1420 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 10 client.17571 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 20 client.17571 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c01440 0x55f3f6c03830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 4 client.17571 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3f6b1a5a0 con 0x55f3f6c01440 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c01440 0x55f3f6c03830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c01440 0x55f3f6c03830 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b7166eb967ae59 server_cookie=dd15a8c4fda30b36 in_seq=0 out_seq=0 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.650+0000 7fdf7bfff640 10 client.17571 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.651+0000 7fdf7bfff640 1 -- 172.21.15.45:0/1877682015 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3f6b1a5a0 con 0x55f3f6c01440 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.651+0000 7fdf7bfff640 10 client.17571 handle_command_reply: tid=0 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.651+0000 7fdfa5f9f740 1 client.17571 shutdown 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.651+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 msgr2=0x55f3f6beed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.651+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009ee0 tx=0x7fdf88010680 comp rx=0 tx=0).stop 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 shutdown_connections 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c01440 0x55f3f6c03830 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6bf1420 0x55f3f6beed80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdf7bfff640 0 client.17571 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/1877682015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6be2290 0x55f3f6bee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 >> 172.21.15.45:0/1877682015 conn(0x55f3f6baa950 msgr2=0x55f3f6be45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 shutdown_connections 2024-04-25T00:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/1877682015 wait complete. 2024-04-25T00:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.652+0000 7fdfa5f9f740 20 client.17571 trim_cache size 0 max 16384 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.669+0000 7fdfa5f9f740 1 Processor -- start 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.669+0000 7fdfa5f9f740 1 -- start start 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bdd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6c08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1ac800 con 0x55f3f6be2290 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1ae340 con 0x55f3f6be3980 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1aced0 con 0x55f3f6bf1420 2024-04-25T00:28:34.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36182/0 (socket says 172.21.15.45:36182) 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2985565518 learned_addr learned my addr 172.21.15.45:0/2985565518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bdd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2985565518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 msgr2=0x55f3f6c08f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6c08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2985565518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 msgr2=0x55f3f6bdd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bdd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 -- 172.21.15.45:0/2985565518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f681a170 con 0x55f3f6be2290 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009be0 tx=0x7fdf88004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e562ca6ddc4cb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf867fc640 1 -- 172.21.15.45:0/2985565518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880096e0 con 0x55f3f6be2290 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf867fc640 1 -- 172.21.15.45:0/2985565518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8800f730 con 0x55f3f6be2290 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.670+0000 7fdf867fc640 1 -- 172.21.15.45:0/2985565518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800fe00 con 0x55f3f6be2290 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2985565518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 msgr2=0x55f3f6bfd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009be0 tx=0x7fdf88004ce0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2985565518 shutdown_connections 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6c08f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be3980 0x55f3f6bdd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/2985565518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be2290 0x55f3f6bfd1e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2985565518 >> 172.21.15.45:0/2985565518 conn(0x55f3f6baa950 msgr2=0x55f3f6bdd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2985565518 shutdown_connections 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/2985565518 wait complete. 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 -- start start 2024-04-25T00:28:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.671+0000 7fdfa5f9f740 1 Processor -- start 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be2290 0x55f3f6beb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6bebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1adc70 con 0x55f3f6be3980 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1ad5a0 con 0x55f3f6be2290 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1aa7e0 con 0x55f3f6bf1420 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa5f9f740 10 client.0 _ll_register_callbacks cb 0x7fdf90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36196/0 (socket says 172.21.15.45:36196) 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 -- 172.21.15.45:0/3024776497 learned_addr learned my addr 172.21.15.45:0/3024776497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be2290 0x55f3f6beb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6bebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 -- 172.21.15.45:0/3024776497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 msgr2=0x55f3f6bebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6bebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 -- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be2290 msgr2=0x55f3f6beb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be2290 0x55f3f6beb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.672+0000 7fdfa0f4c640 1 -- 172.21.15.45:0/3024776497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f6c66860 con 0x55f3f6be3980 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa0f4c640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x55f3f6c03f10 tx=0x7fdf900108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc0cf7017e6607f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90026060 con 0x55f3f6be3980 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdf90020110 con 0x55f3f6be3980 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8800f730 con 0x55f3f6be3980 2024-04-25T00:28:34.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf9002d050 con 0x55f3f6be3980 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 10 client.17577 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3f6c67c00 con 0x55f3f6be3980 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8800f730 con 0x55f3f6be3980 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf90003bd0 con 0x55f3f6be3980 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 10 client.17577 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 20 client.17577 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c68360 0x55f3f6c6a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 4 client.17577 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.673+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f3f6b53800 con 0x55f3f6c68360 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.674+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c68360 0x55f3f6c6a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.674+0000 7fdf9bfff640 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c68360 0x55f3f6c6a780 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea066a9e6e6daf81 server_cookie=8437495182823939 in_seq=0 out_seq=0 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.674+0000 7fdf7bfff640 10 client.17577 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3024776497 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3f6b53800 con 0x55f3f6c68360 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdf7bfff640 10 client.17577 handle_command_reply: tid=0 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 client.17577 shutdown 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 msgr2=0x55f3f6beb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x55f3f6c03f10 tx=0x7fdf900108c0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 shutdown_connections 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3f6bf1420 0x55f3f6bebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f3f6c68360 0x55f3f6c6a780 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f3f6be3980 0x55f3f6beb5b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 --2- 172.21.15.45:0/3024776497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f3f6be2290 0x55f3f6beb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 >> 172.21.15.45:0/3024776497 conn(0x55f3f6baa950 msgr2=0x55f3f6bdd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdf7bfff640 0 client.17577 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 shutdown_connections 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.675+0000 7fdfa5f9f740 1 -- 172.21.15.45:0/3024776497 wait complete. 2024-04-25T00:28:34.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa5f9f740 20 client.17577 trim_cache size 0 max 16384 2024-04-25T00:28:34.682 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:28:34.682 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:28:34.682 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "af20edaf-4e84-43fb-bbd8-39c40c2668c5", 2024-04-25T00:28:34.682 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:28:34.682 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 msgr2=0x7fdf8c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 0x7fdf8c0a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf940ee330 tx=0x7fdf9408f010 comp rx=0 tx=0).stop 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 msgr2=0x7fdf9c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005cc0 tx=0x7fdf940093d0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 shutdown_connections 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf8c0a2d00 0x7fdf8c0a51c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9c107730 0x7fdf9c19c820 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c06a780 0x7fdf9c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 --2- 172.21.15.45:0/2039726749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf9c069dc0 0x7fdf9c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 >> 172.21.15.45:0/2039726749 conn(0x7fdf9c0776a0 msgr2=0x7fdf9c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 shutdown_connections 2024-04-25T00:28:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.676+0000 7fdfa31b0640 1 -- 172.21.15.45:0/2039726749 wait complete. 2024-04-25T00:28:34.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:28:34.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:28:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 -- 172.21.15.45:0/3122659917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144107720 msgr2=0x7f0144107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 --2- 172.21.15.45:0/3122659917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144107720 0x7f0144107b00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0134009930 tx=0x7f013402a420 comp rx=0 tx=0).stop 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 -- 172.21.15.45:0/3122659917 shutdown_connections 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 --2- 172.21.15.45:0/3122659917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f014410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 --2- 172.21.15.45:0/3122659917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144101710 0x7f0144101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 --2- 172.21.15.45:0/3122659917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144107720 0x7f0144107b00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.791+0000 7f0149f7e640 1 -- 172.21.15.45:0/3122659917 >> 172.21.15.45:0/3122659917 conn(0x7f01440fd180 msgr2=0x7f01440ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- 172.21.15.45:0/3122659917 shutdown_connections 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- 172.21.15.45:0/3122659917 wait complete. 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 Processor -- start 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- start start 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f01441985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144107720 0x7f014419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0144111b30 con 0x7f0144101710 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01441119b0 con 0x7f0144107720 2024-04-25T00:28:34.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.792+0000 7f0149f7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0144111cb0 con 0x7f01441020d0 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36214/0 (socket says 172.21.15.45:36214) 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 -- 172.21.15.45:0/1789987404 learned_addr learned my addr 172.21.15.45:0/1789987404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0143fff640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144107720 0x7f014419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0142ffd640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f01441985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 -- 172.21.15.45:0/1789987404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 msgr2=0x7f01441985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f01441985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 -- 172.21.15.45:0/1789987404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144107720 msgr2=0x7f014419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144107720 0x7f014419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 -- 172.21.15.45:0/1789987404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014419d060 con 0x7f0144101710 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0142ffd640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f01441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f01437fe640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0134009230 tx=0x7f0134004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0140ff9640 1 -- 172.21.15.45:0/1789987404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013403f030 con 0x7f0144101710 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0140ff9640 1 -- 172.21.15.45:0/1789987404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134003ad0 con 0x7f0144101710 2024-04-25T00:28:34.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014419d2f0 con 0x7f0144101710 2024-04-25T00:28:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0140ff9640 1 -- 172.21.15.45:0/1789987404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134035070 con 0x7f0144101710 2024-04-25T00:28:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.793+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01441a4be0 con 0x7f0144101710 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.795+0000 7f0140ff9640 1 -- 172.21.15.45:0/1789987404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0134042030 con 0x7f0144101710 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 Processor -- start 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 -- start start 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739530 0x55f655739910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655748a30 0x55f65574ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f65569e6b0 con 0x55f655739e50 2024-04-25T00:28:34.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f655646880 con 0x55f655748a30 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f014cd6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f65569a670 con 0x55f655739530 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f0140ff9640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 0x7f012c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36218/0 (socket says 172.21.15.45:36218) 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 -- 172.21.15.45:0/4033438852 learned_addr learned my addr 172.21.15.45:0/4033438852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f0143fff640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 0x7f012c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f0143fff640 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739530 0x55f655739910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f0142ffd640 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655748a30 0x55f65574ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f0140ff9640 1 -- 172.21.15.45:0/1789987404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f01340efdf0 con 0x7f0144101710 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 -- 172.21.15.45:0/4033438852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739530 msgr2=0x55f655739910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739530 0x55f655739910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 -- 172.21.15.45:0/4033438852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655748a30 msgr2=0x55f65574ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.796+0000 7f01437fe640 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655748a30 0x55f65574ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f01437fe640 1 -- 172.21.15.45:0/4033438852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65547c7d0 con 0x55f655739e50 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f0143fff640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 0x7f012c0a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f013c0098b0 tx=0x7f013c009d90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f01437fe640 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f0134047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe0e20b0ab00bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f0129ffb640 1 -- 172.21.15.45:0/4033438852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134049070 con 0x55f655739e50 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f0129ffb640 1 -- 172.21.15.45:0/4033438852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01340bbfb0 con 0x55f655739e50 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f0129ffb640 1 -- 172.21.15.45:0/4033438852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340ed460 con 0x55f655739e50 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f014cd6e740 1 -- 172.21.15.45:0/4033438852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 msgr2=0x55f6557484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f014cd6e740 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f0134047000 comp rx=0 tx=0).stop 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f014cd6e740 1 -- 172.21.15.45:0/4033438852 shutdown_connections 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.797+0000 7f014cd6e740 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655748a30 0x55f65574ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655739e50 0x55f6557484f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 --2- 172.21.15.45:0/4033438852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739530 0x55f655739910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- 172.21.15.45:0/4033438852 >> 172.21.15.45:0/4033438852 conn(0x55f655735950 msgr2=0x55f655735d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- 172.21.15.45:0/4033438852 shutdown_connections 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- 172.21.15.45:0/4033438852 wait complete. 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- start start 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 Processor -- start 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655745910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655745e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f65569e6b0 con 0x55f655748a30 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f655659f50 con 0x55f655739530 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f65574b5b0 con 0x55f655739e50 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.798+0000 7f014cd6e740 10 client.0 _ll_register_callbacks cb 0x55f655695000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655745910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36220/0 (socket says 172.21.15.45:36220) 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0143fff640 1 -- 172.21.15.45:0/2693691501 learned_addr learned my addr 172.21.15.45:0/2693691501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0142ffd640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655745e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 -- 172.21.15.45:0/2693691501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 msgr2=0x55f655745e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655745e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 -- 172.21.15.45:0/2693691501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 msgr2=0x55f655746390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 -- 172.21.15.45:0/2693691501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6553b23f0 con 0x55f655739530 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0143fff640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f01437fe640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655745910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0134009a50 tx=0x7f013402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f32d65f48810aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:28:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134049040 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01340bbfb0 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f01340ed460 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f013403be60 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f014cd6e740 10 client.26891 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.799+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6553720f0 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01340bbfb0 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0134031c60 con 0x55f655739530 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 10 client.26891 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 20 client.26891 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f655758770 0x55f65575ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 4 client.26891 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6556abbf0 con 0x55f655758770 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.800+0000 7f0142ffd640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f655758770 0x55f65575ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f0122ffd640 10 client.26891 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f0142ffd640 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f655758770 0x55f65575ab60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a663f55a3f74b1e1 server_cookie=e7cc759ba14c515a in_seq=0 out_seq=0 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f0122ffd640 1 -- 172.21.15.45:0/2693691501 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6556abbf0 con 0x55f655758770 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f0122ffd640 10 client.26891 handle_command_reply: tid=0 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 client.26891 shutdown 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 msgr2=0x55f655745910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655745910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0134009a50 tx=0x7f013402a420 comp rx=0 tx=0).stop 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 shutdown_connections 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f655758770 0x55f65575ab60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655746390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655745e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f0122ffd640 0 client.26891 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2693691501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655745910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.801+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 >> 172.21.15.45:0/2693691501 conn(0x55f655735950 msgr2=0x55f65573b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.802+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 shutdown_connections 2024-04-25T00:28:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.802+0000 7f014cd6e740 1 -- 172.21.15.45:0/2693691501 wait complete. 2024-04-25T00:28:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.802+0000 7f014cd6e740 20 client.26891 trim_cache size 0 max 16384 2024-04-25T00:28:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 Processor -- start 2024-04-25T00:28:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 -- start start 2024-04-25T00:28:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655748040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655754360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01441ac9d0 con 0x55f655748a30 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01441ae510 con 0x55f655739530 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f014cd6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01441ad0a0 con 0x55f655739e50 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36236/0 (socket says 172.21.15.45:36236) 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.819+0000 7f0143fff640 1 -- 172.21.15.45:0/2567250406 learned_addr learned my addr 172.21.15.45:0/2567250406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0142ffd640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655754360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f01437fe640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655748040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 -- 172.21.15.45:0/2567250406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 msgr2=0x55f655754360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655754360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 -- 172.21.15.45:0/2567250406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 msgr2=0x55f655748040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655748040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 -- 172.21.15.45:0/2567250406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f655727090 con 0x55f655748a30 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0142ffd640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655754360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f0143fff640 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f013c009a90 tx=0x7f013c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d7803a6ceab018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f01297fa640 1 -- 172.21.15.45:0/2567250406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004030 con 0x55f655748a30 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f01297fa640 1 -- 172.21.15.45:0/2567250406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c0046b0 con 0x55f655748a30 2024-04-25T00:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.820+0000 7f01297fa640 1 -- 172.21.15.45:0/2567250406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c020cf0 con 0x55f655748a30 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- 172.21.15.45:0/2567250406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 msgr2=0x55f655744940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f013c009a90 tx=0x7f013c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- 172.21.15.45:0/2567250406 shutdown_connections 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655744940 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655754360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- 172.21.15.45:0/2567250406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655748040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- 172.21.15.45:0/2567250406 >> 172.21.15.45:0/2567250406 conn(0x55f655735950 msgr2=0x55f6557458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- 172.21.15.45:0/2567250406 shutdown_connections 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- 172.21.15.45:0/2567250406 wait complete. 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- start start 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 Processor -- start 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655742610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655742b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01441ade40 con 0x55f655748a30 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01441ad770 con 0x55f655739530 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.821+0000 7f014cd6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01441aa9b0 con 0x55f655739e50 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f014cd6e740 10 client.0 _ll_register_callbacks cb 0x7f01380095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36250/0 (socket says 172.21.15.45:36250) 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 -- 172.21.15.45:0/1866810381 learned_addr learned my addr 172.21.15.45:0/1866810381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f01437fe640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655742610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0142ffd640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655742b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 -- 172.21.15.45:0/1866810381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 msgr2=0x55f655742b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655742b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 -- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 msgr2=0x55f655742610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655742610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 -- 172.21.15.45:0/1866810381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6557bdc60 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0143fff640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f013c020870 tx=0x7f013c020ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a599d17794bd291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004230 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f013c0043d0 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013c0046b0 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f013c033430 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f014cd6e740 10 client.17595 fetch_fsmap learned FSMap version 28 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.822+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6557beea0 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c0046b0 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f013c01f070 con 0x55f655748a30 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 10 client.17595 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:28:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 20 client.17595 populate_metadata read hostname 'smithi045' 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f6557bf590 0x55f6557c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 4 client.17595 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f655744fc0 con 0x55f6557bf590 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f01437fe640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f6557bf590 0x55f6557c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f0122ffd640 10 client.17595 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.823+0000 7f01437fe640 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f6557bf590 0x55f6557c19b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27cde221dc6f3d0 server_cookie=4094d904d61551a4 in_seq=0 out_seq=0 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f0122ffd640 1 -- 172.21.15.45:0/1866810381 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f655744fc0 con 0x55f6557bf590 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f0122ffd640 10 client.17595 handle_command_reply: tid=0 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 client.17595 shutdown 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 msgr2=0x55f655743090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f013c020870 tx=0x7f013c020ca0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 shutdown_connections 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f6557bf590 0x55f6557c19b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f655748a30 0x55f655743090 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f655739e50 0x55f655742b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 --2- 172.21.15.45:0/1866810381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f655739530 0x55f655742610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f0122ffd640 0 client.17595 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 >> 172.21.15.45:0/1866810381 conn(0x55f655735950 msgr2=0x55f6557447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 shutdown_connections 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 1 -- 172.21.15.45:0/1866810381 wait complete. 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.824+0000 7f014cd6e740 20 client.17595 trim_cache size 0 max 16384 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:28:34.832 INFO:teuthology.orchestra.run.smithi045.stdout: "af20edaf-4e84-43fb-bbd8-39c40c2668c5": { 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "af20edaf-4e84-43fb-bbd8-39c40c2668c5", 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:28:34.833 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 msgr2=0x7f012c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 0x7f012c0a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f013c0098b0 tx=0x7f013c009d90 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 msgr2=0x7f0144198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0134009230 tx=0x7f0134004270 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 shutdown_connections 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f012c0a2d00 0x7f012c0a51c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0144107720 0x7f014419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01441020d0 0x7f01441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 --2- 172.21.15.45:0/1789987404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0144101710 0x7f0144198070 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:34.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 >> 172.21.15.45:0/1789987404 conn(0x7f01440fd180 msgr2=0x7f01440fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 shutdown_connections 2024-04-25T00:28:34.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:28:34.825+0000 7f0149f7e640 1 -- 172.21.15.45:0/1789987404 wait complete. 2024-04-25T00:28:34.836 INFO:tasks.cephfs.filesystem:scrub status for tag:af20edaf-4e84-43fb-bbd8-39c40c2668c5 - {'path': '/', 'tag': 'af20edaf-4e84-43fb-bbd8-39c40c2668c5', 'options': 'recursive,force'} 2024-04-25T00:28:35.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:35 smithi045 ceph-mon[29194]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:35 smithi122 ceph-mon[32959]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:35 smithi071 ceph-mon[32667]: pgmap v201: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:36 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:28:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:36 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:36 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:28:36.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:36 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:36.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:36 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:28:36.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:36 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:36.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:36 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:28:36.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:36 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:36 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:28:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:36 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:36 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:28:36.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:36 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:28:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:37 smithi122 ceph-mon[32959]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:37 smithi071 ceph-mon[32667]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:37.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:37 smithi045 ceph-mon[29194]: pgmap v202: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:38.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:38 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:28:38.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:38 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:28:38.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:38 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:28:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:39 smithi122 ceph-mon[32959]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:39 smithi071 ceph-mon[32667]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:39 smithi045 ceph-mon[29194]: pgmap v203: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:41 smithi122 ceph-mon[32959]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:41 smithi071 ceph-mon[32667]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:41 smithi045 ceph-mon[29194]: pgmap v204: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:43 smithi122 ceph-mon[32959]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:43 smithi071 ceph-mon[32667]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:43 smithi045 ceph-mon[29194]: pgmap v205: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:45 smithi122 ceph-mon[32959]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:28:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:45 smithi071 ceph-mon[32667]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:28:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:45 smithi045 ceph-mon[29194]: pgmap v206: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:28:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:47 smithi122 ceph-mon[32959]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:28:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:47 smithi071 ceph-mon[32667]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:28:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:47 smithi045 ceph-mon[29194]: pgmap v207: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:28:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:49 smithi122 ceph-mon[32959]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:49 smithi071 ceph-mon[32667]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:49 smithi045 ceph-mon[29194]: pgmap v208: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:28:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:51 smithi071 ceph-mon[32667]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:51 smithi045 ceph-mon[29194]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:51.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:51 smithi122 ceph-mon[32959]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:28:53.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:53 smithi071 ceph-mon[32667]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:53 smithi045 ceph-mon[29194]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:53 smithi122 ceph-mon[32959]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:55 smithi071 ceph-mon[32667]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:55 smithi045 ceph-mon[29194]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:55 smithi122 ceph-mon[32959]: pgmap v211: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:28:57.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:57 smithi071 ceph-mon[32667]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:57 smithi045 ceph-mon[29194]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:57.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:57 smithi122 ceph-mon[32959]: pgmap v212: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:28:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:28:59 smithi071 ceph-mon[32667]: pgmap v213: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:28:59 smithi045 ceph-mon[29194]: pgmap v213: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:28:59.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:28:59 smithi122 ceph-mon[32959]: pgmap v213: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:01 smithi071 ceph-mon[32667]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:01.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:01 smithi045 ceph-mon[29194]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:01.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:01 smithi122 ceph-mon[32959]: pgmap v214: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:03 smithi071 ceph-mon[32667]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:03 smithi045 ceph-mon[29194]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:03 smithi122 ceph-mon[32959]: pgmap v215: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:04.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:29:04.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 -- 172.21.15.45:0/1711765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5230100610 msgr2=0x7f5230100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 --2- 172.21.15.45:0/1711765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5230100610 0x7f5230100a90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5218009930 tx=0x7f521802a420 comp rx=0 tx=0).stop 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 -- 172.21.15.45:0/1711765260 shutdown_connections 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 --2- 172.21.15.45:0/1711765260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100fd0 0x7f523010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 --2- 172.21.15.45:0/1711765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5230100610 0x7f5230100a90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 --2- 172.21.15.45:0/1711765260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52301066b0 0x7f5230106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 -- 172.21.15.45:0/1711765260 >> 172.21.15.45:0/1711765260 conn(0x7f5230077740 msgr2=0x7f5230077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 -- 172.21.15.45:0/1711765260 shutdown_connections 2024-04-25T00:29:04.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.954+0000 7f5236d4f640 1 -- 172.21.15.45:0/1711765260 wait complete. 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 Processor -- start 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 -- start start 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100610 0x7f5230198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230100fd0 0x7f5230199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5230110a30 con 0x7f52301066b0 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52301108b0 con 0x7f5230100610 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5236d4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5230110bb0 con 0x7f5230100fd0 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35980/0 (socket says 172.21.15.45:35980) 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 -- 172.21.15.45:0/2777456950 learned_addr learned my addr 172.21.15.45:0/2777456950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f522ffff640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230100fd0 0x7f5230199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f5234aeb640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100610 0x7f5230198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 -- 172.21.15.45:0/2777456950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230100fd0 msgr2=0x7f5230199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230100fd0 0x7f5230199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 -- 172.21.15.45:0/2777456950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100610 msgr2=0x7f5230198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100610 0x7f5230198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 -- 172.21.15.45:0/2777456950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301a43f0 con 0x7f52301066b0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.955+0000 7f52352ec640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f522800eaa0 tx=0x7f522800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.956+0000 7f522dffb640 1 -- 172.21.15.45:0/2777456950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522800cdb0 con 0x7f52301066b0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.956+0000 7f522dffb640 1 -- 172.21.15.45:0/2777456950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228005020 con 0x7f52301066b0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.956+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301a45c0 con 0x7f52301066b0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.956+0000 7f522dffb640 1 -- 172.21.15.45:0/2777456950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280116b0 con 0x7f52301066b0 2024-04-25T00:29:04.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.956+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301a4ae0 con 0x7f52301066b0 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.958+0000 7f522dffb640 1 -- 172.21.15.45:0/2777456950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5228005560 con 0x7f52301066b0 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 Processor -- start 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522dffb640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 0x7f521c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 -- start start 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522dffb640 1 -- 172.21.15.45:0/2777456950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f52280cfa50 con 0x7f52301066b0 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522ffff640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 0x7f521c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc40e7d6b0 con 0x55bc40f18f50 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc40e25880 con 0x55bc40f18630 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5239a5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc40e79670 con 0x55bc40f23620 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35992/0 (socket says 172.21.15.45:35992) 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f522ffff640 1 -- 172.21.15.45:0/2360946157 learned_addr learned my addr 172.21.15.45:0/2360946157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.959+0000 7f5234aeb640 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f52352ec640 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 0x7f521c0a5240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5218005ab0 tx=0x7f5218032f60 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 -- 172.21.15.45:0/2360946157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 msgr2=0x55bc40f25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 -- 172.21.15.45:0/2360946157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 msgr2=0x55bc40f18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 -- 172.21.15.45:0/2360946157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc40c5b7d0 con 0x55bc40f18f50 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f522ffff640 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f521802a940 tx=0x7f5218047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff70b21681d27b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f5226ffd640 1 -- 172.21.15.45:0/2360946157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035050 con 0x55bc40f18f50 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f5226ffd640 1 -- 172.21.15.45:0/2360946157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521804e070 con 0x55bc40f18f50 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.960+0000 7f5226ffd640 1 -- 172.21.15.45:0/2360946157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52180035f0 con 0x55bc40f18f50 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- 172.21.15.45:0/2360946157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 msgr2=0x55bc40f230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f521802a940 tx=0x7f5218047000 comp rx=0 tx=0).stop 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- 172.21.15.45:0/2360946157 shutdown_connections 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f230e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 --2- 172.21.15.45:0/2360946157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- 172.21.15.45:0/2360946157 >> 172.21.15.45:0/2360946157 conn(0x55bc40f14950 msgr2=0x55bc40f14d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- 172.21.15.45:0/2360946157 shutdown_connections 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- 172.21.15.45:0/2360946157 wait complete. 2024-04-25T00:29:04.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 -- start start 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.961+0000 7f5239a5c740 1 Processor -- start 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f20560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f20aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc40f263b0 con 0x55bc40f23620 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc40e38f50 con 0x55bc40f18f50 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc40f26530 con 0x55bc40f18630 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 10 client.0 _ll_register_callbacks cb 0x55bc40e74000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36008/0 (socket says 172.21.15.45:36008) 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 -- 172.21.15.45:0/312385501 learned_addr learned my addr 172.21.15.45:0/312385501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5234aeb640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f20560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f522ffff640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f20aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 -- 172.21.15.45:0/312385501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 msgr2=0x55bc40f20560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f20560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 -- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 msgr2=0x55bc40f20aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f20aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 -- 172.21.15.45:0/312385501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc40b913f0 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f52352ec640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5228098070 tx=0x7f522809bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2101b55eea4e6df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f520ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522801f070 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f522809b060 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.962+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521804e070 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f522809b260 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 10 client.17607 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc40b510f0 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521804e070 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f522800f040 con 0x55bc40f23620 2024-04-25T00:29:04.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 10 client.17607 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 20 client.17607 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f33890 0x55bc40f35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 4 client.17607 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc40e8abf0 con 0x55bc40f33890 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.963+0000 7f5234aeb640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f33890 0x55bc40f35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.964+0000 7f520ffff640 10 client.17607 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.964+0000 7f5234aeb640 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f33890 0x55bc40f35c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d9c560eef769e server_cookie=846e234723e5dae8 in_seq=0 out_seq=0 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f520ffff640 1 -- 172.21.15.45:0/312385501 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc40e8abf0 con 0x55bc40f33890 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f520ffff640 10 client.17607 handle_command_reply: tid=0 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 client.17607 shutdown 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 msgr2=0x55bc40f20fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5228098070 tx=0x7f522809bd50 comp rx=0 tx=0).stop 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 shutdown_connections 2024-04-25T00:29:04.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f33890 0x55bc40f35c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f20fe0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f20aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 --2- 172.21.15.45:0/312385501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f20560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 >> 172.21.15.45:0/312385501 conn(0x55bc40f14950 msgr2=0x55bc40f13af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f520ffff640 0 client.17607 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 shutdown_connections 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 1 -- 172.21.15.45:0/312385501 wait complete. 2024-04-25T00:29:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.965+0000 7f5239a5c740 20 client.17607 trim_cache size 0 max 16384 2024-04-25T00:29:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 Processor -- start 2024-04-25T00:29:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 -- start start 2024-04-25T00:29:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f3b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52301aeb00 con 0x55bc40f18f50 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b0640 con 0x55bc40f18630 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5239a5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52301af1d0 con 0x55bc40f23620 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36010/0 (socket says 172.21.15.45:36010) 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 -- 172.21.15.45:0/808649041 learned_addr learned my addr 172.21.15.45:0/808649041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f52352ec640 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f3b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f5234aeb640 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 -- 172.21.15.45:0/808649041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 msgr2=0x55bc40f3b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f3b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 -- 172.21.15.45:0/808649041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 msgr2=0x55bc40f2f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f2f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.983+0000 7f522ffff640 1 -- 172.21.15.45:0/808649041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc40f36260 con 0x55bc40f18f50 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f522ffff640 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5218033000 tx=0x7f521802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275a40dc0f47ff42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f52267fc640 1 -- 172.21.15.45:0/808649041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218005240 con 0x55bc40f18f50 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f52267fc640 1 -- 172.21.15.45:0/808649041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521804e040 con 0x55bc40f18f50 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f52267fc640 1 -- 172.21.15.45:0/808649041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52180035f0 con 0x55bc40f18f50 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 -- 172.21.15.45:0/808649041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 msgr2=0x55bc40f20e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5218033000 tx=0x7f521802a420 comp rx=0 tx=0).stop 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 -- 172.21.15.45:0/808649041 shutdown_connections 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f23620 0x55bc40f3b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f18f50 0x55bc40f20e00 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 --2- 172.21.15.45:0/808649041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18630 0x55bc40f2f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.984+0000 7f5239a5c740 1 -- 172.21.15.45:0/808649041 >> 172.21.15.45:0/808649041 conn(0x55bc40f14950 msgr2=0x55bc40f22510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- 172.21.15.45:0/808649041 shutdown_connections 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- 172.21.15.45:0/808649041 wait complete. 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- start start 2024-04-25T00:29:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 Processor -- start 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f1cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52301aff70 con 0x55bc40f23620 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52301af8a0 con 0x55bc40f18f50 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52301acae0 con 0x55bc40f18630 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f5239a5c740 10 client.0 _ll_register_callbacks cb 0x7f5228097f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.985+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5234aeb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f1cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5234aeb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f2b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:44708/0 (socket says 172.21.15.45:44708) 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f52352ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36024/0 (socket says 172.21.15.45:36024) 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5234aeb640 1 -- 172.21.15.45:0/4081708581 learned_addr learned my addr 172.21.15.45:0/4081708581 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 -- 172.21.15.45:0/4081708581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 msgr2=0x55bc40f2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f2b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 -- 172.21.15.45:0/4081708581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 msgr2=0x55bc40f1d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 -- 172.21.15.45:0/4081708581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc40f1d8f0 con 0x55bc40f18f50 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f52352ec640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f522ffff640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f1cdf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55bc40f26060 tx=0x7f5218004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c854bbb14494605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f520ffff640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218063030 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521804e040 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f5218004ff0 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5218032cb0 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5239a5c740 10 client.26915 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.986+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc40f99b30 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521804e040 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5218003a70 con 0x55bc40f18f50 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 10 client.26915 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 20 client.26915 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f99e10 0x55bc40f9c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 4 client.26915 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:29:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc40e84f40 con 0x55bc40f99e10 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.987+0000 7f5234aeb640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f99e10 0x55bc40f9c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f520ffff640 10 client.26915 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5234aeb640 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f99e10 0x55bc40f9c200 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc457bee7abf2b3 server_cookie=4f641ae7acb59573 in_seq=0 out_seq=0 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f520ffff640 1 -- 172.21.15.45:0/4081708581 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc40e84f40 con 0x55bc40f99e10 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f520ffff640 10 client.26915 handle_command_reply: tid=0 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 client.26915 shutdown 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 msgr2=0x55bc40f1cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f1cdf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55bc40f26060 tx=0x7f5218004ab0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 shutdown_connections 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bc40f99e10 0x55bc40f9c200 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc40f23620 0x55bc40f1d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc40f18f50 0x55bc40f1cdf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 --2- 172.21.15.45:0/4081708581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc40f18630 0x55bc40f2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f520ffff640 0 client.26915 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 >> 172.21.15.45:0/4081708581 conn(0x55bc40f14950 msgr2=0x55bc40f13260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 shutdown_connections 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 1 -- 172.21.15.45:0/4081708581 wait complete. 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.988+0000 7f5239a5c740 20 client.26915 trim_cache size 0 max 16384 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:29:04.995 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 msgr2=0x7f521c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 0x7f521c0a5240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5218005ab0 tx=0x7f5218032f60 comp rx=0 tx=0).stop 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 msgr2=0x7f52301a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f522800eaa0 tx=0x7f522800ef60 comp rx=0 tx=0).stop 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 shutdown_connections 2024-04-25T00:29:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f521c0a2d80 0x7f521c0a5240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52301066b0 0x7f52301a3eb0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230100fd0 0x7f5230199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 --2- 172.21.15.45:0/2777456950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5230100610 0x7f5230198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 >> 172.21.15.45:0/2777456950 conn(0x7f5230077740 msgr2=0x7f52300ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 shutdown_connections 2024-04-25T00:29:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:04.989+0000 7f5236d4f640 1 -- 172.21.15.45:0/2777456950 wait complete. 2024-04-25T00:29:05.000 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:29:05.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:29:05.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 -- 172.21.15.45:0/2996863352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80100690 msgr2=0x7f3d80100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2996863352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80100690 0x7f3d80100b10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009930 tx=0x7f3d6802a420 comp rx=0 tx=0).stop 2024-04-25T00:29:05.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 -- 172.21.15.45:0/2996863352 shutdown_connections 2024-04-25T00:29:05.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2996863352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80101050 0x7f3d8010d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2996863352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80100690 0x7f3d80100b10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2996863352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d80106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 -- 172.21.15.45:0/2996863352 >> 172.21.15.45:0/2996863352 conn(0x7f3d800777a0 msgr2=0x7f3d80077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 -- 172.21.15.45:0/2996863352 shutdown_connections 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.103+0000 7f3d8584c640 1 -- 172.21.15.45:0/2996863352 wait complete. 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 Processor -- start 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 -- start start 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 0x7f3d80197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d8019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80111b60 con 0x7f3d80101050 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801119e0 con 0x7f3d80100690 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d8584c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80111ce0 con 0x7f3d80106650 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d77fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d77fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36050/0 (socket says 172.21.15.45:36050) 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d77fff640 1 -- 172.21.15.45:0/2051501224 learned_addr learned my addr 172.21.15.45:0/2051501224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d7effd640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 0x7f3d80197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.104+0000 7f3d7f7fe640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d8019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 -- 172.21.15.45:0/2051501224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 msgr2=0x7f3d8019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d8019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 -- 172.21.15.45:0/2051501224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 msgr2=0x7f3d80197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 0x7f3d80197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 -- 172.21.15.45:0/2051501224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8019cfd0 con 0x7f3d80101050 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d7f7fe640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d8019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d7effd640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 0x7f3d80197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d77fff640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009230 tx=0x7f3d68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6803f030 con 0x7f3d80101050 2024-04-25T00:29:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8019d260 con 0x7f3d80101050 2024-04-25T00:29:05.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d68003ad0 con 0x7f3d80101050 2024-04-25T00:29:05.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68035070 con 0x7f3d80101050 2024-04-25T00:29:05.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.105+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a4b50 con 0x7f3d80101050 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.106+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d800689b0 con 0x7f3d80101050 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.107+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3d68042030 con 0x7f3d80101050 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.108+0000 7f3d8484a640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 0x7f3d6c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.108+0000 7f3d7effd640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 0x7f3d6c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.108+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f3d680f0190 con 0x7f3d80101050 2024-04-25T00:29:05.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.108+0000 7f3d7effd640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 0x7f3d6c0a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005e10 tx=0x7f3d70005da0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.111+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3d680bc9f0 con 0x7f3d80101050 2024-04-25T00:29:05.149 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:05 smithi045 ceph-mon[29194]: pgmap v216: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-25T00:29:05.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.334+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d80108bd0 con 0x7f3d80101050 2024-04-25T00:29:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.335+0000 7f3d8484a640 1 -- 172.21.15.45:0/2051501224 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f3d680bc2f0 con 0x7f3d80101050 2024-04-25T00:29:05.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:29:05.337 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:29:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:29:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 msgr2=0x7f3d6c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 0x7f3d6c0a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005e10 tx=0x7f3d70005da0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 msgr2=0x7f3d80198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009230 tx=0x7f3d68004270 comp rx=0 tx=0).stop 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 shutdown_connections 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3d6c0a2d00 0x7f3d6c0a51c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d80106650 0x7f3d8019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d80101050 0x7f3d80198520 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 --2- 172.21.15.45:0/2051501224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d80100690 0x7f3d80197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 >> 172.21.15.45:0/2051501224 conn(0x7f3d800777a0 msgr2=0x7f3d801031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 shutdown_connections 2024-04-25T00:29:05.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.337+0000 7f3d8584c640 1 -- 172.21.15.45:0/2051501224 wait complete. 2024-04-25T00:29:05.349 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:29:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:05 smithi071 ceph-mon[32667]: pgmap v216: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 -- 172.21.15.45:0/2839037448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac107730 msgr2=0x7f48ac107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2839037448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac107730 0x7f48ac107b10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4894009930 tx=0x7f489402a420 comp rx=0 tx=0).stop 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 -- 172.21.15.45:0/2839037448 shutdown_connections 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2839037448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2839037448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac069dc0 0x7f48ac06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2839037448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac107730 0x7f48ac107b10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 -- 172.21.15.45:0/2839037448 >> 172.21.15.45:0/2839037448 conn(0x7f48ac0776a0 msgr2=0x7f48ac077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 -- 172.21.15.45:0/2839037448 shutdown_connections 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.453+0000 7f48b28ba640 1 -- 172.21.15.45:0/2839037448 wait complete. 2024-04-25T00:29:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 Processor -- start 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 -- start start 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac107730 0x7f48ac1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac111b00 con 0x7f48ac069dc0 2024-04-25T00:29:05.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac111980 con 0x7f48ac06a780 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48b28ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac111c80 con 0x7f48ac107730 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36054/0 (socket says 172.21.15.45:36054) 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48abfff640 1 -- 172.21.15.45:0/2809222904 learned_addr learned my addr 172.21.15.45:0/2809222904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.454+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48b0e57640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac107730 0x7f48ac1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 -- 172.21.15.45:0/2809222904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac107730 msgr2=0x7f48ac1010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac107730 0x7f48ac1010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 -- 172.21.15.45:0/2809222904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 msgr2=0x7f48ac100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 -- 172.21.15.45:0/2809222904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac1a6400 con 0x7f48ac069dc0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48abfff640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4894009230 tx=0x7f4894004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48a97fa640 1 -- 172.21.15.45:0/2809222904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489403f030 con 0x7f48ac069dc0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48a97fa640 1 -- 172.21.15.45:0/2809222904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894003ad0 con 0x7f48ac069dc0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac1a6690 con 0x7f48ac069dc0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48a97fa640 1 -- 172.21.15.45:0/2809222904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894035070 con 0x7f48ac069dc0 2024-04-25T00:29:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.455+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a6b80 con 0x7f48ac069dc0 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.457+0000 7f48a97fa640 1 -- 172.21.15.45:0/2809222904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4894042030 con 0x7f48ac069dc0 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 Processor -- start 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 -- start start 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588eb30a670 con 0x5588eb3b0570 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5588eb2b2970 con 0x5588eb3a5580 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b55c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588eb306630 con 0x5588eb3a5ea0 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48a97fa640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 0x7f48980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36056/0 (socket says 172.21.15.45:36056) 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48ab7fe640 1 -- 172.21.15.45:0/979422026 learned_addr learned my addr 172.21.15.45:0/979422026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.458+0000 7f48b0e57640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 0x7f48980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48b0e57640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48a97fa640 1 -- 172.21.15.45:0/2809222904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f48940efd20 con 0x7f48ac069dc0 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48abfff640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 -- 172.21.15.45:0/979422026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 msgr2=0x5588eb3b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 -- 172.21.15.45:0/979422026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 msgr2=0x5588eb3a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 -- 172.21.15.45:0/979422026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588eb2d6400 con 0x5588eb3b0570 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48b0e57640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48abfff640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f489c00f470 tx=0x7f489c00f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1acce7969bba9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48b0e57640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 0x7f48980a51c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f48a4005fd0 tx=0x7f48a4013040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48a27fc640 1 -- 172.21.15.45:0/979422026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c005220 con 0x5588eb3b0570 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48a27fc640 1 -- 172.21.15.45:0/979422026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489c004c80 con 0x5588eb3b0570 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.459+0000 7f48a27fc640 1 -- 172.21.15.45:0/979422026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c010d40 con 0x5588eb3b0570 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- 172.21.15.45:0/979422026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 msgr2=0x5588eb3b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f489c00f470 tx=0x7f489c00f930 comp rx=0 tx=0).stop 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- 172.21.15.45:0/979422026 shutdown_connections 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3b2960 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 --2- 172.21.15.45:0/979422026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- 172.21.15.45:0/979422026 >> 172.21.15.45:0/979422026 conn(0x5588eb3a18a0 msgr2=0x5588eb3a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- 172.21.15.45:0/979422026 shutdown_connections 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- 172.21.15.45:0/979422026 wait complete. 2024-04-25T00:29:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 -- start start 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.460+0000 7f48b55c8740 1 Processor -- start 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3b0570 0x5588eb3ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588eb3b3360 con 0x5588eb3a5ea0 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5588eb2c6050 con 0x5588eb3b0570 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588eb3b34e0 con 0x5588eb3a5580 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 10 client.0 _ll_register_callbacks cb 0x5588eb01dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36068/0 (socket says 172.21.15.45:36068) 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 -- 172.21.15.45:0/4170773391 learned_addr learned my addr 172.21.15.45:0/4170773391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b0e57640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3b0570 0x5588eb3ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48abfff640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 -- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 msgr2=0x5588eb3afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 -- 172.21.15.45:0/4170773391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3b0570 msgr2=0x5588eb3ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3b0570 0x5588eb3ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 -- 172.21.15.45:0/4170773391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588eb297dc0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48abfff640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f489c000c00 tx=0x7f489c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61b4a61daf71270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f488b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c027070 con 0x5588eb3a5ea0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f489c005000 con 0x5588eb3a5ea0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.461+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f489c004c80 con 0x5588eb3a5ea0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f489c033050 con 0x5588eb3a5ea0 2024-04-25T00:29:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 10 client.17631 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588eb01e3f0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489c004c80 con 0x5588eb3a5ea0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f489c018ad0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 10 client.17631 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 20 client.17631 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb3c03f0 0x5588eb3c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 4 client.17631 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588eb317bb0 con 0x5588eb3c03f0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.462+0000 7f48abfff640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb3c03f0 0x5588eb3c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.463+0000 7f48abfff640 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb3c03f0 0x5588eb3c27e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50f1afd612d8af7c server_cookie=f89081964c194a1b in_seq=0 out_seq=0 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.463+0000 7f488b7fe640 10 client.17631 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.463+0000 7f488b7fe640 1 -- 172.21.15.45:0/4170773391 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588eb317bb0 con 0x5588eb3c03f0 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.463+0000 7f488b7fe640 10 client.17631 handle_command_reply: tid=0 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.463+0000 7f48b55c8740 1 client.17631 shutdown 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 msgr2=0x5588eb3ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f489c000c00 tx=0x7f489c010c70 comp rx=0 tx=0).stop 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 shutdown_connections 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb3c03f0 0x5588eb3c27e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3b0570 0x5588eb3ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f488b7fe640 0 client.17631 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3ad140 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 --2- 172.21.15.45:0/4170773391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 >> 172.21.15.45:0/4170773391 conn(0x5588eb3a18a0 msgr2=0x5588eb3a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 shutdown_connections 2024-04-25T00:29:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 1 -- 172.21.15.45:0/4170773391 wait complete. 2024-04-25T00:29:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.464+0000 7f48b55c8740 20 client.17631 trim_cache size 0 max 16384 2024-04-25T00:29:05.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 Processor -- start 2024-04-25T00:29:05.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 -- start start 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 0x5588eb3c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1b0c30 con 0x5588eb3b0570 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1b2770 con 0x5588eb3a5ea0 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b55c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1b1300 con 0x5588eb3a5580 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36076/0 (socket says 172.21.15.45:36076) 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 -- 172.21.15.45:0/1759521705 learned_addr learned my addr 172.21.15.45:0/1759521705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48abfff640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 0x5588eb3c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 -- 172.21.15.45:0/1759521705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 msgr2=0x5588eb3bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 -- 172.21.15.45:0/1759521705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 msgr2=0x5588eb3c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 0x5588eb3c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.477+0000 7f48b0e57640 1 -- 172.21.15.45:0/1759521705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588eb0e87d0 con 0x5588eb3b0570 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 0x5588eb3c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b0e57640 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f48a4003e70 tx=0x7f48a4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4841f2f0a5dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48a1ffb640 1 -- 172.21.15.45:0/1759521705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a40046b0 con 0x5588eb3b0570 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48a1ffb640 1 -- 172.21.15.45:0/1759521705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a4004850 con 0x5588eb3b0570 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48a1ffb640 1 -- 172.21.15.45:0/1759521705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4005890 con 0x5588eb3b0570 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 -- 172.21.15.45:0/1759521705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 msgr2=0x5588eb3c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f48a4003e70 tx=0x7f48a4009500 comp rx=0 tx=0).stop 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 -- 172.21.15.45:0/1759521705 shutdown_connections 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3b0570 0x5588eb3c8380 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5ea0 0x5588eb3c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1759521705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3a5580 0x5588eb3bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 -- 172.21.15.45:0/1759521705 >> 172.21.15.45:0/1759521705 conn(0x5588eb3a18a0 msgr2=0x5588eb3a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 -- 172.21.15.45:0/1759521705 shutdown_connections 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.478+0000 7f48b55c8740 1 -- 172.21.15.45:0/1759521705 wait complete. 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 -- start start 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 Processor -- start 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3b0570 0x5588eb3b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1b20a0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1b19d0 con 0x5588eb3a5580 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac1aec10 con 0x5588eb3b0570 2024-04-25T00:29:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b55c8740 10 client.0 _ll_register_callbacks cb 0x7f489c00aeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36092/0 (socket says 172.21.15.45:36092) 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48ab7fe640 1 -- 172.21.15.45:0/1178674833 learned_addr learned my addr 172.21.15.45:0/1178674833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48abfff640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.479+0000 7f48b0e57640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3b0570 0x5588eb3b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 -- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3b0570 msgr2=0x5588eb3b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3b0570 0x5588eb3b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 -- 172.21.15.45:0/1178674833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 msgr2=0x5588eb3b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 -- 172.21.15.45:0/1178674833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588eb4258c0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48ab7fe640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5588eb3ae710 tx=0x7f489c023550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399b9ebb49385eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c03b020 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48a4004850 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f489c0049d0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f489c046050 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 10 client.17637 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588eb426b10 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a4004850 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f489c023cd0 con 0x5588eb3a5ea0 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 10 client.17637 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 10 client.17637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 20 client.17637 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.480+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb427280 0x5588eb4296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 4 client.17637 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588eb311f00 con 0x5588eb427280 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48abfff640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb427280 0x5588eb4296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48abfff640 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb427280 0x5588eb4296a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2264926f6828c24 server_cookie=5d8f74b9743410b0 in_seq=0 out_seq=0 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f488b7fe640 10 client.17637 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f488b7fe640 1 -- 172.21.15.45:0/1178674833 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588eb311f00 con 0x5588eb427280 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f488b7fe640 10 client.17637 handle_command_reply: tid=0 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 client.17637 shutdown 2024-04-25T00:29:05.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 msgr2=0x5588eb3b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x5588eb3ae710 tx=0x7f489c023550 comp rx=0 tx=0).stop 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 shutdown_connections 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.481+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5588eb427280 0x5588eb4296a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588eb3b0570 0x5588eb3b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588eb3a5ea0 0x5588eb3b6890 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f488b7fe640 0 client.17637 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 --2- 172.21.15.45:0/1178674833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588eb3a5580 0x5588eb3b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 >> 172.21.15.45:0/1178674833 conn(0x5588eb3a18a0 msgr2=0x5588eb3bb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 shutdown_connections 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 1 -- 172.21.15.45:0/1178674833 wait complete. 2024-04-25T00:29:05.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b55c8740 20 client.17637 trim_cache size 0 max 16384 2024-04-25T00:29:05.491 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 msgr2=0x7f48980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 0x7f48980a51c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f48a4005fd0 tx=0x7f48a4013040 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 msgr2=0x7f48ac100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4894009230 tx=0x7f4894004270 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 shutdown_connections 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f48980a2d00 0x7f48980a51c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48ac107730 0x7f48ac1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac06a780 0x7f48ac100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 --2- 172.21.15.45:0/2809222904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48ac069dc0 0x7f48ac100670 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 >> 172.21.15.45:0/2809222904 conn(0x7f48ac0776a0 msgr2=0x7f48ac0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 shutdown_connections 2024-04-25T00:29:05.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.482+0000 7f48b28ba640 1 -- 172.21.15.45:0/2809222904 wait complete. 2024-04-25T00:29:05.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:29:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:05 smithi122 ceph-mon[32959]: pgmap v216: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 -- 172.21.15.45:0/2895163467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701029a0 msgr2=0x7f6270102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 --2- 172.21.15.45:0/2895163467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701029a0 0x7f6270102d80 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6260009900 tx=0x7f626002a480 comp rx=0 tx=0).stop 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 -- 172.21.15.45:0/2895163467 shutdown_connections 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 --2- 172.21.15.45:0/2895163467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 --2- 172.21.15.45:0/2895163467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701032c0 0x7f627010b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 --2- 172.21.15.45:0/2895163467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701029a0 0x7f6270102d80 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 -- 172.21.15.45:0/2895163467 >> 172.21.15.45:0/2895163467 conn(0x7f62700769c0 msgr2=0x7f6270076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 -- 172.21.15.45:0/2895163467 shutdown_connections 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.597+0000 7f6274de9640 1 -- 172.21.15.45:0/2895163467 wait complete. 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 Processor -- start 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 -- start start 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 0x7f6270198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6270111b00 con 0x7f62701032c0 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6270111980 con 0x7f62701029a0 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f6274de9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6270111c80 con 0x7f627010bd70 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36106/0 (socket says 172.21.15.45:36106) 2024-04-25T00:29:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f626dd9b640 1 -- 172.21.15.45:0/3723245367 learned_addr learned my addr 172.21.15.45:0/3723245367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f626ed9d640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.598+0000 7f626e59c640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 0x7f6270198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 -- 172.21.15.45:0/3723245367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 msgr2=0x7f627019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 -- 172.21.15.45:0/3723245367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 msgr2=0x7f6270198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 0x7f6270198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 -- 172.21.15.45:0/3723245367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627019d030 con 0x7f62701032c0 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626ed9d640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626e59c640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 0x7f6270198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f626dd9b640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f625f7fe640 1 -- 172.21.15.45:0/3723245367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264017070 con 0x7f62701032c0 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627019d320 con 0x7f62701032c0 2024-04-25T00:29:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f625f7fe640 1 -- 172.21.15.45:0/3723245367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264009c30 con 0x7f62701032c0 2024-04-25T00:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f625f7fe640 1 -- 172.21.15.45:0/3723245367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626400f6b0 con 0x7f62701032c0 2024-04-25T00:29:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.599+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a4bb0 con 0x7f62701032c0 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.601+0000 7f6277bcb740 1 Processor -- start 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 -- start start 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f625f7fe640 1 -- 172.21.15.45:0/3723245367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f62640109d0 con 0x7f62701032c0 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x56525780a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x56525781bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56525776f670 con 0x56525780ada0 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x565257717970 con 0x565257819980 2024-04-25T00:29:05.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f6277bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56525776b630 con 0x56525780a480 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36120/0 (socket says 172.21.15.45:36120) 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 -- 172.21.15.45:0/1585994637 learned_addr learned my addr 172.21.15.45:0/1585994637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626dd9b640 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x56525781bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626ed9d640 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x56525780a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 -- 172.21.15.45:0/1585994637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 msgr2=0x56525780a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x56525780a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 -- 172.21.15.45:0/1585994637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 msgr2=0x56525781bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x56525781bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 -- 172.21.15.45:0/1585994637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525773b400 con 0x56525780ada0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f625f7fe640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 0x7f62580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.602+0000 7f626e59c640 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6260033000 tx=0x7f6260032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891c8bc93dcc66e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f626ed9d640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 0x7f62580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f625f7fe640 1 -- 172.21.15.45:0/3723245367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f62640cf290 con 0x7f62701032c0 2024-04-25T00:29:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f625cff9640 1 -- 172.21.15.45:0/1585994637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626003f070 con 0x56525780ada0 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f625cff9640 1 -- 172.21.15.45:0/1585994637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62600317b0 con 0x56525780ada0 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f625cff9640 1 -- 172.21.15.45:0/1585994637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260035070 con 0x56525780ada0 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f6277bcb740 1 -- 172.21.15.45:0/1585994637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 msgr2=0x565257819440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f626ed9d640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 0x7f62580a51c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f627006ae40 tx=0x7f62680098d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.603+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6260033000 tx=0x7f6260032cb0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- 172.21.15.45:0/1585994637 shutdown_connections 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x56525781bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257819440 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1585994637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x56525780a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- 172.21.15.45:0/1585994637 >> 172.21.15.45:0/1585994637 conn(0x5652578068a0 msgr2=0x565257806c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- 172.21.15.45:0/1585994637 shutdown_connections 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- 172.21.15.45:0/1585994637 wait complete. 2024-04-25T00:29:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- start start 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 Processor -- start 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257819250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 0x565257816540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56525781c6e0 con 0x565257819980 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56525772b050 con 0x56525780ada0 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56525781c860 con 0x56525780a480 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f6277bcb740 10 client.0 _ll_register_callbacks cb 0x565257482be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f626ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f626ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36128/0 (socket says 172.21.15.45:36128) 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.604+0000 7f626ed9d640 1 -- 172.21.15.45:0/2531387420 learned_addr learned my addr 172.21.15.45:0/2531387420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626dd9b640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 0x565257816540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626e59c640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257819250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 -- 172.21.15.45:0/2531387420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 msgr2=0x565257819250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257819250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 -- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 msgr2=0x565257816540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 0x565257816540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 -- 172.21.15.45:0/2531387420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652576fcdc0 con 0x565257819980 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626dd9b640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 0x565257816540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f626ed9d640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f626801d9a0 tx=0x7f6268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57003b7d4483ca9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268010980 con 0x565257819980 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f6268010b20 con 0x565257819980 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62600317b0 con 0x565257819980 2024-04-25T00:29:05.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6268010d20 con 0x565257819980 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6277bcb740 10 client.17649 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652574833f0 con 0x565257819980 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.605+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62600317b0 con 0x565257819980 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6268024070 con 0x565257819980 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 10 client.17649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 20 client.17649 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x56525782be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 4 client.17649 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56525777cbb0 con 0x5652578299a0 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f626e59c640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x56525782be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f626e59c640 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x56525782be20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3544d1b51d34f742 server_cookie=3a47524057b58739 in_seq=0 out_seq=0 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.606+0000 7f6255ffb640 10 client.17649 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6255ffb640 1 -- 172.21.15.45:0/2531387420 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56525777cbb0 con 0x5652578299a0 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6255ffb640 10 client.17649 handle_command_reply: tid=0 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 client.17649 shutdown 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 msgr2=0x565257816a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f626801d9a0 tx=0x7f6268004a40 comp rx=0 tx=0).stop 2024-04-25T00:29:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 shutdown_connections 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x56525782be20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6255ffb640 0 client.17649 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565257819980 0x565257816a80 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780ada0 0x565257816540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 --2- 172.21.15.45:0/2531387420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257819250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 >> 172.21.15.45:0/2531387420 conn(0x5652578068a0 msgr2=0x56525780c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 shutdown_connections 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 1 -- 172.21.15.45:0/2531387420 wait complete. 2024-04-25T00:29:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.607+0000 7f6277bcb740 20 client.17649 trim_cache size 0 max 16384 2024-04-25T00:29:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 Processor -- start 2024-04-25T00:29:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 -- start start 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 0x5652578255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565257819980 0x565257818d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62701ac9c0 con 0x56525780ada0 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62701ae500 con 0x56525780a480 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f6277bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62701ad090 con 0x565257819980 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36132/0 (socket says 172.21.15.45:36132) 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 -- 172.21.15.45:0/1812364238 learned_addr learned my addr 172.21.15.45:0/1812364238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626ed9d640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565257819980 0x565257818d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626e59c640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 0x5652578255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 -- 172.21.15.45:0/1812364238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565257819980 msgr2=0x565257818d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565257819980 0x565257818d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 -- 172.21.15.45:0/1812364238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 msgr2=0x5652578255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 0x5652578255e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 -- 172.21.15.45:0/1812364238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525754d7d0 con 0x56525780ada0 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626e59c640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 0x5652578255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.621+0000 7f626dd9b640 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f62640ccc80 tx=0x7f626400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58387cca689b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6257fff640 1 -- 172.21.15.45:0/1812364238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626409adb0 con 0x56525780ada0 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6257fff640 1 -- 172.21.15.45:0/1812364238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264009de0 con 0x56525780ada0 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6257fff640 1 -- 172.21.15.45:0/1812364238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264097ac0 con 0x56525780ada0 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- 172.21.15.45:0/1812364238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 msgr2=0x565257811af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f62640ccc80 tx=0x7f626400df40 comp rx=0 tx=0).stop 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- 172.21.15.45:0/1812364238 shutdown_connections 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565257819980 0x565257818d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257811af0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 --2- 172.21.15.45:0/1812364238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56525780a480 0x5652578255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- 172.21.15.45:0/1812364238 >> 172.21.15.45:0/1812364238 conn(0x5652578068a0 msgr2=0x5652578113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- 172.21.15.45:0/1812364238 shutdown_connections 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- 172.21.15.45:0/1812364238 wait complete. 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.622+0000 7f6277bcb740 1 -- start start 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 Processor -- start 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257821de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x565257813410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62701ade30 con 0x56525780ada0 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62701ad760 con 0x565257819980 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62701aa9a0 con 0x56525780a480 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 10 client.0 _ll_register_callbacks cb 0x7f62680096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36142/0 (socket says 172.21.15.45:36142) 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 -- 172.21.15.45:0/881051071 learned_addr learned my addr 172.21.15.45:0/881051071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626ed9d640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x565257813410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626e59c640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257821de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 -- 172.21.15.45:0/881051071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 msgr2=0x565257821de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257821de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 -- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 msgr2=0x565257813410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x565257813410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 -- 172.21.15.45:0/881051071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565257813b50 con 0x56525780ada0 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f626dd9b640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6264097e20 tx=0x7f6264099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251c120d975ac9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62640b3060 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f62640a3c30 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.623+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6264009de0 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f626409b450 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 10 client.17655 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56525788fe40 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264009de0 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f62640b4250 con 0x56525780ada0 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 10 client.17655 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 10 client.17655 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 20 client.17655 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x565257892470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 4 client.17655 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565257776f00 con 0x5652578299a0 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.624+0000 7f626e59c640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x565257892470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6255ffb640 10 client.17655 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f626e59c640 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x565257892470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5646ba53a1cbf5bb server_cookie=da9459da18e0395c in_seq=0 out_seq=0 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6255ffb640 1 -- 172.21.15.45:0/881051071 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565257776f00 con 0x5652578299a0 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6255ffb640 10 client.17655 handle_command_reply: tid=0 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 client.17655 shutdown 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 msgr2=0x565257812ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6264097e20 tx=0x7f6264099960 comp rx=0 tx=0).stop 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 shutdown_connections 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5652578299a0 0x565257892470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565257819980 0x565257813410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6255ffb640 0 client.17655 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56525780ada0 0x565257812ed0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 --2- 172.21.15.45:0/881051071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56525780a480 0x565257821de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 >> 172.21.15.45:0/881051071 conn(0x5652578068a0 msgr2=0x56525780bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 shutdown_connections 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 1 -- 172.21.15.45:0/881051071 wait complete. 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6277bcb740 20 client.17655 trim_cache size 0 max 16384 2024-04-25T00:29:05.636 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 msgr2=0x7f62580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 0x7f62580a51c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f627006ae40 tx=0x7f62680098d0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 msgr2=0x7f6270198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.625+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f626400d950 tx=0x7f626400de10 comp rx=0 tx=0).stop 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 shutdown_connections 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f62580a2d00 0x7f62580a51c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f627010bd70 0x7f627019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62701032c0 0x7f6270198580 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 --2- 172.21.15.45:0/3723245367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62701029a0 0x7f6270198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 >> 172.21.15.45:0/3723245367 conn(0x7f62700769c0 msgr2=0x7f62700fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 shutdown_connections 2024-04-25T00:29:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.626+0000 7f6274de9640 1 -- 172.21.15.45:0/3723245367 wait complete. 2024-04-25T00:29:05.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- 172.21.15.45:0/1538839733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1874069dc0 msgr2=0x7f187406a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1538839733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1874069dc0 0x7f187406a240 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1860009930 tx=0x7f186002a420 comp rx=0 tx=0).stop 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- 172.21.15.45:0/1538839733 shutdown_connections 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1538839733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187406a780 0x7f187410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1538839733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1874069dc0 0x7f187406a240 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1538839733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f1874107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- 172.21.15.45:0/1538839733 >> 172.21.15.45:0/1538839733 conn(0x7f18740776a0 msgr2=0x7f1874077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- 172.21.15.45:0/1538839733 shutdown_connections 2024-04-25T00:29:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- 172.21.15.45:0/1538839733 wait complete. 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 Processor -- start 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.742+0000 7f187ac8c640 1 -- start start 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1874069dc0 0x7f1874198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f187419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111b40 con 0x7f187406a780 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18741119c0 con 0x7f1874107730 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f187ac8c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1874111cc0 con 0x7f1874069dc0 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36150/0 (socket says 172.21.15.45:36150) 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 -- 172.21.15.45:0/1463530282 learned_addr learned my addr 172.21.15.45:0/1463530282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1879229640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f187419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1878a28640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1874069dc0 0x7f1874198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 -- 172.21.15.45:0/1463530282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1874069dc0 msgr2=0x7f1874198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1874069dc0 0x7f1874198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 -- 172.21.15.45:0/1463530282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 msgr2=0x7f187419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f187419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 -- 172.21.15.45:0/1463530282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187419d000 con 0x7f187406a780 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1879229640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f187419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1873fff640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1860009230 tx=0x7f1860004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.743+0000 7f1871ffb640 1 -- 172.21.15.45:0/1463530282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186003f030 con 0x7f187406a780 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.744+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187419d290 con 0x7f187406a780 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.744+0000 7f1871ffb640 1 -- 172.21.15.45:0/1463530282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1860003ad0 con 0x7f187406a780 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.744+0000 7f1871ffb640 1 -- 172.21.15.45:0/1463530282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860035070 con 0x7f187406a780 2024-04-25T00:29:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.744+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a4b80 con 0x7f187406a780 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.746+0000 7f1871ffb640 1 -- 172.21.15.45:0/1463530282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1860042030 con 0x7f187406a780 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.746+0000 7f187d999740 1 Processor -- start 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.746+0000 7f187d999740 1 -- start start 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cf9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf2c4f670 con 0x555bf2cf9980 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555bf2bf7970 con 0x555bf2cea480 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f187d999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555bf2c4b630 con 0x555bf2ceada0 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1871ffb640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 0x7f18640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1879229640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 0x7f18640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36160/0 (socket says 172.21.15.45:36160) 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 -- 172.21.15.45:0/482858855 learned_addr learned my addr 172.21.15.45:0/482858855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1879229640 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1878a28640 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cf9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1871ffb640 1 -- 172.21.15.45:0/1463530282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f18600efd00 con 0x7f187406a780 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 -- 172.21.15.45:0/482858855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 msgr2=0x555bf2cf9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cf9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 -- 172.21.15.45:0/482858855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 msgr2=0x555bf2cea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 -- 172.21.15.45:0/482858855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf2c1b400 con 0x555bf2cf9980 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1879229640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 0x7f18640a51c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f186c005fd0 tx=0x7f186c0125b0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f1873fff640 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f18600b8830 tx=0x7f186008de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e46b5d40faaa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.747+0000 7f185effd640 1 -- 172.21.15.45:0/482858855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860049040 con 0x555bf2cf9980 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f185effd640 1 -- 172.21.15.45:0/482858855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18600c0210 con 0x555bf2cf9980 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f185effd640 1 -- 172.21.15.45:0/482858855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600b84d0 con 0x555bf2cf9980 2024-04-25T00:29:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 -- 172.21.15.45:0/482858855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 msgr2=0x555bf2cfbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f18600b8830 tx=0x7f186008de20 comp rx=0 tx=0).stop 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 -- 172.21.15.45:0/482858855 shutdown_connections 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfbd70 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cf9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 --2- 172.21.15.45:0/482858855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 -- 172.21.15.45:0/482858855 >> 172.21.15.45:0/482858855 conn(0x555bf2ce68a0 msgr2=0x555bf2ce6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 -- 172.21.15.45:0/482858855 shutdown_connections 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.748+0000 7f187d999740 1 -- 172.21.15.45:0/482858855 wait complete. 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 -- start start 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 Processor -- start 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2cf6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf2cfc500 con 0x555bf2cf9980 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555bf2c0b050 con 0x555bf2ceada0 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555bf2cfc680 con 0x555bf2cea480 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f187d999740 10 client.0 _ll_register_callbacks cb 0x555bf2962be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36164/0 (socket says 172.21.15.45:36164) 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 -- 172.21.15.45:0/205814110 learned_addr learned my addr 172.21.15.45:0/205814110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1873fff640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2cf6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1878a28640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 -- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 msgr2=0x555bf2cf9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 -- 172.21.15.45:0/205814110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 msgr2=0x555bf2cf6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2cf6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.749+0000 7f1879229640 1 -- 172.21.15.45:0/205814110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf2bdcdc0 con 0x555bf2cf9980 2024-04-25T00:29:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1879229640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f186c01e630 tx=0x7f186c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4577e9a671774ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c020790 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f186c020930 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18600c0210 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f186c020b30 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 10 client.17667 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bf29633f0 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18600c0210 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f186c01f070 con 0x555bf2cf9980 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 10 client.17667 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 10 client.17667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 20 client.17667 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d09720 0x555bf2d0bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 4 client.17667 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.750+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf2c5cbb0 con 0x555bf2d09720 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.751+0000 7f1878a28640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d09720 0x555bf2d0bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.751+0000 7f1878a28640 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d09720 0x555bf2d0bb30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ba9269eada8f566 server_cookie=fe657e7f7917306 in_seq=0 out_seq=0 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.751+0000 7f1853fff640 10 client.17667 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f1853fff640 1 -- 172.21.15.45:0/205814110 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bf2c5cbb0 con 0x555bf2d09720 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f1853fff640 10 client.17667 handle_command_reply: tid=0 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 client.17667 shutdown 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 msgr2=0x555bf2cf6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f186c01e630 tx=0x7f186c01c000 comp rx=0 tx=0).stop 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 shutdown_connections 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d09720 0x555bf2d0bb30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf6ab0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2cf6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f1853fff640 0 client.17667 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 --2- 172.21.15.45:0/205814110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 >> 172.21.15.45:0/205814110 conn(0x555bf2ce68a0 msgr2=0x555bf2cec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 shutdown_connections 2024-04-25T00:29:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 1 -- 172.21.15.45:0/205814110 wait complete. 2024-04-25T00:29:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.752+0000 7f187d999740 20 client.17667 trim_cache size 0 max 16384 2024-04-25T00:29:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.765+0000 7f187d999740 1 Processor -- start 2024-04-25T00:29:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 -- start start 2024-04-25T00:29:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2ce59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ac8d0 con 0x555bf2cf9980 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ae410 con 0x555bf2ceada0 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f187d999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18741acfa0 con 0x555bf2cea480 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36184/0 (socket says 172.21.15.45:36184) 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 -- 172.21.15.45:0/211282572 learned_addr learned my addr 172.21.15.45:0/211282572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1878a28640 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1873fff640 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2ce59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 -- 172.21.15.45:0/211282572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 msgr2=0x555bf2cf56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 -- 172.21.15.45:0/211282572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 msgr2=0x555bf2ce59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2ce59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 -- 172.21.15.45:0/211282572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf2cf2150 con 0x555bf2cf9980 2024-04-25T00:29:05.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f1879229640 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f186c010cd0 tx=0x7f186c0129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f666b9f9f0059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.766+0000 7f185e7fc640 1 -- 172.21.15.45:0/211282572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c03e030 con 0x555bf2cf9980 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f185e7fc640 1 -- 172.21.15.45:0/211282572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c012cb0 con 0x555bf2cf9980 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f185e7fc640 1 -- 172.21.15.45:0/211282572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c020790 con 0x555bf2cf9980 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- 172.21.15.45:0/211282572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 msgr2=0x555bf2cf8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f186c010cd0 tx=0x7f186c0129f0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- 172.21.15.45:0/211282572 shutdown_connections 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cf8db0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2ceada0 0x555bf2ce59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 --2- 172.21.15.45:0/211282572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2cea480 0x555bf2cf56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- 172.21.15.45:0/211282572 >> 172.21.15.45:0/211282572 conn(0x555bf2ce68a0 msgr2=0x555bf2ce52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- 172.21.15.45:0/211282572 shutdown_connections 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- 172.21.15.45:0/211282572 wait complete. 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.767+0000 7f187d999740 1 -- start start 2024-04-25T00:29:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 Processor -- start 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cfd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cfda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18741add40 con 0x555bf2cf9980 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ad670 con 0x555bf2cea480 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18741aa8b0 con 0x555bf2ceada0 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f187d999740 10 client.0 _ll_register_callbacks cb 0x7f186c0127c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36192/0 (socket says 172.21.15.45:36192) 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 -- 172.21.15.45:0/192859946 learned_addr learned my addr 172.21.15.45:0/192859946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1878a28640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cfd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1873fff640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cfda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 -- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 msgr2=0x555bf2cfda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cfda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 -- 172.21.15.45:0/192859946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 msgr2=0x555bf2cfd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cfd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.768+0000 7f1879229640 1 -- 172.21.15.45:0/192859946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf2a2d7d0 con 0x555bf2cf9980 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1879229640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f186c02ae30 tx=0x7f186c0121b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f724d051c741fd59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c03e040 con 0x555bf2cf9980 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c012cb0 con 0x555bf2cf9980 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f186c020790 con 0x555bf2cf9980 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f186c020990 con 0x555bf2cf9980 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 10 client.17673 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bf2d6ff10 con 0x555bf2cf9980 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c012cb0 con 0x555bf2cf9980 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f186c00f040 con 0x555bf2cf9980 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 10 client.17673 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 10 client.17673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 20 client.17673 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d70330 0x555bf2d72720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 4 client.17673 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.769+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bf2cf6e40 con 0x555bf2d70330 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1878a28640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d70330 0x555bf2d72720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1878a28640 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d70330 0x555bf2d72720 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4551189a09093837 server_cookie=d087ccc6450e2b45 in_seq=0 out_seq=0 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1853fff640 10 client.17673 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1853fff640 1 -- 172.21.15.45:0/192859946 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bf2cf6e40 con 0x555bf2d70330 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1853fff640 10 client.17673 handle_command_reply: tid=0 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 client.17673 shutdown 2024-04-25T00:29:05.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 msgr2=0x555bf2cfdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f186c02ae30 tx=0x7f186c0121b0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 shutdown_connections 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x555bf2d70330 0x555bf2d72720 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf2cf9980 0x555bf2cfdf40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555bf2ceada0 0x555bf2cfda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f1853fff640 0 client.17673 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 --2- 172.21.15.45:0/192859946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bf2cea480 0x555bf2cfd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 >> 172.21.15.45:0/192859946 conn(0x555bf2ce68a0 msgr2=0x555bf2ce5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.770+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 shutdown_connections 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187d999740 1 -- 172.21.15.45:0/192859946 wait complete. 2024-04-25T00:29:05.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187d999740 20 client.17673 trim_cache size 0 max 16384 2024-04-25T00:29:05.779 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 msgr2=0x7f18640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 0x7f18640a51c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f186c005fd0 tx=0x7f186c0125b0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 msgr2=0x7f18741985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1860009230 tx=0x7f1860004270 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 shutdown_connections 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18640a2d00 0x7f18640a51c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1874107730 0x7f187419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f187406a780 0x7f18741985a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 --2- 172.21.15.45:0/1463530282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1874069dc0 0x7f1874198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 >> 172.21.15.45:0/1463530282 conn(0x7f18740776a0 msgr2=0x7f18740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 shutdown_connections 2024-04-25T00:29:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.771+0000 7f187ac8c640 1 -- 172.21.15.45:0/1463530282 wait complete. 2024-04-25T00:29:05.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:29:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 -- 172.21.15.45:0/2967026046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff110107720 msgr2=0x7ff110107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 --2- 172.21.15.45:0/2967026046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff110107720 0x7ff110107b00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff100009930 tx=0x7ff10002a420 comp rx=0 tx=0).stop 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 -- 172.21.15.45:0/2967026046 shutdown_connections 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 --2- 172.21.15.45:0/2967026046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1101020d0 0x7ff11010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 --2- 172.21.15.45:0/2967026046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110101710 0x7ff110101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 --2- 172.21.15.45:0/2967026046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff110107720 0x7ff110107b00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 -- 172.21.15.45:0/2967026046 >> 172.21.15.45:0/2967026046 conn(0x7ff1100fd180 msgr2=0x7ff1100ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 -- 172.21.15.45:0/2967026046 shutdown_connections 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.886+0000 7ff1155ce640 1 -- 172.21.15.45:0/2967026046 wait complete. 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 Processor -- start 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 -- start start 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 0x7ff110198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 0x7ff11019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff110111b30 con 0x7ff1101020d0 2024-04-25T00:29:05.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101119b0 con 0x7ff110107720 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff1155ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff110111cb0 con 0x7ff110101710 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36220/0 (socket says 172.21.15.45:36220) 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff107fff640 1 -- 172.21.15.45:0/342841955 learned_addr learned my addr 172.21.15.45:0/342841955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff10effd640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 0x7ff110198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.887+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 0x7ff11019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 -- 172.21.15.45:0/342841955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 msgr2=0x7ff110198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 0x7ff110198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 -- 172.21.15.45:0/342841955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 msgr2=0x7ff11019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 0x7ff11019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 -- 172.21.15.45:0/342841955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11019d040 con 0x7ff1101020d0 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff10effd640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 0x7ff110198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 0x7ff11019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff107fff640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff10cff9640 1 -- 172.21.15.45:0/342841955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8017070 con 0x7ff1101020d0 2024-04-25T00:29:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff10cff9640 1 -- 172.21.15.45:0/342841955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8009c30 con 0x7ff1101020d0 2024-04-25T00:29:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11019d330 con 0x7ff1101020d0 2024-04-25T00:29:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff10cff9640 1 -- 172.21.15.45:0/342841955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800f6b0 con 0x7ff1101020d0 2024-04-25T00:29:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.888+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101a4bc0 con 0x7ff1101020d0 2024-04-25T00:29:05.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff10cff9640 1 -- 172.21.15.45:0/342841955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0f80109d0 con 0x7ff1101020d0 2024-04-25T00:29:05.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 Processor -- start 2024-04-25T00:29:05.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 -- start start 2024-04-25T00:29:05.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 0x5569f447a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f4485570 0x5569f4487960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569f43df670 con 0x5569f447aea0 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5569f4387970 con 0x5569f447a580 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff1183bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5569f43db630 con 0x5569f4485570 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.891+0000 7ff10effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36228/0 (socket says 172.21.15.45:36228) 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 -- 172.21.15.45:0/901581006 learned_addr learned my addr 172.21.15.45:0/901581006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff107fff640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f4485570 0x5569f4487960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 0x5569f447a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10cff9640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 0x7ff0fc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 -- 172.21.15.45:0/901581006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f4485570 msgr2=0x5569f4487960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f4485570 0x5569f4487960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 -- 172.21.15.45:0/901581006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 msgr2=0x5569f447a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 0x5569f447a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 -- 172.21.15.45:0/901581006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f43ab400 con 0x5569f447aea0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 0x5569f447a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 0x7ff0fc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10cff9640 1 -- 172.21.15.45:0/342841955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff0f80cf3f0 con 0x7ff1101020d0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff10effd640 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff100009930 tx=0x7ff100032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7343e961123920f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff1057fa640 1 -- 172.21.15.45:0/901581006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10003f070 con 0x5569f447aea0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff1057fa640 1 -- 172.21.15.45:0/901581006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1000317b0 con 0x5569f447aea0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.892+0000 7ff1057fa640 1 -- 172.21.15.45:0/901581006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100035070 con 0x5569f447aea0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- 172.21.15.45:0/901581006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 msgr2=0x5569f4485030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 0x7ff0fc0a5240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff108005fd0 tx=0x7ff108004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff100009930 tx=0x7ff100032d60 comp rx=0 tx=0).stop 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- 172.21.15.45:0/901581006 shutdown_connections 2024-04-25T00:29:05.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f4485570 0x5569f4487960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4485030 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 --2- 172.21.15.45:0/901581006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447a580 0x5569f447a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- 172.21.15.45:0/901581006 >> 172.21.15.45:0/901581006 conn(0x5569f44768a0 msgr2=0x5569f4476c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- 172.21.15.45:0/901581006 shutdown_connections 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- 172.21.15.45:0/901581006 wait complete. 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.893+0000 7ff1183bb740 1 -- start start 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 Processor -- start 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f4484f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447aea0 0x5569f44821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569f4488300 con 0x5569f4485570 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5569f439b050 con 0x5569f447aea0 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5569f4488480 con 0x5569f447a580 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff1183bb740 10 client.0 _ll_register_callbacks cb 0x5569f40f2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36240/0 (socket says 172.21.15.45:36240) 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 -- 172.21.15.45:0/406888793 learned_addr learned my addr 172.21.15.45:0/406888793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff107fff640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447aea0 0x5569f44821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10effd640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f4484f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 -- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 msgr2=0x5569f4484f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f4484f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 -- 172.21.15.45:0/406888793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447aea0 msgr2=0x5569f44821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447aea0 0x5569f44821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 -- 172.21.15.45:0/406888793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f436cdc0 con 0x5569f4485570 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1080119a0 tx=0x7ff108008420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab0729081a16c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.894+0000 7ff0f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10801cd80 con 0x5569f4485570 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff10801a450 con 0x5569f4485570 2024-04-25T00:29:05.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1000317b0 con 0x5569f4485570 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff10801a650 con 0x5569f4485570 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 10 client.17685 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569f40f33f0 con 0x5569f4485570 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1000317b0 con 0x5569f4485570 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff108023070 con 0x5569f4485570 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 10 client.17685 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 20 client.17685 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f4495680 0x5569f4497ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 4 client.17685 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.895+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f43ecbb0 con 0x5569f4495680 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.896+0000 7ff10effd640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f4495680 0x5569f4497ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.896+0000 7ff10effd640 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f4495680 0x5569f4497ab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13c5a5f8db10bae9 server_cookie=5bf472e8ab77d746 in_seq=0 out_seq=0 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.896+0000 7ff0f67fc640 10 client.17685 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:05.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff0f67fc640 1 -- 172.21.15.45:0/406888793 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569f43ecbb0 con 0x5569f4495680 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff0f67fc640 10 client.17685 handle_command_reply: tid=0 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 client.17685 shutdown 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 msgr2=0x5569f4482770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1080119a0 tx=0x7ff108008420 comp rx=0 tx=0).stop 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 shutdown_connections 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f4495680 0x5569f4497ab0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f4485570 0x5569f4482770 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f447aea0 0x5569f44821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff0f67fc640 0 client.17685 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 --2- 172.21.15.45:0/406888793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f4484f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 >> 172.21.15.45:0/406888793 conn(0x5569f44768a0 msgr2=0x5569f4475b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 shutdown_connections 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 1 -- 172.21.15.45:0/406888793 wait complete. 2024-04-25T00:29:05.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.897+0000 7ff1183bb740 20 client.17685 trim_cache size 0 max 16384 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 Processor -- start 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 -- start start 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 0x5569f4484a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f449d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101aeaf0 con 0x5569f447a580 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b0630 con 0x5569f4485570 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff1183bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101af1c0 con 0x5569f447aea0 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36244/0 (socket says 172.21.15.45:36244) 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 -- 172.21.15.45:0/425702885 learned_addr learned my addr 172.21.15.45:0/425702885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f449d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff107fff640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 0x5569f4484a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 -- 172.21.15.45:0/425702885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 msgr2=0x5569f4484a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 0x5569f4484a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 -- 172.21.15.45:0/425702885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 msgr2=0x5569f449d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f449d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10effd640 1 -- 172.21.15.45:0/425702885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f41bd7d0 con 0x5569f447a580 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f449d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.911+0000 7ff107fff640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 0x5569f4484a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff10effd640 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff1000054c0 tx=0x7ff100005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78c1fafa2bbb6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff104ff9640 1 -- 172.21.15.45:0/425702885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100051070 con 0x5569f447a580 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff104ff9640 1 -- 172.21.15.45:0/425702885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10004cbc0 con 0x5569f447a580 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff104ff9640 1 -- 172.21.15.45:0/425702885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10003f040 con 0x5569f447a580 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 -- 172.21.15.45:0/425702885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 msgr2=0x5569f44911c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff1000054c0 tx=0x7ff100005450 comp rx=0 tx=0).stop 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 -- 172.21.15.45:0/425702885 shutdown_connections 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f449d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447aea0 0x5569f4484a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 --2- 172.21.15.45:0/425702885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447a580 0x5569f44911c0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.912+0000 7ff1183bb740 1 -- 172.21.15.45:0/425702885 >> 172.21.15.45:0/425702885 conn(0x5569f44768a0 msgr2=0x5569f447d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- 172.21.15.45:0/425702885 shutdown_connections 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- 172.21.15.45:0/425702885 wait complete. 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- start start 2024-04-25T00:29:05.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 Processor -- start 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f448da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f44892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101aff60 con 0x5569f447aea0 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101af890 con 0x5569f4485570 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101acad0 con 0x5569f447a580 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff1183bb740 10 client.0 _ll_register_callbacks cb 0x7ff108008e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36246/0 (socket says 172.21.15.45:36246) 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff107fff640 1 -- 172.21.15.45:0/3772937633 learned_addr learned my addr 172.21.15.45:0/3772937633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff107fff640 1 -- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 msgr2=0x5569f448da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.913+0000 7ff10effd640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f448da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff107fff640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f448da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f44892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff107fff640 1 -- 172.21.15.45:0/3772937633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 msgr2=0x5569f44892d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff107fff640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f44892d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff107fff640 1 -- 172.21.15.45:0/3772937633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f4489ad0 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff10effd640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f448da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff10f7fe640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f44892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff107fff640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8097290 tx=0x7ff0f800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f29ba399c5d4d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8020040 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7ff0f8097c20 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10004cbc0 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0f809bcb0 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 10 client.17691 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569f44fbaf0 con 0x5569f447aea0 2024-04-25T00:29:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10004cbc0 con 0x5569f447aea0 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0f8097300 con 0x5569f447aea0 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 10 client.17691 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 10 client.17691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 20 client.17691 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f44fc220 0x5569f44fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 4 client.17691 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.914+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569f43e6f00 con 0x5569f44fc220 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff10effd640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f44fc220 0x5569f44fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff0f67fc640 10 client.17691 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff10effd640 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f44fc220 0x5569f44fe640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e46cd7313998039b server_cookie=779b29b9f93600b2 in_seq=0 out_seq=0 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff0f67fc640 1 -- 172.21.15.45:0/3772937633 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569f43e6f00 con 0x5569f44fc220 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff0f67fc640 10 client.17691 handle_command_reply: tid=0 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 client.17691 shutdown 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 msgr2=0x5569f4488d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8097290 tx=0x7ff0f800bf60 comp rx=0 tx=0).stop 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 shutdown_connections 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5569f44fc220 0x5569f44fe640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5569f4485570 0x5569f44892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569f447aea0 0x5569f4488d90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff0f67fc640 0 client.17691 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 --2- 172.21.15.45:0/3772937633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5569f447a580 0x5569f448da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.915+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 >> 172.21.15.45:0/3772937633 conn(0x5569f44768a0 msgr2=0x5569f447bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 shutdown_connections 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1183bb740 1 -- 172.21.15.45:0/3772937633 wait complete. 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1183bb740 20 client.17691 trim_cache size 0 max 16384 2024-04-25T00:29:05.923 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 msgr2=0x7ff0fc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 0x7ff0fc0a5240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff108005fd0 tx=0x7ff108004a40 comp rx=0 tx=0).stop 2024-04-25T00:29:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 msgr2=0x7ff110198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).stop 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 shutdown_connections 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0fc0a2d80 0x7ff0fc0a5240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110107720 0x7ff11019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1101020d0 0x7ff110198590 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 --2- 172.21.15.45:0/342841955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff110101710 0x7ff110198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 >> 172.21.15.45:0/342841955 conn(0x7ff1100fd180 msgr2=0x7ff1100fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 shutdown_connections 2024-04-25T00:29:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:05.916+0000 7ff1155ce640 1 -- 172.21.15.45:0/342841955 wait complete. 2024-04-25T00:29:05.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 -- 172.21.15.45:0/546757206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf41020d0 msgr2=0x7fddf410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 --2- 172.21.15.45:0/546757206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf41020d0 0x7fddf410e5e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009930 tx=0x7fdde402a420 comp rx=0 tx=0).stop 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 -- 172.21.15.45:0/546757206 shutdown_connections 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 --2- 172.21.15.45:0/546757206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf41020d0 0x7fddf410e5e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 --2- 172.21.15.45:0/546757206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 0x7fddf4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 --2- 172.21.15.45:0/546757206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf4107720 0x7fddf4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 -- 172.21.15.45:0/546757206 >> 172.21.15.45:0/546757206 conn(0x7fddf4078b90 msgr2=0x7fddf4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 -- 172.21.15.45:0/546757206 shutdown_connections 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.033+0000 7fddf985d640 1 -- 172.21.15.45:0/546757206 wait complete. 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 Processor -- start 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 -- start start 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 0x7fddf4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf41020d0 0x7fddf4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4111b30 con 0x7fddf4107720 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41119b0 con 0x7fddf41020d0 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf985d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4111cb0 con 0x7fddf4101710 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36274/0 (socket says 172.21.15.45:36274) 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 -- 172.21.15.45:0/2706151743 learned_addr learned my addr 172.21.15.45:0/2706151743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf27fc640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf41020d0 0x7fddf4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 0x7fddf4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 -- 172.21.15.45:0/2706151743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 msgr2=0x7fddf4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 0x7fddf4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 -- 172.21.15.45:0/2706151743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf41020d0 msgr2=0x7fddf4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf41020d0 0x7fddf4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.034+0000 7fddf37fe640 1 -- 172.21.15.45:0/2706151743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf419d040 con 0x7fddf4107720 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf37fe640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005ab0 tx=0x7fdde40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf885b640 1 -- 172.21.15.45:0/2706151743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde403f030 con 0x7fddf4107720 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf885b640 1 -- 172.21.15.45:0/2706151743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde40044c0 con 0x7fddf4107720 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf419d2d0 con 0x7fddf4107720 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf885b640 1 -- 172.21.15.45:0/2706151743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4035070 con 0x7fddf4107720 2024-04-25T00:29:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.035+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf41a4bc0 con 0x7fddf4107720 2024-04-25T00:29:06.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddf885b640 1 -- 172.21.15.45:0/2706151743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdde4042030 con 0x7fddf4107720 2024-04-25T00:29:06.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 Processor -- start 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 -- start start 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588b0ab7670 con 0x5588b0b5d570 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5588b0a5f970 con 0x5588b0b52580 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.037+0000 7fddfc56b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588b0ab3630 con 0x5588b0b52ea0 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf885b640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 0x7fdddc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:44960/0 (socket says 172.21.15.45:44960) 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 -- 172.21.15.45:0/3785451654 learned_addr learned my addr 172.21.15.45:0/3785451654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:06.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf37fe640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 0x7fdddc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf885b640 1 -- 172.21.15.45:0/2706151743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdde40efb20 con 0x7fddf4107720 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 -- 172.21.15.45:0/3785451654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 msgr2=0x5588b0b52960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b52960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 -- 172.21.15.45:0/3785451654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 msgr2=0x5588b0b5f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 -- 172.21.15.45:0/3785451654 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588b0a83400 con 0x5588b0b52ea0 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fddec010370 tx=0x7fddec010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f92533560e1c62f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddf37fe640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 0x7fdddc0a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdde40ed510 tx=0x7fdde4009260 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddd9ffb640 1 -- 172.21.15.45:0/3785451654 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec016070 con 0x5588b0b52ea0 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddd9ffb640 1 -- 172.21.15.45:0/3785451654 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec004d20 con 0x5588b0b52ea0 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.038+0000 7fddd9ffb640 1 -- 172.21.15.45:0/3785451654 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec005480 con 0x5588b0b52ea0 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- 172.21.15.45:0/3785451654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 msgr2=0x5588b0b5d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fddec010370 tx=0x7fddec010830 comp rx=0 tx=0).stop 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- 172.21.15.45:0/3785451654 shutdown_connections 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5d030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3785451654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b52960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- 172.21.15.45:0/3785451654 >> 172.21.15.45:0/3785451654 conn(0x5588b0b4e8a0 msgr2=0x5588b0b4ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- 172.21.15.45:0/3785451654 shutdown_connections 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- 172.21.15.45:0/3785451654 wait complete. 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.039+0000 7fddfc56b740 1 -- start start 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 Processor -- start 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b60360 con 0x5588b0b5d570 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5588b0a73050 con 0x5588b0b52580 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588b0b604e0 con 0x5588b0b52ea0 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddfc56b740 10 client.0 _ll_register_callbacks cb 0x5588b07cabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:06.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36302/0 (socket says 172.21.15.45:36302) 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 -- 172.21.15.45:0/3499429546 learned_addr learned my addr 172.21.15.45:0/3499429546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 -- 172.21.15.45:0/3499429546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 msgr2=0x5588b0b5a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 -- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 msgr2=0x5588b0b5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b5ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.040+0000 7fddf27fc640 1 -- 172.21.15.45:0/3499429546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588b0a44dc0 con 0x5588b0b5d570 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddf27fc640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdde8010e90 tx=0x7fdde8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b202b16d559515f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8019040 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdde8005020 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec004d20 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdde802c050 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 10 client.17703 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588b07cb3f0 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec004d20 con 0x5588b0b5d570 2024-04-25T00:29:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdde8021470 con 0x5588b0b5d570 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 10 client.17703 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 20 client.17703 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0b6d670 0x5588b0b6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 4 client.17703 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.041+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588b0ac4bb0 con 0x5588b0b6d670 2024-04-25T00:29:06.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.042+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0b6d670 0x5588b0b6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.042+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0b6d670 0x5588b0b6faa0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be78b6b0d4772cc server_cookie=56d7326db9043d79 in_seq=0 out_seq=0 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.042+0000 7fddd2ffd640 10 client.17703 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddd2ffd640 1 -- 172.21.15.45:0/3499429546 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588b0ac4bb0 con 0x5588b0b6d670 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddd2ffd640 10 client.17703 handle_command_reply: tid=0 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 client.17703 shutdown 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 msgr2=0x5588b0b5a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdde8010e90 tx=0x7fdde8009b40 comp rx=0 tx=0).stop 2024-04-25T00:29:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 shutdown_connections 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0b6d670 0x5588b0b6faa0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0b5a720 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b5a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddd2ffd640 0 client.17703 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 --2- 172.21.15.45:0/3499429546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0b5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 >> 172.21.15.45:0/3499429546 conn(0x5588b0b4e8a0 msgr2=0x5588b0b546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 shutdown_connections 2024-04-25T00:29:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 1 -- 172.21.15.45:0/3499429546 wait complete. 2024-04-25T00:29:06.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.043+0000 7fddfc56b740 20 client.17703 trim_cache size 0 max 16384 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.056+0000 7fddfc56b740 1 Processor -- start 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.056+0000 7fddfc56b740 1 -- start start 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b65b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b5d570 0x5588b0b5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41aeb10 con 0x5588b0b52580 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b0650 con 0x5588b0b5d570 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddfc56b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41af1e0 con 0x5588b0b52ea0 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36310/0 (socket says 172.21.15.45:36310) 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 -- 172.21.15.45:0/2775601797 learned_addr learned my addr 172.21.15.45:0/2775601797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b65b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b5d570 0x5588b0b5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 -- 172.21.15.45:0/2775601797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 msgr2=0x5588b0b65b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b65b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 -- 172.21.15.45:0/2775601797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b5d570 msgr2=0x5588b0b5ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b5d570 0x5588b0b5ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 -- 172.21.15.45:0/2775601797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588b08957d0 con 0x5588b0b52580 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddf27fc640 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009920 tx=0x7fdde80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48e20a8ebe0a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddd97fa640 1 -- 172.21.15.45:0/2775601797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800fd10 con 0x5588b0b52580 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddd97fa640 1 -- 172.21.15.45:0/2775601797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde8021470 con 0x5588b0b52580 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.057+0000 7fddd97fa640 1 -- 172.21.15.45:0/2775601797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8005650 con 0x5588b0b52580 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- 172.21.15.45:0/2775601797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 msgr2=0x5588b0b55980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009920 tx=0x7fdde80098b0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- 172.21.15.45:0/2775601797 shutdown_connections 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b5d570 0x5588b0b5ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0b65b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 --2- 172.21.15.45:0/2775601797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b52580 0x5588b0b55980 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- 172.21.15.45:0/2775601797 >> 172.21.15.45:0/2775601797 conn(0x5588b0b4e8a0 msgr2=0x5588b0b545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- 172.21.15.45:0/2775601797 shutdown_connections 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- 172.21.15.45:0/2775601797 wait complete. 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 -- start start 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.058+0000 7fddfc56b740 1 Processor -- start 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0bcbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0bcc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41aff80 con 0x5588b0b5d570 2024-04-25T00:29:06.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41af8b0 con 0x5588b0b52580 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41acaf0 con 0x5588b0b52ea0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 10 client.0 _ll_register_callbacks cb 0x7fdde8010ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36324/0 (socket says 172.21.15.45:36324) 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 -- 172.21.15.45:0/1280088308 learned_addr learned my addr 172.21.15.45:0/1280088308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0bcc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0bcbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 -- 172.21.15.45:0/1280088308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 msgr2=0x5588b0bcc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0bcc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 -- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 msgr2=0x5588b0bcbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0bcbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 -- 172.21.15.45:0/1280088308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588b0bd6dd0 con 0x5588b0b5d570 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddf27fc640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdde8010be0 tx=0x7fdde8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c36384c667cd0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8021620 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fdde8021ab0 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.059+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde8021470 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdde80035f0 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 10 client.17709 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588b0bd80d0 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde8021470 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdde803f720 con 0x5588b0b5d570 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 10 client.17709 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 20 client.17709 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0bd86f0 0x5588b0bdab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 4 client.17709 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588b0abef00 con 0x5588b0bd86f0 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.060+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0bd86f0 0x5588b0bdab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddd2ffd640 10 client.17709 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddf2ffd640 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0bd86f0 0x5588b0bdab10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c517adb016bc9f88 server_cookie=409ba8ca54617e8 in_seq=0 out_seq=0 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddd2ffd640 1 -- 172.21.15.45:0/1280088308 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588b0abef00 con 0x5588b0bd86f0 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddd2ffd640 10 client.17709 handle_command_reply: tid=0 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 client.17709 shutdown 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 msgr2=0x5588b0bcc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdde8010be0 tx=0x7fdde8005230 comp rx=0 tx=0).stop 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 shutdown_connections 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5588b0bd86f0 0x5588b0bdab10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588b0b5d570 0x5588b0bcc620 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddd2ffd640 0 client.17709 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588b0b52ea0 0x5588b0bcc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 --2- 172.21.15.45:0/1280088308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588b0b52580 0x5588b0bcbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 >> 172.21.15.45:0/1280088308 conn(0x5588b0b4e8a0 msgr2=0x5588b0b53cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 shutdown_connections 2024-04-25T00:29:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 1 -- 172.21.15.45:0/1280088308 wait complete. 2024-04-25T00:29:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.061+0000 7fddfc56b740 20 client.17709 trim_cache size 0 max 16384 2024-04-25T00:29:06.070 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 msgr2=0x7fdddc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 0x7fdddc0a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdde40ed510 tx=0x7fdde4009260 comp rx=0 tx=0).stop 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 msgr2=0x7fddf419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005ab0 tx=0x7fdde40040c0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 shutdown_connections 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdddc0a2d00 0x7fdddc0a51c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf4107720 0x7fddf419c910 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf41020d0 0x7fddf4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 --2- 172.21.15.45:0/2706151743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddf4101710 0x7fddf4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 >> 172.21.15.45:0/2706151743 conn(0x7fddf4078b90 msgr2=0x7fddf40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 shutdown_connections 2024-04-25T00:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:06.062+0000 7fddf985d640 1 -- 172.21.15.45:0/2706151743 wait complete. 2024-04-25T00:29:06.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2051501224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:06.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2051501224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:06.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2051501224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:07.073 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:29:07.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:29:07.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:29:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 -- 172.21.15.45:0/1865630736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101610 msgr2=0x7f1844101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 --2- 172.21.15.45:0/1865630736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101610 0x7f1844101a90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f182c009930 tx=0x7f182c02a420 comp rx=0 tx=0).stop 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 -- 172.21.15.45:0/1865630736 shutdown_connections 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 --2- 172.21.15.45:0/1865630736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844101fd0 0x7f184410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 --2- 172.21.15.45:0/1865630736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101610 0x7f1844101a90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 --2- 172.21.15.45:0/1865630736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18441076b0 0x7f1844107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 -- 172.21.15.45:0/1865630736 >> 172.21.15.45:0/1865630736 conn(0x7f18440fd100 msgr2=0x7f18440ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 -- 172.21.15.45:0/1865630736 shutdown_connections 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.193+0000 7f18489ff640 1 -- 172.21.15.45:0/1865630736 wait complete. 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 Processor -- start 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 -- start start 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1844101610 0x7f1844073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18441076b0 0x7f18440744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18441119f0 con 0x7f1844101fd0 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111870 con 0x7f18441076b0 2024-04-25T00:29:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.194+0000 7f18489ff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111b70 con 0x7f1844101610 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36344/0 (socket says 172.21.15.45:36344) 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 -- 172.21.15.45:0/2277574839 learned_addr learned my addr 172.21.15.45:0/2277574839 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f1842d9d640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1844101610 0x7f1844073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f184359e640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18441076b0 0x7f18440744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 -- 172.21.15.45:0/2277574839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1844101610 msgr2=0x7f1844073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1844101610 0x7f1844073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 -- 172.21.15.45:0/2277574839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18441076b0 msgr2=0x7f18440744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18441076b0 0x7f18440744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 -- 172.21.15.45:0/2277574839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18441a63b0 con 0x7f1844101fd0 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f183bfff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f182c009230 tx=0x7f182c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f1840d99640 1 -- 172.21.15.45:0/2277574839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c03f030 con 0x7f1844101fd0 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f1840d99640 1 -- 172.21.15.45:0/2277574839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182c004440 con 0x7f1844101fd0 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18441a6640 con 0x7f1844101fd0 2024-04-25T00:29:07.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f1840d99640 1 -- 172.21.15.45:0/2277574839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c035070 con 0x7f1844101fd0 2024-04-25T00:29:07.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.195+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a6b30 con 0x7f1844101fd0 2024-04-25T00:29:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.198+0000 7f1840d99640 1 -- 172.21.15.45:0/2277574839 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f182c042030 con 0x7f1844101fd0 2024-04-25T00:29:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.198+0000 7f184bfee740 1 Processor -- start 2024-04-25T00:29:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 -- start start 2024-04-25T00:29:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 0x560e27cd4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 0x560e27ce25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e27c392c0 con 0x560e27cd5480 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e27be1910 con 0x560e27cd6b70 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184bfee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e27c35280 con 0x560e27ce0200 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36348/0 (socket says 172.21.15.45:36348) 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 -- 172.21.15.45:0/3452678189 learned_addr learned my addr 172.21.15.45:0/3452678189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1840d99640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 0x7f18300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184359e640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 0x560e27cd4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f183bfff640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 0x560e27ce25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 -- 172.21.15.45:0/3452678189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 msgr2=0x560e27ce25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 0x560e27ce25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 -- 172.21.15.45:0/3452678189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 msgr2=0x560e27cd4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 0x560e27cd4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f1842d9d640 1 -- 172.21.15.45:0/3452678189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e27bf2fb0 con 0x560e27cd5480 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f184359e640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 0x560e27cd4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.199+0000 7f183bfff640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 0x560e27ce25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f184359e640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 0x7f18300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f1840d99640 1 -- 172.21.15.45:0/2277574839 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f182c0efa20 con 0x7f1844101fd0 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1834009c20 tx=0x7f1834011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9aabf26ab4312f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f18397fa640 1 -- 172.21.15.45:0/3452678189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834004220 con 0x560e27cd5480 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f18397fa640 1 -- 172.21.15.45:0/3452678189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18340043c0 con 0x560e27cd5480 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.200+0000 7f18397fa640 1 -- 172.21.15.45:0/3452678189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834019760 con 0x560e27cd5480 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184359e640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 0x7f18300a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- 172.21.15.45:0/3452678189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 msgr2=0x560e27cd5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1834009c20 tx=0x7f1834011810 comp rx=0 tx=0).stop 2024-04-25T00:29:07.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- 172.21.15.45:0/3452678189 shutdown_connections 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27ce0200 0x560e27ce25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cd5900 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 --2- 172.21.15.45:0/3452678189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27cd6b70 0x560e27cd4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- 172.21.15.45:0/3452678189 >> 172.21.15.45:0/3452678189 conn(0x560e27c9d950 msgr2=0x560e27cd19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- 172.21.15.45:0/3452678189 shutdown_connections 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- 172.21.15.45:0/3452678189 wait complete. 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 -- start start 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.201+0000 7f184bfee740 1 Processor -- start 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e27ce3090 con 0x560e27cd5480 2024-04-25T00:29:07.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e27bf4e90 con 0x560e27ce0200 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e27ce3210 con 0x560e27cd6b70 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 10 client.0 _ll_register_callbacks cb 0x560e27c30970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36362/0 (socket says 172.21.15.45:36362) 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 -- 172.21.15.45:0/1935906904 learned_addr learned my addr 172.21.15.45:0/1935906904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f183bfff640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184359e640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 -- 172.21.15.45:0/1935906904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 msgr2=0x560e27cdd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 -- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 msgr2=0x560e27cddb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cddb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 -- 172.21.15.45:0/1935906904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e27a157e0 con 0x560e27cd5480 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184359e640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f1842d9d640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1834009bf0 tx=0x7f1834009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94657246cef98e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f182a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834028070 con 0x560e27cd5480 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f18340035f0 con 0x560e27cd5480 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.202+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18340043c0 con 0x560e27cd5480 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1834034050 con 0x560e27cd5480 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 10 client.17721 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e27a75f70 con 0x560e27cd5480 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18340043c0 con 0x560e27cd5480 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1834004a80 con 0x560e27cd5480 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 10 client.17721 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 20 client.17721 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27cf0070 0x560e27cf2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 4 client.17721 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.203+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e27c0d5a0 con 0x560e27cf0070 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.204+0000 7f183bfff640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27cf0070 0x560e27cf2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.204+0000 7f182a7fc640 10 client.17721 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.204+0000 7f183bfff640 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27cf0070 0x560e27cf2460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d844b75dadbe419 server_cookie=336df4ca220cd871 in_seq=0 out_seq=0 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f182a7fc640 1 -- 172.21.15.45:0/1935906904 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e27c0d5a0 con 0x560e27cf0070 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f182a7fc640 10 client.17721 handle_command_reply: tid=0 2024-04-25T00:29:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 client.17721 shutdown 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 msgr2=0x560e27cdd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1834009bf0 tx=0x7f1834009970 comp rx=0 tx=0).stop 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 shutdown_connections 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cddb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27cf0070 0x560e27cf2460 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 --2- 172.21.15.45:0/1935906904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cdd090 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f182a7fc640 0 client.17721 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 >> 172.21.15.45:0/1935906904 conn(0x560e27c9d950 msgr2=0x560e27cd7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 shutdown_connections 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 1 -- 172.21.15.45:0/1935906904 wait complete. 2024-04-25T00:29:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.205+0000 7f184bfee740 20 client.17721 trim_cache size 0 max 16384 2024-04-25T00:29:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 Processor -- start 2024-04-25T00:29:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 -- start start 2024-04-25T00:29:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b0bc0 con 0x560e27cd5480 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b2700 con 0x560e27ce0200 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184bfee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b1290 con 0x560e27cd6b70 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f1842d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36376/0 (socket says 172.21.15.45:36376) 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f1842d9d640 1 -- 172.21.15.45:0/3441190991 learned_addr learned my addr 172.21.15.45:0/3441190991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f183bfff640 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.223+0000 7f184359e640 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 -- 172.21.15.45:0/3441190991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 msgr2=0x560e27cdc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 -- 172.21.15.45:0/3441190991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 msgr2=0x560e27cdd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 -- 172.21.15.45:0/3441190991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e27cf27d0 con 0x560e27cd5480 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1842d9d640 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1834004280 tx=0x7f1834005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330085500262c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1838ff9640 1 -- 172.21.15.45:0/3441190991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834028040 con 0x560e27cd5480 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1838ff9640 1 -- 172.21.15.45:0/3441190991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834009780 con 0x560e27cd5480 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.224+0000 7f1838ff9640 1 -- 172.21.15.45:0/3441190991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834019c30 con 0x560e27cd5480 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- 172.21.15.45:0/3441190991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 msgr2=0x560e27cebf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1834004280 tx=0x7f1834005f90 comp rx=0 tx=0).stop 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- 172.21.15.45:0/3441190991 shutdown_connections 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd6b70 0x560e27cdc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- 172.21.15.45:0/3441190991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd5480 0x560e27cebf90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- 172.21.15.45:0/3441190991 >> 172.21.15.45:0/3441190991 conn(0x560e27c9d950 msgr2=0x560e27cdd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- 172.21.15.45:0/3441190991 shutdown_connections 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- 172.21.15.45:0/3441190991 wait complete. 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- start start 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 Processor -- start 2024-04-25T00:29:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd5480 0x560e27cdcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b2030 con 0x560e27cd6b70 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b1960 con 0x560e27ce0200 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18441aeba0 con 0x560e27cd5480 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f184bfee740 10 client.0 _ll_register_callbacks cb 0x7f1834009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f183bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f183bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36378/0 (socket says 172.21.15.45:36378) 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.225+0000 7f183bfff640 1 -- 172.21.15.45:0/477567560 learned_addr learned my addr 172.21.15.45:0/477567560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f1842d9d640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd5480 0x560e27cdcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 -- 172.21.15.45:0/477567560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd5480 msgr2=0x560e27cdcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd5480 0x560e27cdcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 -- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 msgr2=0x560e27cdd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f184359e640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 -- 172.21.15.45:0/477567560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2790d170 con 0x560e27cd6b70 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f184359e640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f183bfff640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b80f0 tx=0x7f182c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5fab77746709ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c049040 con 0x560e27cd6b70 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7f182c003d20 con 0x560e27cd6b70 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1834009780 con 0x560e27cd6b70 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f182c0c4850 con 0x560e27cd6b70 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f184bfee740 10 client.17727 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e27d56960 con 0x560e27cd6b70 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834009780 con 0x560e27cd6b70 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.226+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f182c031bd0 con 0x560e27cd6b70 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 10 client.17727 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 20 client.17727 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27d57080 0x560e27d59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 4 client.17727 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e27ccccd0 con 0x560e27d57080 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f1842d9d640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27d57080 0x560e27d59470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f1842d9d640 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27d57080 0x560e27d59470 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d79bad12ffdd4e server_cookie=fca6284a8ea9dbf9 in_seq=0 out_seq=0 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.227+0000 7f182a7fc640 10 client.17727 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f182a7fc640 1 -- 172.21.15.45:0/477567560 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e27ccccd0 con 0x560e27d57080 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f182a7fc640 10 client.17727 handle_command_reply: tid=0 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 client.17727 shutdown 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 msgr2=0x560e27cdd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b80f0 tx=0x7f182c02a420 comp rx=0 tx=0).stop 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 shutdown_connections 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e27ce0200 0x560e27cdd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.228+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e27d57080 0x560e27d59470 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e27cd6b70 0x560e27cdd440 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f182a7fc640 0 client.17727 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 1 --2- 172.21.15.45:0/477567560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e27cd5480 0x560e27cdcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 >> 172.21.15.45:0/477567560 conn(0x560e27c9d950 msgr2=0x560e27cd77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 shutdown_connections 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 1 -- 172.21.15.45:0/477567560 wait complete. 2024-04-25T00:29:07.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f184bfee740 20 client.17727 trim_cache size 0 max 16384 2024-04-25T00:29:07.238 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:07.238 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:29:07.238 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "cd47f752-1c1a-4e26-8a5f-6f8787f32925", 2024-04-25T00:29:07.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:29:07.238 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 msgr2=0x7f18300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 0x7f18300a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c004a40 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 msgr2=0x7f1844073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f182c009230 tx=0x7f182c004040 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 shutdown_connections 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18300a2d30 0x7f18300a51f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18441076b0 0x7f18440744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844101fd0 0x7f1844073f80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 --2- 172.21.15.45:0/2277574839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1844101610 0x7f1844073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 >> 172.21.15.45:0/2277574839 conn(0x7f18440fd100 msgr2=0x7f18440fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 shutdown_connections 2024-04-25T00:29:07.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.229+0000 7f18489ff640 1 -- 172.21.15.45:0/2277574839 wait complete. 2024-04-25T00:29:07.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:29:07.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 -- 172.21.15.45:0/1017246367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101710 msgr2=0x7fde3c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 --2- 172.21.15.45:0/1017246367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101710 0x7fde3c101b90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fde30009930 tx=0x7fde3002a420 comp rx=0 tx=0).stop 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 -- 172.21.15.45:0/1017246367 shutdown_connections 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 --2- 172.21.15.45:0/1017246367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 --2- 172.21.15.45:0/1017246367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101710 0x7fde3c101b90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 --2- 172.21.15.45:0/1017246367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c107720 0x7fde3c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 -- 172.21.15.45:0/1017246367 >> 172.21.15.45:0/1017246367 conn(0x7fde3c0fd180 msgr2=0x7fde3c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 -- 172.21.15.45:0/1017246367 shutdown_connections 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.345+0000 7fde40d90640 1 -- 172.21.15.45:0/1017246367 wait complete. 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 Processor -- start 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 -- start start 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 0x7fde3c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111b30 con 0x7fde3c107720 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1119b0 con 0x7fde3c1020d0 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde40d90640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111cb0 con 0x7fde3c101710 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde3ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde3ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36392/0 (socket says 172.21.15.45:36392) 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde3ad9d640 1 -- 172.21.15.45:0/2910119695 learned_addr learned my addr 172.21.15.45:0/2910119695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde3a59c640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 0x7fde3c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.346+0000 7fde39d9b640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 -- 172.21.15.45:0/2910119695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 msgr2=0x7fde3c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 0x7fde3c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 -- 172.21.15.45:0/2910119695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 msgr2=0x7fde3c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 -- 172.21.15.45:0/2910119695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c19d0a0 con 0x7fde3c107720 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde39d9b640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3a59c640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 0x7fde3c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde3ad9d640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d950 tx=0x7fde3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde2b7fe640 1 -- 172.21.15.45:0/2910119695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34017030 con 0x7fde3c107720 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c19d390 con 0x7fde3c107720 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde2b7fe640 1 -- 172.21.15.45:0/2910119695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde34004020 con 0x7fde3c107720 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde2b7fe640 1 -- 172.21.15.45:0/2910119695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34003c00 con 0x7fde3c107720 2024-04-25T00:29:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.347+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c1a4c20 con 0x7fde3c107720 2024-04-25T00:29:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.349+0000 7fde2b7fe640 1 -- 172.21.15.45:0/2910119695 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde3401a070 con 0x7fde3c107720 2024-04-25T00:29:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.349+0000 7fde43b78740 1 Processor -- start 2024-04-25T00:29:07.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 -- start start 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6aef50 0x564cbf6b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564cbf6136b0 con 0x564cbf6ae630 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564cbf5bb880 con 0x564cbf6b9620 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde43b78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564cbf60f670 con 0x564cbf6aef50 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36398/0 (socket says 172.21.15.45:36398) 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 -- 172.21.15.45:0/618390753 learned_addr learned my addr 172.21.15.45:0/618390753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde2b7fe640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 0x7fde240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde3ad9d640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 0x7fde240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde3a59c640 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6aef50 0x564cbf6b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 -- 172.21.15.45:0/618390753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6aef50 msgr2=0x564cbf6b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6aef50 0x564cbf6b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 -- 172.21.15.45:0/618390753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 msgr2=0x564cbf6bba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde39d9b640 1 -- 172.21.15.45:0/618390753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbf3f17d0 con 0x564cbf6ae630 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.350+0000 7fde2b7fe640 1 -- 172.21.15.45:0/2910119695 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fde340cf140 con 0x7fde3c107720 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde39d9b640 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fde30031550 tx=0x7fde30005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc6dd4252b65527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde3ad9d640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 0x7fde240a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde340cc5e0 tx=0x7fde3406d2c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde28ff9640 1 -- 172.21.15.45:0/618390753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30035050 con 0x564cbf6ae630 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde28ff9640 1 -- 172.21.15.45:0/618390753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30005040 con 0x564cbf6ae630 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde28ff9640 1 -- 172.21.15.45:0/618390753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300048c0 con 0x564cbf6ae630 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde43b78740 1 -- 172.21.15.45:0/618390753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 msgr2=0x564cbf6aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.351+0000 7fde43b78740 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fde30031550 tx=0x7fde30005bb0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- 172.21.15.45:0/618390753 shutdown_connections 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6aef50 0x564cbf6b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- 172.21.15.45:0/618390753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6ae630 0x564cbf6aea10 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- 172.21.15.45:0/618390753 >> 172.21.15.45:0/618390753 conn(0x564cbf6aa950 msgr2=0x564cbf6aad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- 172.21.15.45:0/618390753 shutdown_connections 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- 172.21.15.45:0/618390753 wait complete. 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- start start 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 Processor -- start 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 0x564cbf6b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564cbf6bc3b0 con 0x564cbf6b9620 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564cbf5cef50 con 0x564cbf6aef50 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564cbf6bc530 con 0x564cbf6ae630 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde43b78740 10 client.0 _ll_register_callbacks cb 0x564cbf60a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde39d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.352+0000 7fde3a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 0x564cbf6b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde39d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36410/0 (socket says 172.21.15.45:36410) 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde39d9b640 1 -- 172.21.15.45:0/3547178203 learned_addr learned my addr 172.21.15.45:0/3547178203 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 -- 172.21.15.45:0/3547178203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 msgr2=0x564cbf6b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 -- 172.21.15.45:0/3547178203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 msgr2=0x564cbf6b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 -- 172.21.15.45:0/3547178203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbf3273f0 con 0x564cbf6aef50 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 0x564cbf6b6b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0116e0 tx=0x7fde2c00af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33db715e48afa00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c003b70 con 0x564cbf6aef50 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde30005040 con 0x564cbf6aef50 2024-04-25T00:29:07.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.353+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fde2c003d10 con 0x564cbf6aef50 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde2c00f940 con 0x564cbf6aef50 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 10 client.27014 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cbf2e70f0 con 0x564cbf6aef50 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30005040 con 0x564cbf6aef50 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde2c012450 con 0x564cbf6aef50 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 10 client.27014 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 10 client.27014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 20 client.27014 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf6c99d0 0x564cbf6cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 4 client.27014 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cbf620bf0 con 0x564cbf6c99d0 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.354+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf6c99d0 0x564cbf6cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf6c99d0 0x564cbf6cbdf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e22a13e4e8cd44 server_cookie=8c60eb2f2c5f641f in_seq=0 out_seq=0 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde21ffb640 10 client.27014 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde21ffb640 1 -- 172.21.15.45:0/3547178203 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cbf620bf0 con 0x564cbf6c99d0 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde21ffb640 10 client.27014 handle_command_reply: tid=0 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 client.27014 shutdown 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 msgr2=0x564cbf6b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 0x564cbf6b6b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0116e0 tx=0x7fde2c00af90 comp rx=0 tx=0).stop 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 shutdown_connections 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf6c99d0 0x564cbf6cbdf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6b9620 0x564cbf6b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6aef50 0x564cbf6b6b20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 --2- 172.21.15.45:0/3547178203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde21ffb640 0 client.27014 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.355+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 >> 172.21.15.45:0/3547178203 conn(0x564cbf6aa950 msgr2=0x564cbf6b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.356+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 shutdown_connections 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.356+0000 7fde43b78740 1 -- 172.21.15.45:0/3547178203 wait complete. 2024-04-25T00:29:07.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.356+0000 7fde43b78740 20 client.27014 trim_cache size 0 max 16384 2024-04-25T00:29:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:07 smithi071 ceph-mon[32667]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:29:07.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 Processor -- start 2024-04-25T00:29:07.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 -- start start 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6ae630 0x564cbf6c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6b9620 0x564cbf6b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1aeb30 con 0x564cbf6aef50 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b0670 con 0x564cbf6ae630 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde43b78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1af200 con 0x564cbf6b9620 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36412/0 (socket says 172.21.15.45:36412) 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde3a59c640 1 -- 172.21.15.45:0/1442629529 learned_addr learned my addr 172.21.15.45:0/1442629529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.373+0000 7fde39d9b640 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6ae630 0x564cbf6c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 -- 172.21.15.45:0/1442629529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6b9620 msgr2=0x564cbf6b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6b9620 0x564cbf6b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 -- 172.21.15.45:0/1442629529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6ae630 msgr2=0x564cbf6c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6ae630 0x564cbf6c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 -- 172.21.15.45:0/1442629529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbf69c090 con 0x564cbf6aef50 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde3a59c640 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00a300 tx=0x7fde2c0059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e590baf56ef70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde23fff640 1 -- 172.21.15.45:0/1442629529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c033070 con 0x564cbf6aef50 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde23fff640 1 -- 172.21.15.45:0/1442629529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2c00f730 con 0x564cbf6aef50 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde23fff640 1 -- 172.21.15.45:0/1442629529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c02e770 con 0x564cbf6aef50 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 -- 172.21.15.45:0/1442629529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 msgr2=0x564cbf6a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00a300 tx=0x7fde2c0059f0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 -- 172.21.15.45:0/1442629529 shutdown_connections 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6b9620 0x564cbf6b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6a9af0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 --2- 172.21.15.45:0/1442629529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6ae630 0x564cbf6c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 -- 172.21.15.45:0/1442629529 >> 172.21.15.45:0/1442629529 conn(0x564cbf6aa950 msgr2=0x564cbf6b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.374+0000 7fde43b78740 1 -- 172.21.15.45:0/1442629529 shutdown_connections 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 -- 172.21.15.45:0/1442629529 wait complete. 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 -- start start 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 Processor -- start 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1affa0 con 0x564cbf6aef50 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1af8d0 con 0x564cbf6b9620 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1acb10 con 0x564cbf6ae630 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde43b78740 10 client.0 _ll_register_callbacks cb 0x7fde2c00a5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36414/0 (socket says 172.21.15.45:36414) 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 -- 172.21.15.45:0/3670080458 learned_addr learned my addr 172.21.15.45:0/3670080458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 -- 172.21.15.45:0/3670080458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 msgr2=0x564cbf6c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 -- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 msgr2=0x564cbf6bd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.375+0000 7fde3a59c640 1 -- 172.21.15.45:0/3670080458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbf6bdbe0 con 0x564cbf6aef50 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde3a59c640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fde2c002230 tx=0x7fde2c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd278d5e8afd272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c033070 con 0x564cbf6aef50 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9387+0+0 (secure 0 0 0) 0x7fde2c003970 con 0x564cbf6aef50 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2c00f730 con 0x564cbf6aef50 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde2c003b70 con 0x564cbf6aef50 2024-04-25T00:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 10 client.17745 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564cbf72ff50 con 0x564cbf6aef50 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde2c00f730 con 0x564cbf6aef50 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde2c02e5d0 con 0x564cbf6aef50 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 10 client.17745 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 20 client.17745 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf7305e0 0x564cbf732a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 4 client.17745 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.376+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564cbf61af40 con 0x564cbf7305e0 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf7305e0 0x564cbf732a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde39d9b640 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf7305e0 0x564cbf732a00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac40ec9803d8e9d1 server_cookie=4acd0c710ca4befd in_seq=0 out_seq=0 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde21ffb640 10 client.17745 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde21ffb640 1 -- 172.21.15.45:0/3670080458 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564cbf61af40 con 0x564cbf7305e0 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde21ffb640 10 client.17745 handle_command_reply: tid=0 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 client.17745 shutdown 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 msgr2=0x564cbf6bcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fde2c002230 tx=0x7fde2c011660 comp rx=0 tx=0).stop 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 shutdown_connections 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564cbf6b9620 0x564cbf6bd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x564cbf7305e0 0x564cbf732a00 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564cbf6aef50 0x564cbf6bcec0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 --2- 172.21.15.45:0/3670080458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564cbf6ae630 0x564cbf6c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde21ffb640 0 client.17745 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 >> 172.21.15.45:0/3670080458 conn(0x564cbf6aa950 msgr2=0x564cbf6a9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 shutdown_connections 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 1 -- 172.21.15.45:0/3670080458 wait complete. 2024-04-25T00:29:07.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.377+0000 7fde43b78740 20 client.17745 trim_cache size 0 max 16384 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "cd47f752-1c1a-4e26-8a5f-6f8787f32925": { 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "cd47f752-1c1a-4e26-8a5f-6f8787f32925", 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:29:07.390 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 msgr2=0x7fde240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 0x7fde240a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde340cc5e0 tx=0x7fde3406d2c0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 msgr2=0x7fde3c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d950 tx=0x7fde3400de10 comp rx=0 tx=0).stop 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 shutdown_connections 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fde240a2d00 0x7fde240a51c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c107720 0x7fde3c19c970 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c1020d0 0x7fde3c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 --2- 172.21.15.45:0/2910119695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde3c101710 0x7fde3c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 >> 172.21.15.45:0/2910119695 conn(0x7fde3c0fd180 msgr2=0x7fde3c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 shutdown_connections 2024-04-25T00:29:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:07.378+0000 7fde40d90640 1 -- 172.21.15.45:0/2910119695 wait complete. 2024-04-25T00:29:07.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:07 smithi045 ceph-mon[29194]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:29:07.393 INFO:tasks.cephfs.filesystem:scrub status for tag:cd47f752-1c1a-4e26-8a5f-6f8787f32925 - {'path': '/', 'tag': 'cd47f752-1c1a-4e26-8a5f-6f8787f32925', 'options': 'recursive,force'} 2024-04-25T00:29:07.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:07 smithi122 ceph-mon[32959]: pgmap v217: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:29:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:08 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:29:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:08 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:08 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:29:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:08 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:08 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:29:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:08 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:29:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:08 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:08 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:29:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:08 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:08 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:29:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:08 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:29:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:08 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:08 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:29:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:08 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:08.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:08 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:29:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:29:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:29:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:09 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:29:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:29:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:29:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:09 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: pgmap v218: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:09 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:29:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:11 smithi071 ceph-mon[32667]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:11 smithi045 ceph-mon[29194]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:11 smithi122 ceph-mon[32959]: pgmap v220: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:13.358 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:13 smithi122 ceph-mon[32959]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:13 smithi071 ceph-mon[32667]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:13 smithi045 ceph-mon[29194]: pgmap v221: 129 pgs: 129 active+clean; 259 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-25T00:29:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:15 smithi071 ceph-mon[32667]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:29:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:15 smithi045 ceph-mon[29194]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:29:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:15 smithi122 ceph-mon[32959]: pgmap v222: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:29:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:17 smithi071 ceph-mon[32667]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:29:17.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:17 smithi071 ceph-mon[32667]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:29:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:17 smithi045 ceph-mon[29194]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:29:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:17 smithi045 ceph-mon[29194]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:29:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:17 smithi122 ceph-mon[32959]: pgmap v223: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:29:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:17 smithi122 ceph-mon[32959]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:29:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:19 smithi071 ceph-mon[32667]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 B/s wr, 11 op/s 2024-04-25T00:29:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:19 smithi045 ceph-mon[29194]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 B/s wr, 11 op/s 2024-04-25T00:29:19.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:19 smithi122 ceph-mon[32959]: pgmap v225: 129 pgs: 129 active+clean; 259 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 B/s wr, 11 op/s 2024-04-25T00:29:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:21 smithi071 ceph-mon[32667]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:21 smithi045 ceph-mon[29194]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:21 smithi122 ceph-mon[32959]: pgmap v226: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:23 smithi071 ceph-mon[32667]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:23 smithi045 ceph-mon[29194]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:23 smithi122 ceph-mon[32959]: pgmap v227: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:29:25.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:25 smithi071 ceph-mon[32667]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:25 smithi045 ceph-mon[29194]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:25.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:25 smithi122 ceph-mon[32959]: pgmap v228: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:27 smithi045 ceph-mon[29194]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:27.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:27 smithi122 ceph-mon[32959]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:27.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:27 smithi071 ceph-mon[32667]: pgmap v229: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T00:29:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:29 smithi045 ceph-mon[29194]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:29 smithi122 ceph-mon[32959]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:29.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:29 smithi071 ceph-mon[32667]: pgmap v230: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:31 smithi045 ceph-mon[29194]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:31.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:31 smithi122 ceph-mon[32959]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:31.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:31 smithi071 ceph-mon[32667]: pgmap v231: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:32 smithi045 ceph-mon[29194]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:32.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:32 smithi122 ceph-mon[32959]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:32.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:32 smithi071 ceph-mon[32667]: pgmap v232: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:34.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:34 smithi071 ceph-mon[32667]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:34.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:34 smithi045 ceph-mon[29194]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:35.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:34 smithi122 ceph-mon[32959]: pgmap v233: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:29:36.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:36 smithi071 ceph-mon[32667]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:36.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:36 smithi045 ceph-mon[29194]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:37.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:36 smithi122 ceph-mon[32959]: pgmap v234: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:29:37.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:29:37.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:29:37.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 -- 172.21.15.45:0/349736947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d10100e60 msgr2=0x7f3d1010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 --2- 172.21.15.45:0/349736947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d10100e60 0x7f3d1010dbb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009900 tx=0x7f3d0402a410 comp rx=0 tx=0).stop 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 -- 172.21.15.45:0/349736947 shutdown_connections 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 --2- 172.21.15.45:0/349736947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1010e0f0 0x7f3d101104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 --2- 172.21.15.45:0/349736947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d10100e60 0x7f3d1010dbb0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 --2- 172.21.15.45:0/349736947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100540 0x7f3d10100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 -- 172.21.15.45:0/349736947 >> 172.21.15.45:0/349736947 conn(0x7f3d10077740 msgr2=0x7f3d10077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 -- 172.21.15.45:0/349736947 shutdown_connections 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.515+0000 7f3d1482a640 1 -- 172.21.15.45:0/349736947 wait complete. 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 Processor -- start 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 -- start start 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d10100540 0x7f3d1019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 0x7f3d1019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10113d20 con 0x7f3d1010e0f0 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10113ba0 con 0x7f3d10100540 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d1482a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10113ea0 con 0x7f3d10100e60 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34772/0 (socket says 172.21.15.45:34772) 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.516+0000 7f3d0ed9d640 1 -- 172.21.15.45:0/1548323607 learned_addr learned my addr 172.21.15.45:0/1548323607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d10100540 0x7f3d1019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 0x7f3d1019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 -- 172.21.15.45:0/1548323607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 msgr2=0x7f3d1019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 0x7f3d1019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 -- 172.21.15.45:0/1548323607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d10100540 msgr2=0x7f3d1019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d10100540 0x7f3d1019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 -- 172.21.15.45:0/1548323607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1019f170 con 0x7f3d1010e0f0 2024-04-25T00:29:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 0x7f3d1019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d0ed9d640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c970 tx=0x7f3d0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3cff7fe640 1 -- 172.21.15.45:0/1548323607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004230 con 0x7f3d1010e0f0 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3cff7fe640 1 -- 172.21.15.45:0/1548323607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d080043d0 con 0x7f3d1010e0f0 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3cff7fe640 1 -- 172.21.15.45:0/1548323607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800f450 con 0x7f3d1010e0f0 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1019f460 con 0x7f3d1010e0f0 2024-04-25T00:29:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.517+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a6cf0 con 0x7f3d1010e0f0 2024-04-25T00:29:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.519+0000 7f3cff7fe640 1 -- 172.21.15.45:0/1548323607 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3d080079b0 con 0x7f3d1010e0f0 2024-04-25T00:29:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 Processor -- start 2024-04-25T00:29:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 -- start start 2024-04-25T00:29:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3cff7fe640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 0x7f3cf80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b959ea26b0 con 0x55b959f48620 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b959e4a880 con 0x55b959f3d630 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d17611740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b959e9e670 con 0x55b959f3df50 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d0ed9d640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 0x7f3cf80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.520+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34780/0 (socket says 172.21.15.45:34780) 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/147935755 learned_addr learned my addr 172.21.15.45:0/147935755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3cff7fe640 1 -- 172.21.15.45:0/1548323607 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f3d080cf5c0 con 0x7f3d1010e0f0 2024-04-25T00:29:37.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/147935755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 msgr2=0x55b959f480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/147935755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 msgr2=0x55b959f3da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f3da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/147935755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b959c807d0 con 0x55b959f48620 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0ed9d640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 0x7f3cf80a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d08098090 tx=0x7f3d08090b50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3d0402a930 tx=0x7f3d04031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe139b9f7a646b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3cfcff9640 1 -- 172.21.15.45:0/147935755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04035040 con 0x55b959f48620 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3cfcff9640 1 -- 172.21.15.45:0/147935755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d040319b0 con 0x55b959f48620 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.521+0000 7f3cfcff9640 1 -- 172.21.15.45:0/147935755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04004ae0 con 0x55b959f48620 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- 172.21.15.45:0/147935755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 msgr2=0x55b959f4aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3d0402a930 tx=0x7f3d04031400 comp rx=0 tx=0).stop 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- 172.21.15.45:0/147935755 shutdown_connections 2024-04-25T00:29:37.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f4aa10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 --2- 172.21.15.45:0/147935755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- 172.21.15.45:0/147935755 >> 172.21.15.45:0/147935755 conn(0x55b959f39950 msgr2=0x55b959f39d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- 172.21.15.45:0/147935755 shutdown_connections 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- 172.21.15.45:0/147935755 wait complete. 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.522+0000 7f3d17611740 1 -- start start 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 Processor -- start 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f45ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b959f4b3b0 con 0x55b959f48620 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b959e5df50 con 0x55b959f3d630 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b959f4b530 con 0x55b959f3df50 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d17611740 10 client.0 _ll_register_callbacks cb 0x55b959e99000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34790/0 (socket says 172.21.15.45:34790) 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/4290825473 learned_addr learned my addr 172.21.15.45:0/4290825473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f45ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/4290825473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 msgr2=0x55b959f45ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f45ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 msgr2=0x55b959f455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/4290825473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959bb63f0 con 0x55b959f48620 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.523+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f45ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3d04001eb0 tx=0x7f3d040045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28599ebf8650687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04035070 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f3d04004a80 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d040319b0 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d04004c80 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 10 client.17757 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b959b760f0 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d040319b0 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d0403a040 con 0x55b959f48620 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 10 client.17757 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 20 client.17757 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959f58710 0x55b959f5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 4 client.17757 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.524+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b959eafbf0 con 0x55b959f58710 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.525+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959f58710 0x55b959f5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.525+0000 7f3cf5ffb640 10 client.17757 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.525+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959f58710 0x55b959f5ab00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fc2b3e2706e7b3 server_cookie=2ae6f0fa94128737 in_seq=0 out_seq=0 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/4290825473 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b959eafbf0 con 0x55b959f58710 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3cf5ffb640 10 client.17757 handle_command_reply: tid=0 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 client.17757 shutdown 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 msgr2=0x55b959f46020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3d04001eb0 tx=0x7f3d040045d0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 shutdown_connections 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959f58710 0x55b959f5ab00 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f46020 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f45ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 --2- 172.21.15.45:0/4290825473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3cf5ffb640 0 client.17757 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 >> 172.21.15.45:0/4290825473 conn(0x55b959f39950 msgr2=0x55b959f3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 shutdown_connections 2024-04-25T00:29:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 1 -- 172.21.15.45:0/4290825473 wait complete. 2024-04-25T00:29:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.526+0000 7f3d17611740 20 client.17757 trim_cache size 0 max 16384 2024-04-25T00:29:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 Processor -- start 2024-04-25T00:29:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 -- start start 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f43d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b0d50 con 0x55b959f48620 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b2890 con 0x55b959f3d630 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d17611740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b1420 con 0x55b959f3df50 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34798/0 (socket says 172.21.15.45:34798) 2024-04-25T00:29:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1497144643 learned_addr learned my addr 172.21.15.45:0/1497144643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f43d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1497144643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 msgr2=0x55b959f43d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f43d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1497144643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 msgr2=0x55b959f47cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f47cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.544+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1497144643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b959f2b090 con 0x55b959f48620 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005740 tx=0x7f3d04031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd9607e71390f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3cf7fff640 1 -- 172.21.15.45:0/1497144643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04035040 con 0x55b959f48620 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3cf7fff640 1 -- 172.21.15.45:0/1497144643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04009520 con 0x55b959f48620 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3cf7fff640 1 -- 172.21.15.45:0/1497144643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04031bf0 con 0x55b959f48620 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 -- 172.21.15.45:0/1497144643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 msgr2=0x55b959f3fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005740 tx=0x7f3d04031400 comp rx=0 tx=0).stop 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 -- 172.21.15.45:0/1497144643 shutdown_connections 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f3fe80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f43d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 --2- 172.21.15.45:0/1497144643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f47cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 -- 172.21.15.45:0/1497144643 >> 172.21.15.45:0/1497144643 conn(0x55b959f39950 msgr2=0x55b959f54010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 -- 172.21.15.45:0/1497144643 shutdown_connections 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.545+0000 7f3d17611740 1 -- 172.21.15.45:0/1497144643 wait complete. 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 -- start start 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 Processor -- start 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f5dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b21c0 con 0x55b959f48620 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101b1af0 con 0x55b959f3d630 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101aed50 con 0x55b959f3df50 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d17611740 10 client.0 _ll_register_callbacks cb 0x7f3d04032e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34812/0 (socket says 172.21.15.45:34812) 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1793948470 learned_addr learned my addr 172.21.15.45:0/1793948470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f5dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1793948470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 msgr2=0x55b959f5b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f5b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 msgr2=0x55b959f5dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f5dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.546+0000 7f3d0dd9b640 1 -- 172.21.15.45:0/1793948470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b959f5c190 con 0x55b959f48620 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d0dd9b640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004650 tx=0x7f3d04004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0e360e61c87de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04035040 con 0x55b959f48620 2024-04-25T00:29:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f3d04031970 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d04009520 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d04031b70 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 10 client.17763 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b959fbea00 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04009520 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3d04049440 con 0x55b959f48620 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 10 client.17763 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 20 client.17763 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959fbeed0 0x55b959fc12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 4 client.17763 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.547+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b959ea9f40 con 0x55b959fbeed0 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959fbeed0 0x55b959fc12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3cf5ffb640 10 client.17763 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d0e59c640 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959fbeed0 0x55b959fc12c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab6aa4a71f31f27 server_cookie=bbde84022be23e1f in_seq=0 out_seq=0 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3cf5ffb640 1 -- 172.21.15.45:0/1793948470 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b959ea9f40 con 0x55b959fbeed0 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3cf5ffb640 10 client.17763 handle_command_reply: tid=0 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 client.17763 shutdown 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 msgr2=0x55b959f5b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004650 tx=0x7f3d04004be0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 shutdown_connections 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55b959fbeed0 0x55b959fc12c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b959f48620 0x55b959f5b990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b959f3df50 0x55b959f5b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 --2- 172.21.15.45:0/1793948470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b959f3d630 0x55b959f5dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3cf5ffb640 0 client.17763 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 >> 172.21.15.45:0/1793948470 conn(0x55b959f39950 msgr2=0x55b959f45560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 shutdown_connections 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 1 -- 172.21.15.45:0/1793948470 wait complete. 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.548+0000 7f3d17611740 20 client.17763 trim_cache size 0 max 16384 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 msgr2=0x7f3cf80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 0x7f3cf80a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d08098090 tx=0x7f3d08090b50 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 msgr2=0x7f3d1019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c970 tx=0x7f3d0800ce30 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 shutdown_connections 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f3cf80a2d00 0x7f3cf80a51c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d1010e0f0 0x7f3d1019ea40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d10100e60 0x7f3d1019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 --2- 172.21.15.45:0/1548323607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d10100540 0x7f3d1019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 >> 172.21.15.45:0/1548323607 conn(0x7f3d10077740 msgr2=0x7f3d1010f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 shutdown_connections 2024-04-25T00:29:37.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.549+0000 7f3d1482a640 1 -- 172.21.15.45:0/1548323607 wait complete. 2024-04-25T00:29:37.564 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:29:37.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3866306380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64101050 msgr2=0x7fce6410d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3866306380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64101050 0x7fce6410d510 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009cd0 tx=0x7fce5c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3866306380 shutdown_connections 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3866306380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64101050 0x7fce6410d510 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3866306380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64100690 0x7fce64100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3866306380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64106650 0x7fce64106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3866306380 >> 172.21.15.45:0/3866306380 conn(0x7fce640777a0 msgr2=0x7fce64077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3866306380 shutdown_connections 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.666+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3866306380 wait complete. 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 Processor -- start 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 -- start start 2024-04-25T00:29:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64100690 0x7fce64198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 0x7fce64199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce64110a60 con 0x7fce64106650 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce641108e0 con 0x7fce64101050 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6b6ae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce64110be0 con 0x7fce64100690 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34828/0 (socket says 172.21.15.45:34828) 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 -- 172.21.15.45:0/3549691748 learned_addr learned my addr 172.21.15.45:0/3549691748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce6944a640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64100690 0x7fce64198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce68c49640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 0x7fce64199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 -- 172.21.15.45:0/3549691748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64100690 msgr2=0x7fce64198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64100690 0x7fce64198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 -- 172.21.15.45:0/3549691748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 msgr2=0x7fce64199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 0x7fce64199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce69c4b640 1 -- 172.21.15.45:0/3549691748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641a43a0 con 0x7fce64106650 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.667+0000 7fce68c49640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 0x7fce64199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce69c4b640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005cc0 tx=0x7fce5c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004040 con 0x7fce64106650 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c0046c0 con 0x7fce64106650 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641a4570 con 0x7fce64106650 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c030e00 con 0x7fce64106650 2024-04-25T00:29:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.668+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641a4a60 con 0x7fce64106650 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.669+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce640689b0 con 0x7fce64106650 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.670+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce5c0305a0 con 0x7fce64106650 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.671+0000 7fce5a7fc640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 0x7fce500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.671+0000 7fce6944a640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 0x7fce500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.671+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7fce5c02e030 con 0x7fce64106650 2024-04-25T00:29:37.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.671+0000 7fce6944a640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 0x7fce500a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce6000b510 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:37.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.675+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce5c04b060 con 0x7fce64106650 2024-04-25T00:29:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.898+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce64108bb0 con 0x7fce64106650 2024-04-25T00:29:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.899+0000 7fce5a7fc640 1 -- 172.21.15.45:0/3549691748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fce5c0bda70 con 0x7fce64106650 2024-04-25T00:29:37.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:29:37.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:29:37.900 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 msgr2=0x7fce500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 0x7fce500a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce6000b510 comp rx=0 tx=0).stop 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 msgr2=0x7fce641a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005cc0 tx=0x7fce5c032f40 comp rx=0 tx=0).stop 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 shutdown_connections 2024-04-25T00:29:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce500a2d00 0x7fce500a51c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce64106650 0x7fce641a3e60 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce64101050 0x7fce64199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 --2- 172.21.15.45:0/3549691748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce64100690 0x7fce64198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 >> 172.21.15.45:0/3549691748 conn(0x7fce640777a0 msgr2=0x7fce64103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 shutdown_connections 2024-04-25T00:29:37.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:37.901+0000 7fce6b6ae640 1 -- 172.21.15.45:0/3549691748 wait complete. 2024-04-25T00:29:37.913 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 -- 172.21.15.45:0/2843561382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8107710 msgr2=0x7ff0a8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 --2- 172.21.15.45:0/2843561382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8107710 0x7ff0a8107af0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009930 tx=0x7ff0a402a420 comp rx=0 tx=0).stop 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 -- 172.21.15.45:0/2843561382 shutdown_connections 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 --2- 172.21.15.45:0/2843561382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a81020c0 0x7ff0a810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 --2- 172.21.15.45:0/2843561382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8101700 0x7ff0a8101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 --2- 172.21.15.45:0/2843561382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8107710 0x7ff0a8107af0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 -- 172.21.15.45:0/2843561382 >> 172.21.15.45:0/2843561382 conn(0x7ff0a80fd1b0 msgr2=0x7ff0a80ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 -- 172.21.15.45:0/2843561382 shutdown_connections 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.019+0000 7ff0af990640 1 -- 172.21.15.45:0/2843561382 wait complete. 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 Processor -- start 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 -- start start 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a81020c0 0x7ff0a8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a8107710 0x7ff0a819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111b20 con 0x7ff0a8101700 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81119a0 con 0x7ff0a81020c0 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0af990640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8111ca0 con 0x7ff0a8107710 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34844/0 (socket says 172.21.15.45:34844) 2024-04-25T00:29:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3376698226 learned_addr learned my addr 172.21.15.45:0/3376698226 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a81020c0 0x7ff0a8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a8107710 0x7ff0a819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3376698226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a8107710 msgr2=0x7ff0a819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a8107710 0x7ff0a819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3376698226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a81020c0 msgr2=0x7ff0a8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a81020c0 0x7ff0a8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.020+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3376698226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a819d030 con 0x7ff0a8101700 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009230 tx=0x7ff0a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff09e7fc640 1 -- 172.21.15.45:0/3376698226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a403f030 con 0x7ff0a8101700 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff09e7fc640 1 -- 172.21.15.45:0/3376698226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4003ad0 con 0x7ff0a8101700 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a819d2c0 con 0x7ff0a8101700 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff09e7fc640 1 -- 172.21.15.45:0/3376698226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4035070 con 0x7ff0a8101700 2024-04-25T00:29:38.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.021+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81a4bb0 con 0x7ff0a8101700 2024-04-25T00:29:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff09e7fc640 1 -- 172.21.15.45:0/3376698226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0a4042030 con 0x7ff0a8101700 2024-04-25T00:29:38.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 Processor -- start 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 -- start start 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf50b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf50d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5643cf465670 con 0x5643cf500580 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5643cf40d970 con 0x5643cf50b570 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0b277e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643cf461630 con 0x5643cf500ea0 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0adf2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0adf2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34848/0 (socket says 172.21.15.45:34848) 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.024+0000 7ff0adf2d640 1 -- 172.21.15.45:0/2790201761 learned_addr learned my addr 172.21.15.45:0/2790201761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf50b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf50d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 -- 172.21.15.45:0/2790201761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 msgr2=0x5643cf50b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf50b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff09e7fc640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 0x7ff0940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 -- 172.21.15.45:0/2790201761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 msgr2=0x5643cf50d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf50d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 -- 172.21.15.45:0/2790201761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf431400 con 0x5643cf500580 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff09e7fc640 1 -- 172.21.15.45:0/3376698226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7ff0a40efd10 con 0x7ff0a8101700 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009d70 tx=0x7ff0a000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3740960f0575c94f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 0x7ff0940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0937fe640 1 -- 172.21.15.45:0/2790201761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0003b70 con 0x5643cf500580 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0937fe640 1 -- 172.21.15.45:0/2790201761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0004510 con 0x5643cf500580 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.025+0000 7ff0937fe640 1 -- 172.21.15.45:0/2790201761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0011d50 con 0x5643cf500580 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- 172.21.15.45:0/2790201761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 msgr2=0x5643cf500960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 0x7ff0940a51c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0019000 tx=0x7ff0a000d830 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009d70 tx=0x7ff0a000f730 comp rx=0 tx=0).stop 2024-04-25T00:29:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- 172.21.15.45:0/2790201761 shutdown_connections 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf50d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf50b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2790201761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf500960 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- 172.21.15.45:0/2790201761 >> 172.21.15.45:0/2790201761 conn(0x5643cf4fc8a0 msgr2=0x5643cf4fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- 172.21.15.45:0/2790201761 shutdown_connections 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- 172.21.15.45:0/2790201761 wait complete. 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 -- start start 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.026+0000 7ff0b277e740 1 Processor -- start 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf508170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5643cf50e360 con 0x5643cf500580 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5643cf421050 con 0x5643cf50b570 2024-04-25T00:29:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5643cf50e4e0 con 0x5643cf500ea0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0b277e740 10 client.0 _ll_register_callbacks cb 0x5643cf178be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34864/0 (socket says 172.21.15.45:34864) 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 -- 172.21.15.45:0/2358674615 learned_addr learned my addr 172.21.15.45:0/2358674615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf508170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 -- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 msgr2=0x5643cf508170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf508170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 -- 172.21.15.45:0/2358674615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 msgr2=0x5643cf5086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 -- 172.21.15.45:0/2358674615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf3f2dc0 con 0x5643cf500580 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.027+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009a50 tx=0x7ff0a40b81d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687a99b8701b8c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4049070 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7ff0a4003c80 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0004510 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a40bb7d0 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 10 client.17781 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643cf1793f0 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0004510 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0a40b84e0 con 0x5643cf500580 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 10 client.17781 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 10 client.17781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 20 client.17781 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf51b400 0x5643cf51d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 4 client.17781 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.028+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cf472bb0 con 0x5643cf51b400 2024-04-25T00:29:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.029+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf51b400 0x5643cf51d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.029+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf51b400 0x5643cf51d870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8b7f229c61aecee server_cookie=f9c245e51bf0772b in_seq=0 out_seq=0 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.029+0000 7ff090ff9640 10 client.17781 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff090ff9640 1 -- 172.21.15.45:0/2358674615 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643cf472bb0 con 0x5643cf51b400 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff090ff9640 10 client.17781 handle_command_reply: tid=0 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 client.17781 shutdown 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 msgr2=0x5643cf50ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009a50 tx=0x7ff0a40b81d0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 shutdown_connections 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf51b400 0x5643cf51d870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf508170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff090ff9640 0 client.17781 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 --2- 172.21.15.45:0/2358674615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf50ae80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 >> 172.21.15.45:0/2358674615 conn(0x5643cf4fc8a0 msgr2=0x5643cf4fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 shutdown_connections 2024-04-25T00:29:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 1 -- 172.21.15.45:0/2358674615 wait complete. 2024-04-25T00:29:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.030+0000 7ff0b277e740 20 client.17781 trim_cache size 0 max 16384 2024-04-25T00:29:38.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.043+0000 7ff0b277e740 1 Processor -- start 2024-04-25T00:29:38.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.043+0000 7ff0b277e740 1 -- start start 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500580 0x5643cf517220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81aeb90 con 0x5643cf500ea0 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b06d0 con 0x5643cf50b570 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0b277e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81af260 con 0x5643cf500580 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34870/0 (socket says 172.21.15.45:34870) 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 -- 172.21.15.45:0/1807851473 learned_addr learned my addr 172.21.15.45:0/1807851473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500580 0x5643cf517220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 -- 172.21.15.45:0/1807851473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500580 msgr2=0x5643cf517220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500580 0x5643cf517220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 -- 172.21.15.45:0/1807851473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 msgr2=0x5643cf5236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 -- 172.21.15.45:0/1807851473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf2437d0 con 0x5643cf500ea0 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff09800bdd0 tx=0x7ff09800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747f12ba33529927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff092ffd640 1 -- 172.21.15.45:0/1807851473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098004220 con 0x5643cf500ea0 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff092ffd640 1 -- 172.21.15.45:0/1807851473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff098023070 con 0x5643cf500ea0 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.044+0000 7ff092ffd640 1 -- 172.21.15.45:0/1807851473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09801f1b0 con 0x5643cf500ea0 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- 172.21.15.45:0/1807851473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 msgr2=0x5643cf523160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff09800bdd0 tx=0x7ff09800b410 comp rx=0 tx=0).stop 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- 172.21.15.45:0/1807851473 shutdown_connections 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf5236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500ea0 0x5643cf523160 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 --2- 172.21.15.45:0/1807851473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500580 0x5643cf517220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- 172.21.15.45:0/1807851473 >> 172.21.15.45:0/1807851473 conn(0x5643cf4fc8a0 msgr2=0x5643cf4fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- 172.21.15.45:0/1807851473 shutdown_connections 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- 172.21.15.45:0/1807851473 wait complete. 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 -- start start 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.045+0000 7ff0b277e740 1 Processor -- start 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf511010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf511750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81b0000 con 0x5643cf500580 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81af930 con 0x5643cf50b570 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a81acb90 con 0x5643cf500ea0 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0b277e740 10 client.0 _ll_register_callbacks cb 0x7ff0a40f3550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34880/0 (socket says 172.21.15.45:34880) 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3108897385 learned_addr learned my addr 172.21.15.45:0/3108897385 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf511010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0adf2d640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf511750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 msgr2=0x5643cf511010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf511010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3108897385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 msgr2=0x5643cf511750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf511750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 -- 172.21.15.45:0/3108897385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf5809e0 con 0x5643cf500580 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff0ad72c640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40f0100 tx=0x7ff0a40b8240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc96f8bc4504b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.046+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4049070 con 0x5643cf500580 2024-04-25T00:29:38.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7ff0a40c4650 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff098023070 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a40bc570 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 10 client.17787 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643cf5809e0 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff098023070 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0a403bb90 con 0x5643cf500580 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 10 client.17787 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 10 client.17787 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 20 client.17787 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf581e50 0x5643cf584240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 4 client.17787 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643cf46cf00 con 0x5643cf581e50 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.047+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf581e50 0x5643cf584240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0acf2b640 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf581e50 0x5643cf584240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5de4118a58e85871 server_cookie=d45c41a298448aca in_seq=0 out_seq=0 2024-04-25T00:29:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff090ff9640 10 client.17787 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff090ff9640 1 -- 172.21.15.45:0/3108897385 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643cf46cf00 con 0x5643cf581e50 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff090ff9640 10 client.17787 handle_command_reply: tid=0 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 client.17787 shutdown 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 msgr2=0x5643cf513c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40f0100 tx=0x7ff0a40b8240 comp rx=0 tx=0).stop 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 shutdown_connections 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5643cf581e50 0x5643cf584240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5643cf50b570 0x5643cf511750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff090ff9640 0 client.17787 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5643cf500ea0 0x5643cf511010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 --2- 172.21.15.45:0/3108897385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5643cf500580 0x5643cf513c80 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 >> 172.21.15.45:0/3108897385 conn(0x5643cf4fc8a0 msgr2=0x5643cf516210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 shutdown_connections 2024-04-25T00:29:38.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.048+0000 7ff0b277e740 1 -- 172.21.15.45:0/3108897385 wait complete. 2024-04-25T00:29:38.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0b277e740 20 client.17787 trim_cache size 0 max 16384 2024-04-25T00:29:38.057 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 msgr2=0x7ff0940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 0x7ff0940a51c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0019000 tx=0x7ff0a000d830 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 msgr2=0x7ff0a8198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009230 tx=0x7ff0a4004270 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 shutdown_connections 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff0940a2d00 0x7ff0940a51c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0a8107710 0x7ff0a819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a81020c0 0x7ff0a8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 --2- 172.21.15.45:0/3376698226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0a8101700 0x7ff0a8198040 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 >> 172.21.15.45:0/3376698226 conn(0x7ff0a80fd1b0 msgr2=0x7ff0a80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 shutdown_connections 2024-04-25T00:29:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.049+0000 7ff0af990640 1 -- 172.21.15.45:0/3376698226 wait complete. 2024-04-25T00:29:38.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- 172.21.15.45:0/2771688987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a0100fd0 msgr2=0x7f97a010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 --2- 172.21.15.45:0/2771688987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a0100fd0 0x7f97a010d4e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f9798009cd0 tx=0x7f979802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- 172.21.15.45:0/2771688987 shutdown_connections 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 --2- 172.21.15.45:0/2771688987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a0100fd0 0x7f97a010d4e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 --2- 172.21.15.45:0/2771688987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100610 0x7f97a0100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 --2- 172.21.15.45:0/2771688987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a01066b0 0x7f97a0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- 172.21.15.45:0/2771688987 >> 172.21.15.45:0/2771688987 conn(0x7f97a0077740 msgr2=0x7f97a0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- 172.21.15.45:0/2771688987 shutdown_connections 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- 172.21.15.45:0/2771688987 wait complete. 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 Processor -- start 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.164+0000 7f97a6438640 1 -- start start 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 0x7f97a0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100fd0 0x7f97a0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01109f0 con 0x7f97a01066b0 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0110870 con 0x7f97a0100fd0 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a6438640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0110b70 con 0x7f97a0100610 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34896/0 (socket says 172.21.15.45:34896) 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 -- 172.21.15.45:0/3773822431 learned_addr learned my addr 172.21.15.45:0/3773822431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f979f7fe640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100fd0 0x7f97a0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f979ffff640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 0x7f97a0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 -- 172.21.15.45:0/3773822431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 msgr2=0x7f97a0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 0x7f97a0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 -- 172.21.15.45:0/3773822431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100fd0 msgr2=0x7f97a0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100fd0 0x7f97a0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 -- 172.21.15.45:0/3773822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a019cef0 con 0x7f97a01066b0 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f979ffff640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 0x7f97a0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.165+0000 7f97a49d5640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9798005cc0 tx=0x7f9798032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.166+0000 7f979d7fa640 1 -- 172.21.15.45:0/3773822431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798004040 con 0x7f97a01066b0 2024-04-25T00:29:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.166+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a019d180 con 0x7f97a01066b0 2024-04-25T00:29:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.166+0000 7f979d7fa640 1 -- 172.21.15.45:0/3773822431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97980046c0 con 0x7f97a01066b0 2024-04-25T00:29:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.166+0000 7f979d7fa640 1 -- 172.21.15.45:0/3773822431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798030e00 con 0x7f97a01066b0 2024-04-25T00:29:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.166+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a01a4a70 con 0x7f97a01066b0 2024-04-25T00:29:38.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f979d7fa640 1 -- 172.21.15.45:0/3773822431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f97980305a0 con 0x7f97a01066b0 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 Processor -- start 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 -- start start 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830c9980 0x55d0830cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d08301f670 con 0x55d0830ba480 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d082fc7970 con 0x55d0830c9980 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f97a9222740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d08301b630 con 0x55d0830bada0 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.168+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979d7fa640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 0x7f97880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34908/0 (socket says 172.21.15.45:34908) 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 -- 172.21.15.45:0/3778004806 learned_addr learned my addr 172.21.15.45:0/3778004806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979ffff640 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f97a49d5640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 0x7f97880a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830c9980 0x55d0830cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 -- 172.21.15.45:0/3778004806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 msgr2=0x55d0830c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 -- 172.21.15.45:0/3778004806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830c9980 msgr2=0x55d0830cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830c9980 0x55d0830cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 -- 172.21.15.45:0/3778004806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d082feb400 con 0x55d0830ba480 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979d7fa640 1 -- 172.21.15.45:0/3773822431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f979802e030 con 0x7f97a01066b0 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f979f7fe640 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9794013620 tx=0x7f9794009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6ef79e40cd8463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f97a49d5640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 0x7f97880a51c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97980bce70 tx=0x7f97980e7d00 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f97867fc640 1 -- 172.21.15.45:0/3778004806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794004050 con 0x55d0830ba480 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f97867fc640 1 -- 172.21.15.45:0/3778004806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97940041f0 con 0x55d0830ba480 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.169+0000 7f97867fc640 1 -- 172.21.15.45:0/3778004806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400f7c0 con 0x55d0830ba480 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- 172.21.15.45:0/3778004806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 msgr2=0x55d0830ba860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9794013620 tx=0x7f9794009850 comp rx=0 tx=0).stop 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- 172.21.15.45:0/3778004806 shutdown_connections 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830c9980 0x55d0830cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 --2- 172.21.15.45:0/3778004806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830ba860 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- 172.21.15.45:0/3778004806 >> 172.21.15.45:0/3778004806 conn(0x55d0830b68a0 msgr2=0x55d0830b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- 172.21.15.45:0/3778004806 shutdown_connections 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- 172.21.15.45:0/3778004806 wait complete. 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 -- start start 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.170+0000 7f97a9222740 1 Processor -- start 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830bada0 0x55d0830c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0830cc6e0 con 0x55d0830ba480 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d082fdb050 con 0x55d0830bada0 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d0830cc860 con 0x55d0830c9980 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 10 client.0 _ll_register_callbacks cb 0x55d082d32be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34916/0 (socket says 172.21.15.45:34916) 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 -- 172.21.15.45:0/1488523267 learned_addr learned my addr 172.21.15.45:0/1488523267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979ffff640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830bada0 0x55d0830c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 -- 172.21.15.45:0/1488523267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 msgr2=0x55d0830c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 -- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830bada0 msgr2=0x55d0830c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830bada0 0x55d0830c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 -- 172.21.15.45:0/1488523267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d082facdc0 con 0x55d0830ba480 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f979f7fe640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9794005890 tx=0x7f9794003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab0eaf5df607c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f977f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979402e060 con 0x55d0830ba480 2024-04-25T00:29:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f979400f580 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.171+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97940041f0 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9794035050 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 10 client.17799 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d082d333f0 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97940041f0 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9794028030 con 0x55d0830ba480 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 10 client.17799 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 10 client.17799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 20 client.17799 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0830d97a0 0x55d0830dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 4 client.17799 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d08302cbb0 con 0x55d0830d97a0 2024-04-25T00:29:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.172+0000 7f979ffff640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0830d97a0 0x55d0830dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.173+0000 7f979ffff640 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0830d97a0 0x55d0830dbb90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56f52e8819b06b1b server_cookie=680fc31ba5672b6f in_seq=0 out_seq=0 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.173+0000 7f977f7fe640 10 client.17799 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.173+0000 7f977f7fe640 1 -- 172.21.15.45:0/1488523267 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d08302cbb0 con 0x55d0830d97a0 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.173+0000 7f977f7fe640 10 client.17799 handle_command_reply: tid=0 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.173+0000 7f97a9222740 1 client.17799 shutdown 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 msgr2=0x55d0830c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9794005890 tx=0x7f9794003b70 comp rx=0 tx=0).stop 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 shutdown_connections 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0830d97a0 0x55d0830dbb90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f977f7fe640 0 client.17799 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830bada0 0x55d0830c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 --2- 172.21.15.45:0/1488523267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830ba480 0x55d0830c9240 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 >> 172.21.15.45:0/1488523267 conn(0x55d0830b68a0 msgr2=0x55d0830b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 shutdown_connections 2024-04-25T00:29:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 1 -- 172.21.15.45:0/1488523267 wait complete. 2024-04-25T00:29:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.174+0000 7f97a9222740 20 client.17799 trim_cache size 0 max 16384 2024-04-25T00:29:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 Processor -- start 2024-04-25T00:29:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 -- start start 2024-04-25T00:29:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01ac880 con 0x55d0830c9980 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01ae3c0 con 0x55d0830ba480 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f97a9222740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01acf50 con 0x55d0830bada0 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34932/0 (socket says 172.21.15.45:34932) 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 -- 172.21.15.45:0/430109302 learned_addr learned my addr 172.21.15.45:0/430109302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979ffff640 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 -- 172.21.15.45:0/430109302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 msgr2=0x55d0830c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 -- 172.21.15.45:0/430109302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 msgr2=0x55d0830c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.187+0000 7f979f7fe640 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f979f7fe640 1 -- 172.21.15.45:0/430109302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0830babd0 con 0x55d0830c9980 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f979f7fe640 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f9794016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170e39e4f1d4174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f9785ffb640 1 -- 172.21.15.45:0/430109302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794003be0 con 0x55d0830c9980 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f9785ffb640 1 -- 172.21.15.45:0/430109302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97940131b0 con 0x55d0830c9980 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f9785ffb640 1 -- 172.21.15.45:0/430109302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400fa60 con 0x55d0830c9980 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 -- 172.21.15.45:0/430109302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 msgr2=0x55d0830e13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f9794016040 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 -- 172.21.15.45:0/430109302 shutdown_connections 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830c9980 0x55d0830e13f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830bada0 0x55d0830c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 --2- 172.21.15.45:0/430109302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 -- 172.21.15.45:0/430109302 >> 172.21.15.45:0/430109302 conn(0x55d0830b68a0 msgr2=0x55d0830bbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 -- 172.21.15.45:0/430109302 shutdown_connections 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.188+0000 7f97a9222740 1 -- 172.21.15.45:0/430109302 wait complete. 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 -- start start 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 Processor -- start 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01adcf0 con 0x55d0830bada0 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01ad620 con 0x55d0830ba480 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01aa880 con 0x55d0830c9980 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f97a9222740 10 client.0 _ll_register_callbacks cb 0x7f9794013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f979ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34938/0 (socket says 172.21.15.45:34938) 2024-04-25T00:29:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.189+0000 7f979ffff640 1 -- 172.21.15.45:0/2540814427 learned_addr learned my addr 172.21.15.45:0/2540814427 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979f7fe640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 -- 172.21.15.45:0/2540814427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 msgr2=0x55d0830cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 -- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 msgr2=0x55d0830d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 -- 172.21.15.45:0/2540814427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d082dfd7d0 con 0x55d0830bada0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979f7fe640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f979ffff640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9790004700 tx=0x7f9790012970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e615b9f1a077b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979001e520 con 0x55d0830bada0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f979001e6c0 con 0x55d0830bada0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97940131b0 con 0x55d0830bada0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f979002f050 con 0x55d0830bada0 2024-04-25T00:29:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f97a9222740 10 client.17805 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d08313fcc0 con 0x55d0830bada0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97940131b0 con 0x55d0830bada0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.190+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9790027490 con 0x55d0830bada0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 10 client.17805 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 10 client.17805 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 20 client.17805 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0831404c0 0x55d0831428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 4 client.17805 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d083026f00 con 0x55d0831404c0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f979f7fe640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0831404c0 0x55d0831428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f979f7fe640 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0831404c0 0x55d0831428e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86e5d1201874292a server_cookie=5587b1d12938b7be in_seq=0 out_seq=0 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f977f7fe640 10 client.17805 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f977f7fe640 1 -- 172.21.15.45:0/2540814427 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d083026f00 con 0x55d0831404c0 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f977f7fe640 10 client.17805 handle_command_reply: tid=0 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 1 client.17805 shutdown 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 msgr2=0x55d0830cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.191+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9790004700 tx=0x7f9790012970 comp rx=0 tx=0).stop 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 shutdown_connections 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55d0831404c0 0x55d0831428e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d0830c9980 0x55d0830cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f977f7fe640 0 client.17805 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0830bada0 0x55d0830cd430 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 --2- 172.21.15.45:0/2540814427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0830ba480 0x55d0830d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 >> 172.21.15.45:0/2540814427 conn(0x55d0830b68a0 msgr2=0x55d0830c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 shutdown_connections 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 1 -- 172.21.15.45:0/2540814427 wait complete. 2024-04-25T00:29:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a9222740 20 client.17805 trim_cache size 0 max 16384 2024-04-25T00:29:38.200 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 msgr2=0x7f97880a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 0x7f97880a51c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97980bce70 tx=0x7f97980e7d00 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 msgr2=0x7f97a019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9798005cc0 tx=0x7f9798032f40 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 shutdown_connections 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97880a2d00 0x7f97880a51c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97a01066b0 0x7f97a019c7c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a0100fd0 0x7f97a0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 --2- 172.21.15.45:0/3773822431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97a0100610 0x7f97a0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 >> 172.21.15.45:0/3773822431 conn(0x7f97a0077740 msgr2=0x7f97a00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 shutdown_connections 2024-04-25T00:29:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.192+0000 7f97a6438640 1 -- 172.21.15.45:0/3773822431 wait complete. 2024-04-25T00:29:38.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:29:38.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 -- 172.21.15.45:0/182943074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 msgr2=0x7f446c10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 --2- 172.21.15.45:0/182943074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c10e5e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4464009cd0 tx=0x7f446402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 -- 172.21.15.45:0/182943074 shutdown_connections 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 --2- 172.21.15.45:0/182943074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c10e5e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 --2- 172.21.15.45:0/182943074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 0x7f446c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 --2- 172.21.15.45:0/182943074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 0x7f446c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 -- 172.21.15.45:0/182943074 >> 172.21.15.45:0/182943074 conn(0x7f446c0fd180 msgr2=0x7f446c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 -- 172.21.15.45:0/182943074 shutdown_connections 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.307+0000 7f447375a640 1 -- 172.21.15.45:0/182943074 wait complete. 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 Processor -- start 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 -- start start 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 0x7f446c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 0x7f446c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f446c111b30 con 0x7f446c1020d0 2024-04-25T00:29:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1119b0 con 0x7f446c107720 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f447375a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c111cb0 con 0x7f446c101710 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34956/0 (socket says 172.21.15.45:34956) 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f4470cf5640 1 -- 172.21.15.45:0/3889087884 learned_addr learned my addr 172.21.15.45:0/3889087884 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.308+0000 7f44714f6640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 0x7f446c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 0x7f446c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 -- 172.21.15.45:0/3889087884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 msgr2=0x7f446c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 0x7f446c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 -- 172.21.15.45:0/3889087884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 msgr2=0x7f446c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 0x7f446c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 -- 172.21.15.45:0/3889087884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c19d060 con 0x7f446c1020d0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f4470cf5640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d770 tx=0x7f445c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f44627fc640 1 -- 172.21.15.45:0/3889087884 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c017070 con 0x7f446c1020d0 2024-04-25T00:29:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f44627fc640 1 -- 172.21.15.45:0/3889087884 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c005500 con 0x7f446c1020d0 2024-04-25T00:29:38.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f44627fc640 1 -- 172.21.15.45:0/3889087884 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c004e00 con 0x7f446c1020d0 2024-04-25T00:29:38.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c19d350 con 0x7f446c1020d0 2024-04-25T00:29:38.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.309+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1a4be0 con 0x7f446c1020d0 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.311+0000 7f44627fc640 1 -- 172.21.15.45:0/3889087884 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f445c004fa0 con 0x7f446c1020d0 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 Processor -- start 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 -- start start 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f50440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5574e1ea6670 con 0x5574e1f50980 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5574e1e4e970 con 0x5574e1f41da0 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4476468740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5574e1ea2630 con 0x5574e1f41480 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34972/0 (socket says 172.21.15.45:34972) 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 -- 172.21.15.45:0/4106023927 learned_addr learned my addr 172.21.15.45:0/4106023927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f44627fc640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 0x7f44580a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4471cf7640 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f44714f6640 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f50440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 0x7f44580a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 -- 172.21.15.45:0/4106023927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 msgr2=0x5574e1f41860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f41860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 -- 172.21.15.45:0/4106023927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 msgr2=0x5574e1f50440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f50440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f44627fc640 1 -- 172.21.15.45:0/3889087884 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f445c0cf170 con 0x7f446c1020d0 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.312+0000 7f4470cf5640 1 -- 172.21.15.45:0/4106023927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e1e72400 con 0x5574e1f50980 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f4470cf5640 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f445c09b080 tx=0x7f445c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6090300fb27cd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 0x7f44580a5240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4464004ab0 tx=0x7f4464004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f44577fe640 1 -- 172.21.15.45:0/4106023927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c020020 con 0x5574e1f50980 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f44577fe640 1 -- 172.21.15.45:0/4106023927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c09b380 con 0x5574e1f50980 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f44577fe640 1 -- 172.21.15.45:0/4106023927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0cc780 con 0x5574e1f50980 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f4476468740 1 -- 172.21.15.45:0/4106023927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 msgr2=0x5574e1f52d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.313+0000 7f4476468740 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f445c09b080 tx=0x7f445c009c30 comp rx=0 tx=0).stop 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- 172.21.15.45:0/4106023927 shutdown_connections 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f52d70 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f50440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- 172.21.15.45:0/4106023927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f41860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- 172.21.15.45:0/4106023927 >> 172.21.15.45:0/4106023927 conn(0x5574e1f3d8a0 msgr2=0x5574e1f3dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- 172.21.15.45:0/4106023927 shutdown_connections 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- 172.21.15.45:0/4106023927 wait complete. 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- start start 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 Processor -- start 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5574e1f53680 con 0x5574e1f41da0 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5574e1e62050 con 0x5574e1f41480 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5574e1f53800 con 0x5574e1f50980 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.314+0000 7f4476468740 10 client.0 _ll_register_callbacks cb 0x5574e1bb9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34976/0 (socket says 172.21.15.45:34976) 2024-04-25T00:29:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 -- 172.21.15.45:0/2698330418 learned_addr learned my addr 172.21.15.45:0/2698330418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f44714f6640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4471cf7640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 -- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 msgr2=0x5574e1f4da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 -- 172.21.15.45:0/2698330418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 msgr2=0x5574e1f50220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f50220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 -- 172.21.15.45:0/2698330418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574e1e33dc0 con 0x5574e1f41da0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f445c0977e0 tx=0x7f445c0a3fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b740ce61242a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c020050 con 0x5574e1f41da0 2024-04-25T00:29:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c09b380 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f445c0979f0 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f445c055280 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4476468740 10 client.17817 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574e1bba3f0 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.315+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c09b380 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f445c0555d0 con 0x5574e1f41da0 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 10 client.17817 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 20 client.17817 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1f60760 0x5574e1f62b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 4 client.17817 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574e1eb3bb0 con 0x5574e1f60760 2024-04-25T00:29:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.316+0000 7f44714f6640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1f60760 0x5574e1f62b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f44714f6640 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1f60760 0x5574e1f62b50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c9911839835fb25 server_cookie=93bee7d365d2a0bb in_seq=0 out_seq=0 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4454ff9640 10 client.17817 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4454ff9640 1 -- 172.21.15.45:0/2698330418 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574e1eb3bb0 con 0x5574e1f60760 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4454ff9640 10 client.17817 handle_command_reply: tid=0 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 client.17817 shutdown 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 msgr2=0x5574e1f4d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f445c0977e0 tx=0x7f445c0a3fc0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 shutdown_connections 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1f60760 0x5574e1f62b50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f4d510 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 --2- 172.21.15.45:0/2698330418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f50220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4454ff9640 0 client.17817 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 >> 172.21.15.45:0/2698330418 conn(0x5574e1f3d8a0 msgr2=0x5574e1f436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.317+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 shutdown_connections 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.318+0000 7f4476468740 1 -- 172.21.15.45:0/2698330418 wait complete. 2024-04-25T00:29:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.318+0000 7f4476468740 20 client.17817 trim_cache size 0 max 16384 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 Processor -- start 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 -- start start 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1aca30 con 0x5574e1f41da0 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1ae570 con 0x5574e1f41480 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4476468740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1ad100 con 0x5574e1f50980 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34992/0 (socket says 172.21.15.45:34992) 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 -- 172.21.15.45:0/2894629198 learned_addr learned my addr 172.21.15.45:0/2894629198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f44714f6640 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 -- 172.21.15.45:0/2894629198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 msgr2=0x5574e1f4fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 -- 172.21.15.45:0/2894629198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 msgr2=0x5574e1f4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f4470cf5640 1 -- 172.21.15.45:0/2894629198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e1c847d0 con 0x5574e1f41da0 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.331+0000 7f44714f6640 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4470cf5640 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc5b0 tx=0x7f445c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b53ff19a1d3d244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4456ffd640 1 -- 172.21.15.45:0/2894629198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c020070 con 0x5574e1f41da0 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4456ffd640 1 -- 172.21.15.45:0/2894629198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c0d5070 con 0x5574e1f41da0 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4456ffd640 1 -- 172.21.15.45:0/2894629198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0979f0 con 0x5574e1f41da0 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 -- 172.21.15.45:0/2894629198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 msgr2=0x5574e1f5c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cc5b0 tx=0x7f445c00b480 comp rx=0 tx=0).stop 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 -- 172.21.15.45:0/2894629198 shutdown_connections 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f50980 0x5574e1f4fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f41da0 0x5574e1f5c200 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 --2- 172.21.15.45:0/2894629198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41480 0x5574e1f4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 -- 172.21.15.45:0/2894629198 >> 172.21.15.45:0/2894629198 conn(0x5574e1f3d8a0 msgr2=0x5574e1f5baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 -- 172.21.15.45:0/2894629198 shutdown_connections 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.332+0000 7f4476468740 1 -- 172.21.15.45:0/2894629198 wait complete. 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 -- start start 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 Processor -- start 2024-04-25T00:29:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f65ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f63250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1adea0 con 0x5574e1f50980 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1ad7d0 con 0x5574e1f41da0 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c1aaa30 con 0x5574e1f41480 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4476468740 10 client.0 _ll_register_callbacks cb 0x7f445c09b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4471cf7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4471cf7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34996/0 (socket says 172.21.15.45:34996) 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.333+0000 7f4471cf7640 1 -- 172.21.15.45:0/3126437264 learned_addr learned my addr 172.21.15.45:0/3126437264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f44714f6640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f65ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4470cf5640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f63250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 -- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 msgr2=0x5574e1f65ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f65ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 -- 172.21.15.45:0/3126437264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 msgr2=0x5574e1f63250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f63250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 -- 172.21.15.45:0/3126437264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574e1f63f40 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4470cf5640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f63250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4471cf7640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4464009cd0 tx=0x7f44640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19e446314c1d51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464004760 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f44640305a0 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c0d5070 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44640307a0 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4476468740 10 client.17823 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.334+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574e1fc6830 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c0d5070 con 0x5574e1f50980 2024-04-25T00:29:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4464030070 con 0x5574e1f50980 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 10 client.17823 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 10 client.17823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 20 client.17823 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1fc7030 0x5574e1fc9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 4 client.17823 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574e1eadf00 con 0x5574e1fc7030 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.335+0000 7f44714f6640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1fc7030 0x5574e1fc9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f44714f6640 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1fc7030 0x5574e1fc9450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20d835e82215f20d server_cookie=caeb298cfe408a64 in_seq=0 out_seq=0 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4454ff9640 10 client.17823 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4454ff9640 1 -- 172.21.15.45:0/3126437264 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574e1eadf00 con 0x5574e1fc7030 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4454ff9640 10 client.17823 handle_command_reply: tid=0 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 client.17823 shutdown 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 msgr2=0x5574e1f63790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4464009cd0 tx=0x7f44640095c0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 shutdown_connections 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5574e1fc7030 0x5574e1fc9450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574e1f50980 0x5574e1f63790 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4454ff9640 0 client.17823 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5574e1f41da0 0x5574e1f63250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 --2- 172.21.15.45:0/3126437264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5574e1f41480 0x5574e1f65ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 >> 172.21.15.45:0/3126437264 conn(0x5574e1f3d8a0 msgr2=0x5574e1f5baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 shutdown_connections 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 1 -- 172.21.15.45:0/3126437264 wait complete. 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.336+0000 7f4476468740 20 client.17823 trim_cache size 0 max 16384 2024-04-25T00:29:38.343 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:38.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 msgr2=0x7f44580a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 0x7f44580a5240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4464004ab0 tx=0x7f4464004a40 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 msgr2=0x7f446c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f445c00d770 tx=0x7f445c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 shutdown_connections 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f44580a2d80 0x7f44580a5240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c107720 0x7f446c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f446c1020d0 0x7f446c1985b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 --2- 172.21.15.45:0/3889087884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c101710 0x7f446c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 >> 172.21.15.45:0/3889087884 conn(0x7f446c0fd180 msgr2=0x7f446c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 shutdown_connections 2024-04-25T00:29:38.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.337+0000 7f447375a640 1 -- 172.21.15.45:0/3889087884 wait complete. 2024-04-25T00:29:38.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:29:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 -- 172.21.15.45:0/683009346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c06a780 msgr2=0x7f0d4c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 --2- 172.21.15.45:0/683009346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c06a780 0x7f0d4c10e5f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009cd0 tx=0x7f0d4402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 -- 172.21.15.45:0/683009346 shutdown_connections 2024-04-25T00:29:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 --2- 172.21.15.45:0/683009346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c06a780 0x7f0d4c10e5f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 --2- 172.21.15.45:0/683009346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 0x7f0d4c06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 --2- 172.21.15.45:0/683009346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c107730 0x7f0d4c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 -- 172.21.15.45:0/683009346 >> 172.21.15.45:0/683009346 conn(0x7f0d4c0776a0 msgr2=0x7f0d4c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 -- 172.21.15.45:0/683009346 shutdown_connections 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.452+0000 7f0d542cd640 1 -- 172.21.15.45:0/683009346 wait complete. 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 Processor -- start 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 -- start start 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 0x7f0d4c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 0x7f0d4c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111b40 con 0x7f0d4c107730 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1119c0 con 0x7f0d4c069dc0 2024-04-25T00:29:38.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d542cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111cc0 con 0x7f0d4c06a780 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d5286a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d5286a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35006/0 (socket says 172.21.15.45:35006) 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.453+0000 7f0d5286a640 1 -- 172.21.15.45:0/361898936 learned_addr learned my addr 172.21.15.45:0/361898936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d51868640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 0x7f0d4c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d52069640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 0x7f0d4c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 -- 172.21.15.45:0/361898936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 msgr2=0x7f0d4c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 0x7f0d4c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 -- 172.21.15.45:0/361898936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 msgr2=0x7f0d4c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 0x7f0d4c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 -- 172.21.15.45:0/361898936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c19cf50 con 0x7f0d4c107730 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d51868640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 0x7f0d4c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d5286a640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005cc0 tx=0x7f0d44003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d437fe640 1 -- 172.21.15.45:0/361898936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44004040 con 0x7f0d4c107730 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d437fe640 1 -- 172.21.15.45:0/361898936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d440046c0 con 0x7f0d4c107730 2024-04-25T00:29:38.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c19d1e0 con 0x7f0d4c107730 2024-04-25T00:29:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d437fe640 1 -- 172.21.15.45:0/361898936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44046480 con 0x7f0d4c107730 2024-04-25T00:29:38.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.454+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a4ad0 con 0x7f0d4c107730 2024-04-25T00:29:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.456+0000 7f0d437fe640 1 -- 172.21.15.45:0/361898936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0d440305a0 con 0x7f0d4c107730 2024-04-25T00:29:38.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 Processor -- start 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 -- start start 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5608b633d670 con 0x5608b63d8460 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5608b62e5970 con 0x5608b63d8d80 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5608b6339630 con 0x5608b63e7980 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d437fe640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 0x7f0d380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d51868640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d51868640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35014/0 (socket says 172.21.15.45:35014) 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d51868640 1 -- 172.21.15.45:0/466485170 learned_addr learned my addr 172.21.15.45:0/466485170 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d5286a640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 0x7f0d380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.457+0000 7f0d52069640 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 -- 172.21.15.45:0/466485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 msgr2=0x5608b63e9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 -- 172.21.15.45:0/466485170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 msgr2=0x5608b63e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d437fe640 1 -- 172.21.15.45:0/361898936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f0d4402e030 con 0x7f0d4c107730 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 -- 172.21.15.45:0/466485170 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b6309400 con 0x5608b63d8460 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d51868640 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c013550 tx=0x7f0d3c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9d7e75693d5d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d5286a640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 0x7f0d380a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d440bd1f0 tx=0x7f0d440c1040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d40bfe640 1 -- 172.21.15.45:0/466485170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c019070 con 0x5608b63d8460 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d40bfe640 1 -- 172.21.15.45:0/466485170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004220 con 0x5608b63d8460 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.458+0000 7f0d40bfe640 1 -- 172.21.15.45:0/466485170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0118e0 con 0x5608b63d8460 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- 172.21.15.45:0/466485170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 msgr2=0x5608b63d8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c013550 tx=0x7f0d3c00b6b0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- 172.21.15.45:0/466485170 shutdown_connections 2024-04-25T00:29:38.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/466485170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8460 0x5608b63d8840 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- 172.21.15.45:0/466485170 >> 172.21.15.45:0/466485170 conn(0x5608b63d48a0 msgr2=0x5608b63d4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- 172.21.15.45:0/466485170 shutdown_connections 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- 172.21.15.45:0/466485170 wait complete. 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 -- start start 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.459+0000 7f0d56fdc740 1 Processor -- start 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5608b63ea7c0 con 0x5608b63d8d80 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5608b62f9050 con 0x5608b63d8460 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5608b63ea940 con 0x5608b63e7980 2024-04-25T00:29:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d56fdc740 10 client.0 _ll_register_callbacks cb 0x5608b6050be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35020/0 (socket says 172.21.15.45:35020) 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 -- 172.21.15.45:0/808757273 learned_addr learned my addr 172.21.15.45:0/808757273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d51868640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d51868640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 -- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 msgr2=0x5608b63e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 -- 172.21.15.45:0/808757273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 msgr2=0x5608b63e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 -- 172.21.15.45:0/808757273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b62cadc0 con 0x5608b63d8d80 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d51868640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d52069640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800bed0 tx=0x7f0d48012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471796099e998ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.460+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480108a0 con 0x5608b63d8d80 2024-04-25T00:29:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f0d48010a40 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c004220 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d48010c40 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 10 client.17835 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5608b60513f0 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004220 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d48019070 con 0x5608b63d8d80 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 10 client.17835 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 10 client.17835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 20 client.17835 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b63f7a10 0x5608b63f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 4 client.17835 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.461+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608b634abb0 con 0x5608b63f7a10 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.462+0000 7f0d51868640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b63f7a10 0x5608b63f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.462+0000 7f0d51868640 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b63f7a10 0x5608b63f9e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e76095fb4eae8b06 server_cookie=766e54413103a129 in_seq=0 out_seq=0 2024-04-25T00:29:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.462+0000 7f0d35ffb640 10 client.17835 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d35ffb640 1 -- 172.21.15.45:0/808757273 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608b634abb0 con 0x5608b63f7a10 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d35ffb640 10 client.17835 handle_command_reply: tid=0 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 client.17835 shutdown 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 msgr2=0x5608b63e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800bed0 tx=0x7f0d48012820 comp rx=0 tx=0).stop 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 shutdown_connections 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b63f7a10 0x5608b63f9e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63e44d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/808757273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 >> 172.21.15.45:0/808757273 conn(0x5608b63d48a0 msgr2=0x5608b63da650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d35ffb640 0 client.17835 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 shutdown_connections 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 1 -- 172.21.15.45:0/808757273 wait complete. 2024-04-25T00:29:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.463+0000 7f0d56fdc740 20 client.17835 trim_cache size 0 max 16384 2024-04-25T00:29:38.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.476+0000 7f0d56fdc740 1 Processor -- start 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 -- start start 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63d8460 0x5608b63f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ac930 con 0x5608b63e7980 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ae470 con 0x5608b63d8d80 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ad000 con 0x5608b63d8460 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35036/0 (socket says 172.21.15.45:35036) 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 -- 172.21.15.45:0/2763141937 learned_addr learned my addr 172.21.15.45:0/2763141937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d52069640 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63d8460 0x5608b63f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 -- 172.21.15.45:0/2763141937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63d8460 msgr2=0x5608b63f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63d8460 0x5608b63f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 -- 172.21.15.45:0/2763141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 msgr2=0x5608b63e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 -- 172.21.15.45:0/2763141937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b611b7d0 con 0x5608b63e7980 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d51868640 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00b5b0 tx=0x7f0d3c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da042a3b26d9fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d37fff640 1 -- 172.21.15.45:0/2763141937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c019040 con 0x5608b63e7980 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d37fff640 1 -- 172.21.15.45:0/2763141937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004160 con 0x5608b63e7980 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.477+0000 7f0d37fff640 1 -- 172.21.15.45:0/2763141937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c005470 con 0x5608b63e7980 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2763141937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 msgr2=0x5608b63ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00b5b0 tx=0x7f0d3c015040 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2763141937 shutdown_connections 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63e7980 0x5608b63ff7a0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8d80 0x5608b63e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2763141937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63d8460 0x5608b63f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2763141937 >> 172.21.15.45:0/2763141937 conn(0x5608b63d48a0 msgr2=0x5608b63e4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2763141937 shutdown_connections 2024-04-25T00:29:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2763141937 wait complete. 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 -- start start 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.478+0000 7f0d56fdc740 1 Processor -- start 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1adda0 con 0x5608b63d8d80 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1ad6d0 con 0x5608b63d8460 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1aa930 con 0x5608b63e7980 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 10 client.0 _ll_register_callbacks cb 0x7f0d4800b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35038/0 (socket says 172.21.15.45:35038) 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 -- 172.21.15.45:0/2670209886 learned_addr learned my addr 172.21.15.45:0/2670209886 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d51868640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d51868640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 -- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 msgr2=0x5608b63ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 -- 172.21.15.45:0/2670209886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 msgr2=0x5608b63ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 -- 172.21.15.45:0/2670209886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b63eb480 con 0x5608b63d8d80 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d51868640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d52069640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5608b63e5aa0 tx=0x7f0d48005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2339d57746f440d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4802d070 con 0x5608b63d8d80 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f0d4801aa50 con 0x5608b63d8d80 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c004160 con 0x5608b63d8d80 2024-04-25T00:29:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.479+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d4801ac50 con 0x5608b63d8d80 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 10 client.17841 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5608b645da10 con 0x5608b63d8d80 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c004160 con 0x5608b63d8d80 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d48019050 con 0x5608b63d8d80 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 10 client.17841 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 10 client.17841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 20 client.17841 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b645e160 0x5608b6460580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 4 client.17841 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608b6344f00 con 0x5608b645e160 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.480+0000 7f0d51868640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b645e160 0x5608b6460580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d35ffb640 10 client.17841 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d51868640 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b645e160 0x5608b6460580 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb02e27b08bdfa9 server_cookie=32600351b1746107 in_seq=0 out_seq=0 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d35ffb640 1 -- 172.21.15.45:0/2670209886 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608b6344f00 con 0x5608b645e160 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d35ffb640 10 client.17841 handle_command_reply: tid=0 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 client.17841 shutdown 2024-04-25T00:29:38.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 msgr2=0x5608b63efe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x5608b63e5aa0 tx=0x7f0d48005cc0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 shutdown_connections 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5608b645e160 0x5608b6460580 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5608b63e7980 0x5608b63ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d35ffb640 0 client.17841 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5608b63d8d80 0x5608b63efe40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 --2- 172.21.15.45:0/2670209886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5608b63d8460 0x5608b63ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 >> 172.21.15.45:0/2670209886 conn(0x5608b63d48a0 msgr2=0x5608b63df3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 shutdown_connections 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 1 -- 172.21.15.45:0/2670209886 wait complete. 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.481+0000 7f0d56fdc740 20 client.17841 trim_cache size 0 max 16384 2024-04-25T00:29:38.488 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:38.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 msgr2=0x7f0d380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 0x7f0d380a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d440bd1f0 tx=0x7f0d440c1040 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 msgr2=0x7f0d4c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005cc0 tx=0x7f0d44003970 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 shutdown_connections 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f0d380a2d00 0x7f0d380a51c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d4c107730 0x7f0d4c19c820 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d4c06a780 0x7f0d4c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 --2- 172.21.15.45:0/361898936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d4c069dc0 0x7f0d4c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 >> 172.21.15.45:0/361898936 conn(0x7f0d4c0776a0 msgr2=0x7f0d4c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 shutdown_connections 2024-04-25T00:29:38.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.482+0000 7f0d542cd640 1 -- 172.21.15.45:0/361898936 wait complete. 2024-04-25T00:29:38.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:29:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- 172.21.15.45:0/3971624775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce4069dc0 msgr2=0x7f9ce406a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 --2- 172.21.15.45:0/3971624775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce4069dc0 0x7f9ce406a240 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009930 tx=0x7f9ccc02a420 comp rx=0 tx=0).stop 2024-04-25T00:29:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- 172.21.15.45:0/3971624775 shutdown_connections 2024-04-25T00:29:38.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 --2- 172.21.15.45:0/3971624775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce406a780 0x7f9ce410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 --2- 172.21.15.45:0/3971624775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce4069dc0 0x7f9ce406a240 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 --2- 172.21.15.45:0/3971624775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 0x7f9ce4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- 172.21.15.45:0/3971624775 >> 172.21.15.45:0/3971624775 conn(0x7f9ce40776a0 msgr2=0x7f9ce4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- 172.21.15.45:0/3971624775 shutdown_connections 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- 172.21.15.45:0/3971624775 wait complete. 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 Processor -- start 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.599+0000 7f9ce9334640 1 -- start start 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 0x7f9ce4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 0x7f9ce419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111b00 con 0x7f9ce406a780 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111980 con 0x7f9ce4107730 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce9334640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4111c80 con 0x7f9ce4069dc0 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35062/0 (socket says 172.21.15.45:35062) 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1421561583 learned_addr learned my addr 172.21.15.45:0/1421561583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 0x7f9ce419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 0x7f9ce4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1421561583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 msgr2=0x7f9ce4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 0x7f9ce4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1421561583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 msgr2=0x7f9ce419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 0x7f9ce419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1421561583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce419cf90 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 0x7f9ce4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009a50 tx=0x7f9ccc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce0ff9640 1 -- 172.21.15.45:0/1421561583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc03f030 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.600+0000 7f9ce0ff9640 1 -- 172.21.15.45:0/1421561583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc003ce0 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.601+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce419d220 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.601+0000 7f9ce0ff9640 1 -- 172.21.15.45:0/1421561583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc035070 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.601+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a4b10 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.603+0000 7f9ce0ff9640 1 -- 172.21.15.45:0/1421561583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ccc042030 con 0x7f9ce406a780 2024-04-25T00:29:38.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.603+0000 7f9cec042740 1 Processor -- start 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 -- start start 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b34960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b3f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582e5a99670 con 0x5582e5b3f570 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5582e5a41970 con 0x5582e5b34580 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cec042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582e5a95630 con 0x5582e5b34ea0 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35074/0 (socket says 172.21.15.45:35074) 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 -- 172.21.15.45:0/4033399905 learned_addr learned my addr 172.21.15.45:0/4033399905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b34960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b3f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9ce0ff9640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 0x7f9cd00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 -- 172.21.15.45:0/4033399905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 msgr2=0x5582e5b3f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b3f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 -- 172.21.15.45:0/4033399905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 msgr2=0x5582e5b34960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b34960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9cdbfff640 1 -- 172.21.15.45:0/4033399905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e5a65400 con 0x5582e5b3f570 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b34960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.604+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 0x7f9cd00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.605+0000 7f9ce0ff9640 1 -- 172.21.15.45:0/1421561583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f9ccc0efdf0 con 0x7f9ce406a780 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.605+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0ed060 tx=0x7f9ccc0314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd72764855d3221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.605+0000 7f9cd97fa640 1 -- 172.21.15.45:0/4033399905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0bba30 con 0x5582e5b3f570 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.605+0000 7f9cd97fa640 1 -- 172.21.15.45:0/4033399905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc07f8e0 con 0x5582e5b3f570 2024-04-25T00:29:38.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.605+0000 7f9cd97fa640 1 -- 172.21.15.45:0/4033399905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0bb850 con 0x5582e5b3f570 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- 172.21.15.45:0/4033399905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 msgr2=0x5582e5b41960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 0x7f9cd00a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00a2c0 tx=0x7f9cdc004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0ed060 tx=0x7f9ccc0314b0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- 172.21.15.45:0/4033399905 shutdown_connections 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b41960 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b3f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- 172.21.15.45:0/4033399905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b34960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- 172.21.15.45:0/4033399905 >> 172.21.15.45:0/4033399905 conn(0x5582e5b308a0 msgr2=0x5582e5b30c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- 172.21.15.45:0/4033399905 shutdown_connections 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- 172.21.15.45:0/4033399905 wait complete. 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- start start 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 Processor -- start 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b3ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582e5b42120 con 0x5582e5b34ea0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5582e5a55050 con 0x5582e5b3f570 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.606+0000 7f9cec042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582e5b422a0 con 0x5582e5b34580 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cec042740 10 client.0 _ll_register_callbacks cb 0x5582e57acbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35082/0 (socket says 172.21.15.45:35082) 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1485366227 learned_addr learned my addr 172.21.15.45:0/1485366227 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b3ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1485366227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 msgr2=0x5582e5b3ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b3ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 msgr2=0x5582e5b3c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b3c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 -- 172.21.15.45:0/1485366227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e5a26dc0 con 0x5582e5b34ea0 2024-04-25T00:29:38.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc032c00 tx=0x7f9ccc004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac19c874f11f95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc049040 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f9ccc0b8560 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582e5aa6bb0 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ccc0bb850 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cec042740 10 client.17853 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582e57ad3f0 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.607+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5582e5aa6bb0 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ccc0810c0 con 0x5582e5b34ea0 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 10 client.17853 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 20 client.17853 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5b4f140 0x5582e5b51530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 4 client.17853 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582e5aa0f00 con 0x5582e5b4f140 2024-04-25T00:29:38.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5b4f140 0x5582e5b51530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5b4f140 0x5582e5b51530 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b396bbba0ba5372b server_cookie=9eec4055d539096b in_seq=0 out_seq=0 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.608+0000 7f9cca7fc640 10 client.17853 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cca7fc640 1 -- 172.21.15.45:0/1485366227 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582e5aa0f00 con 0x5582e5b4f140 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cca7fc640 10 client.17853 handle_command_reply: tid=0 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 client.17853 shutdown 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 msgr2=0x5582e5b3c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc032c00 tx=0x7f9ccc004590 comp rx=0 tx=0).stop 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 shutdown_connections 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b3c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5b4f140 0x5582e5b51530 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b3c200 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 --2- 172.21.15.45:0/1485366227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b3ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cca7fc640 0 client.17853 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:38.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 >> 172.21.15.45:0/1485366227 conn(0x5582e5b308a0 msgr2=0x5582e5b366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 shutdown_connections 2024-04-25T00:29:38.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.609+0000 7f9cec042740 1 -- 172.21.15.45:0/1485366227 wait complete. 2024-04-25T00:29:38.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.610+0000 7f9cec042740 20 client.17853 trim_cache size 0 max 16384 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 Processor -- start 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 -- start start 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b4a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41aebb0 con 0x5582e5b34ea0 2024-04-25T00:29:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41b06f0 con 0x5582e5b3f570 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cec042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41af280 con 0x5582e5b34580 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35086/0 (socket says 172.21.15.45:35086) 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 -- 172.21.15.45:0/2959560944 learned_addr learned my addr 172.21.15.45:0/2959560944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b4a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 -- 172.21.15.45:0/2959560944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 msgr2=0x5582e5b56c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b56c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 -- 172.21.15.45:0/2959560944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 msgr2=0x5582e5b4a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b4a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9cdbfff640 1 -- 172.21.15.45:0/2959560944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e58777d0 con 0x5582e5b34ea0 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.623+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b4a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0bbbf0 tx=0x7f9ccc0ba400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa798d649ab56ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cd8ff9640 1 -- 172.21.15.45:0/2959560944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc049070 con 0x5582e5b34ea0 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cd8ff9640 1 -- 172.21.15.45:0/2959560944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc03be40 con 0x5582e5b34ea0 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cd8ff9640 1 -- 172.21.15.45:0/2959560944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0bba30 con 0x5582e5b34ea0 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cec042740 1 -- 172.21.15.45:0/2959560944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 msgr2=0x5582e5b57160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.624+0000 7f9cec042740 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0bbbf0 tx=0x7f9ccc0ba400 comp rx=0 tx=0).stop 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- 172.21.15.45:0/2959560944 shutdown_connections 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b3f570 0x5582e5b4a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b34ea0 0x5582e5b57160 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- 172.21.15.45:0/2959560944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34580 0x5582e5b56c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- 172.21.15.45:0/2959560944 >> 172.21.15.45:0/2959560944 conn(0x5582e5b308a0 msgr2=0x5582e5b49ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- 172.21.15.45:0/2959560944 shutdown_connections 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- 172.21.15.45:0/2959560944 wait complete. 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- start start 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 Processor -- start 2024-04-25T00:29:38.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41b0020 con 0x5582e5b3f570 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41af950 con 0x5582e5b34580 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41acbb0 con 0x5582e5b34ea0 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cec042740 10 client.0 _ll_register_callbacks cb 0x7f9cd4008310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9ce37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35102/0 (socket says 172.21.15.45:35102) 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9ce37fe640 1 -- 172.21.15.45:0/671897937 learned_addr learned my addr 172.21.15.45:0/671897937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9cdbfff640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.625+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 -- 172.21.15.45:0/671897937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 msgr2=0x5582e5b44b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b44b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 -- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 msgr2=0x5582e5b47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b47790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 -- 172.21.15.45:0/671897937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e5b458c0 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9ce37fe640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00fa50 tx=0x7f9cdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4460f08b5f1df5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc02c060 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582e5a1e820 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f9cdc017070 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9cdc033050 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 10 client.17859 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582e5bb5440 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5582e5a1e820 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9cdc01a3d0 con 0x5582e5b3f570 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 10 client.17859 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:38.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.626+0000 7f9cec042740 20 client.17859 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5bb5bf0 0x5582e5bb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 4 client.17859 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582e5a6ad10 con 0x5582e5bb5bf0 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5bb5bf0 0x5582e5bb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cca7fc640 10 client.17859 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9ce2ffd640 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5bb5bf0 0x5582e5bb8010 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6d8cdb50c1d972 server_cookie=b98e6a7b03aa8414 in_seq=0 out_seq=0 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cca7fc640 1 -- 172.21.15.45:0/671897937 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582e5a6ad10 con 0x5582e5bb5bf0 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cca7fc640 10 client.17859 handle_command_reply: tid=0 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 client.17859 shutdown 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 msgr2=0x5582e5b450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00fa50 tx=0x7f9cdc004ab0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 shutdown_connections 2024-04-25T00:29:38.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5582e5bb5bf0 0x5582e5bb8010 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582e5b3f570 0x5582e5b450b0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582e5b34ea0 0x5582e5b44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 --2- 172.21.15.45:0/671897937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e5b34580 0x5582e5b47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cca7fc640 0 client.17859 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.627+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 >> 172.21.15.45:0/671897937 conn(0x5582e5b308a0 msgr2=0x5582e5b4a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 shutdown_connections 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9cec042740 1 -- 172.21.15.45:0/671897937 wait complete. 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9cec042740 20 client.17859 trim_cache size 0 max 16384 2024-04-25T00:29:38.635 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 msgr2=0x7f9cd00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 0x7f9cd00a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00a2c0 tx=0x7f9cdc004a40 comp rx=0 tx=0).stop 2024-04-25T00:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 msgr2=0x7f9ce41984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009a50 tx=0x7f9ccc004270 comp rx=0 tx=0).stop 2024-04-25T00:29:38.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 shutdown_connections 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9cd00a2d00 0x7f9cd00a51c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ce4107730 0x7f9ce419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ce406a780 0x7f9ce41984e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 --2- 172.21.15.45:0/1421561583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ce4069dc0 0x7f9ce4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 >> 172.21.15.45:0/1421561583 conn(0x7f9ce40776a0 msgr2=0x7f9ce40fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 shutdown_connections 2024-04-25T00:29:38.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:38.628+0000 7f9ce9334640 1 -- 172.21.15.45:0/1421561583 wait complete. 2024-04-25T00:29:38.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:38 smithi071 ceph-mon[32667]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:38.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:38 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3549691748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:38.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:38 smithi045 ceph-mon[29194]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:38.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:38 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3549691748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:38 smithi122 ceph-mon[32959]: pgmap v235: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:39.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:38 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3549691748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:29:39.640 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:29:39.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:29:39.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:29:39.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 -- 172.21.15.45:0/3520898987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 msgr2=0x7f7c0c10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 --2- 172.21.15.45:0/3520898987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c10e5e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009cd0 tx=0x7f7c0402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 -- 172.21.15.45:0/3520898987 shutdown_connections 2024-04-25T00:29:39.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 --2- 172.21.15.45:0/3520898987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c10e5e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 --2- 172.21.15.45:0/3520898987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 --2- 172.21.15.45:0/3520898987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 -- 172.21.15.45:0/3520898987 >> 172.21.15.45:0/3520898987 conn(0x7f7c0c0fd180 msgr2=0x7f7c0c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 -- 172.21.15.45:0/3520898987 shutdown_connections 2024-04-25T00:29:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.759+0000 7f7c11189640 1 -- 172.21.15.45:0/3520898987 wait complete. 2024-04-25T00:29:39.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 Processor -- start 2024-04-25T00:29:39.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 -- start start 2024-04-25T00:29:39.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111b40 con 0x7f7c0c1020d0 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1119c0 con 0x7f7c0c107720 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c11189640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c111cc0 con 0x7f7c0c101710 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.760+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35108/0 (socket says 172.21.15.45:35108) 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 -- 172.21.15.45:0/2574970013 learned_addr learned my addr 172.21.15.45:0/2574970013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 -- 172.21.15.45:0/2574970013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 msgr2=0x7f7c0c19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 -- 172.21.15.45:0/2574970013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 msgr2=0x7f7c0c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 -- 172.21.15.45:0/2574970013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c19f0c0 con 0x7f7c0c1020d0 2024-04-25T00:29:39.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400c920 tx=0x7f7bf400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c08d99640 1 -- 172.21.15.45:0/2574970013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4007c70 con 0x7f7c0c1020d0 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c08d99640 1 -- 172.21.15.45:0/2574970013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf4007e10 con 0x7f7c0c1020d0 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c08d99640 1 -- 172.21.15.45:0/2574970013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4018890 con 0x7f7c0c1020d0 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c19f3b0 con 0x7f7c0c1020d0 2024-04-25T00:29:39.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.761+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a6c40 con 0x7f7c0c1020d0 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c08d99640 1 -- 172.21.15.45:0/2574970013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7bf4018a70 con 0x7f7c0c1020d0 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 Processor -- start 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 -- start start 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x559426083cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x559426084710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559425fe82c0 con 0x55942608f010 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559425f90910 con 0x559426084290 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.764+0000 7f7c13f73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559425fe4280 con 0x559426085980 2024-04-25T00:29:39.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c08d99640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 0x7f7bf80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c0b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x559426083cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35120/0 (socket says 172.21.15.45:35120) 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c0ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x559426084710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 -- 172.21.15.45:0/1726929721 learned_addr learned my addr 172.21.15.45:0/1726929721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 0x7f7bf80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c08d99640 1 -- 172.21.15.45:0/2574970013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f7bf40cfa10 con 0x7f7c0c1020d0 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 -- 172.21.15.45:0/1726929721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 msgr2=0x559426083cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x559426083cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 -- 172.21.15.45:0/1726929721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 msgr2=0x559426084710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x559426084710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 -- 172.21.15.45:0/1726929721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559425fa1fb0 con 0x55942608f010 2024-04-25T00:29:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x559426084710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c03fff640 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40a42b0 tx=0x7f7bf4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb665e9d8e29ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c017fa640 1 -- 172.21.15.45:0/1726929721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400de20 con 0x55942608f010 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.765+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 0x7f7bf80a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009a60 tx=0x7f7c0402a8e0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c017fa640 1 -- 172.21.15.45:0/1726929721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf409bb70 con 0x55942608f010 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c017fa640 1 -- 172.21.15.45:0/1726929721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40981e0 con 0x55942608f010 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 -- 172.21.15.45:0/1726929721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 msgr2=0x559426091400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40a42b0 tx=0x7f7bf4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 -- 172.21.15.45:0/1726929721 shutdown_connections 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x559426091400 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x559426084710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 --2- 172.21.15.45:0/1726929721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x559426083cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 -- 172.21.15.45:0/1726929721 >> 172.21.15.45:0/1726929721 conn(0x55942604c950 msgr2=0x5594260807f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 -- 172.21.15.45:0/1726929721 shutdown_connections 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.766+0000 7f7c13f73740 1 -- 172.21.15.45:0/1726929721 wait complete. 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 -- start start 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 Processor -- start 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x55942608bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x55942608c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559426091de0 con 0x559426085980 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559425fa3e90 con 0x55942608f010 2024-04-25T00:29:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559426091f60 con 0x559426084290 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c13f73740 10 client.0 _ll_register_callbacks cb 0x559425fdf970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35136/0 (socket says 172.21.15.45:35136) 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 -- 172.21.15.45:0/3577513053 learned_addr learned my addr 172.21.15.45:0/3577513053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x55942608bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x55942608c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 -- 172.21.15.45:0/3577513053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 msgr2=0x55942608bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x55942608bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 -- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 msgr2=0x55942608c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x55942608c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c03fff640 1 -- 172.21.15.45:0/3577513053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559425dc47e0 con 0x559426085980 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x55942608bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.767+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x55942608c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c03fff640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409bb00 tx=0x7f7bf4097ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f51d8564f8a400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40a47e0 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f7bf40cd090 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf409bb70 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bf40981e0 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 10 client.17871 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559425e24f70 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf409bb70 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bf409aa30 con 0x559426085980 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 10 client.17871 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 20 client.17871 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55942609ef30 0x5594260a1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 4 client.17871 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.768+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559425fbc5a0 con 0x55942609ef30 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.769+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55942609ef30 0x5594260a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.769+0000 7f7bf27fc640 10 client.17871 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.769+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55942609ef30 0x5594260a1320 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e898186b9079d24b server_cookie=643adb4f1741d66 in_seq=0 out_seq=0 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7bf27fc640 1 -- 172.21.15.45:0/3577513053 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559425fbc5a0 con 0x55942609ef30 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7bf27fc640 10 client.17871 handle_command_reply: tid=0 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 client.17871 shutdown 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 msgr2=0x55942608c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7bf409bb00 tx=0x7f7bf4097ea0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 shutdown_connections 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x55942608c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55942609ef30 0x5594260a1320 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x55942608c460 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7bf27fc640 0 client.17871 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3577513053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x55942608bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 >> 172.21.15.45:0/3577513053 conn(0x55942604c950 msgr2=0x559426086660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 shutdown_connections 2024-04-25T00:29:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 1 -- 172.21.15.45:0/3577513053 wait complete. 2024-04-25T00:29:39.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.770+0000 7f7c13f73740 20 client.17871 trim_cache size 0 max 16384 2024-04-25T00:29:39.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 Processor -- start 2024-04-25T00:29:39.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 -- start start 2024-04-25T00:29:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x55942607f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x5594260a4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b0da0 con 0x55942608f010 2024-04-25T00:29:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b28e0 con 0x559426084290 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c13f73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b1470 con 0x559426085980 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c0b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c0b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35140/0 (socket says 172.21.15.45:35140) 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.788+0000 7f7c0b59e640 1 -- 172.21.15.45:0/3423183441 learned_addr learned my addr 172.21.15.45:0/3423183441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x55942607f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c03fff640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x5594260a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 -- 172.21.15.45:0/3423183441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 msgr2=0x5594260a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x5594260a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 -- 172.21.15.45:0/3423183441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 msgr2=0x55942607f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x55942607f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 -- 172.21.15.45:0/3423183441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559426080a00 con 0x55942608f010 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c03fff640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x5594260a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7c04035000 tx=0x7f7c04005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd493873af2c709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c00ff9640 1 -- 172.21.15.45:0/3423183441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004040 con 0x55942608f010 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c00ff9640 1 -- 172.21.15.45:0/3423183441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04036040 con 0x55942608f010 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.789+0000 7f7c00ff9640 1 -- 172.21.15.45:0/3423183441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04030dd0 con 0x55942608f010 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- 172.21.15.45:0/3423183441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 msgr2=0x5594260a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7c04035000 tx=0x7f7c04005f60 comp rx=0 tx=0).stop 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- 172.21.15.45:0/3423183441 shutdown_connections 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55942608f010 0x5594260a4880 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426085980 0x5594260a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- 172.21.15.45:0/3423183441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559426084290 0x55942607f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- 172.21.15.45:0/3423183441 >> 172.21.15.45:0/3423183441 conn(0x55942604c950 msgr2=0x55942607f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- 172.21.15.45:0/3423183441 shutdown_connections 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- 172.21.15.45:0/3423183441 wait complete. 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 -- start start 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 Processor -- start 2024-04-25T00:29:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x5594260a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.790+0000 7f7c13f73740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x5594260a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b2210 con 0x559426085980 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b1b40 con 0x55942608f010 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1aeda0 con 0x559426084290 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 10 client.0 _ll_register_callbacks cb 0x7f7bfc008310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35156/0 (socket says 172.21.15.45:35156) 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 -- 172.21.15.45:0/2820621632 learned_addr learned my addr 172.21.15.45:0/2820621632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x5594260a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c0b59e640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x5594260a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 -- 172.21.15.45:0/2820621632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 msgr2=0x5594260a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x5594260a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 -- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 msgr2=0x5594260a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x5594260a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 -- 172.21.15.45:0/2820621632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559425cbc170 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x5594260a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c03fff640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4098540 tx=0x7f7bf40cfd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcc48b7d245fb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7bf27fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4098380 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7f7bf40a1250 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c04036040 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bf409ac30 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 10 client.17877 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.791+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559426105e20 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04036040 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bf40a4350 con 0x559426085980 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 10 client.17877 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 20 client.17877 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559426106410 0x559426108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 4 client.17877 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:29:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559425ff5800 con 0x559426106410 2024-04-25T00:29:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559426106410 0x559426108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7bf27fc640 10 client.17877 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.792+0000 7f7c0ad9d640 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559426106410 0x559426108830 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21df79a7a5da86f6 server_cookie=98e705926aa502e in_seq=0 out_seq=0 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.793+0000 7f7bf27fc640 1 -- 172.21.15.45:0/2820621632 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559425ff5800 con 0x559426106410 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.793+0000 7f7bf27fc640 10 client.17877 handle_command_reply: tid=0 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 client.17877 shutdown 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 msgr2=0x5594260a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4098540 tx=0x7f7bf40cfd10 comp rx=0 tx=0).stop 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 shutdown_connections 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55942608f010 0x5594260a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x559426106410 0x559426108830 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559426085980 0x5594260a2980 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 --2- 172.21.15.45:0/2820621632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559426084290 0x5594260a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 >> 172.21.15.45:0/2820621632 conn(0x55942604c950 msgr2=0x55942607f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7bf27fc640 0 client.17877 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 shutdown_connections 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 1 -- 172.21.15.45:0/2820621632 wait complete. 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c13f73740 20 client.17877 trim_cache size 0 max 16384 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "7f6cf0ab-2b86-4342-9f66-1d99112c999c", 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:29:39.800 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:39.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 msgr2=0x7f7bf80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 0x7f7bf80a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009a60 tx=0x7f7c0402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 msgr2=0x7f7c0c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.794+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400c920 tx=0x7f7bf400cde0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 shutdown_connections 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7bf80a2d00 0x7f7bf80a51c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c107720 0x7f7c0c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c0c1020d0 0x7f7c0c19a610 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 --2- 172.21.15.45:0/2574970013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c0c101710 0x7f7c0c19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 >> 172.21.15.45:0/2574970013 conn(0x7f7c0c0fd180 msgr2=0x7f7c0c10cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 shutdown_connections 2024-04-25T00:29:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.795+0000 7f7c11189640 1 -- 172.21.15.45:0/2574970013 wait complete. 2024-04-25T00:29:39.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:29:39.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:29:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/2621183014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf806a780 msgr2=0x7fbcf810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/2621183014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf806a780 0x7fbcf810e5f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009cd0 tx=0x7fbcf002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/2621183014 shutdown_connections 2024-04-25T00:29:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/2621183014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf806a780 0x7fbcf810e5f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/2621183014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf8069dc0 0x7fbcf806a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/2621183014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 0x7fbcf8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/2621183014 >> 172.21.15.45:0/2621183014 conn(0x7fbcf80776a0 msgr2=0x7fbcf8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/2621183014 shutdown_connections 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.910+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/2621183014 wait complete. 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 Processor -- start 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 -- start start 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 0x7fbcf8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 0x7fbcf819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8111b40 con 0x7fbcf8069dc0 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81119c0 con 0x7fbcf806a780 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcfc9ab640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8111cc0 con 0x7fbcf8107730 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcf659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcf659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35178/0 (socket says 172.21.15.45:35178) 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.911+0000 7fbcf659c640 1 -- 172.21.15.45:0/1200018867 learned_addr learned my addr 172.21.15.45:0/1200018867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 0x7fbcf819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 0x7fbcf8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 -- 172.21.15.45:0/1200018867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 msgr2=0x7fbcf819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 0x7fbcf819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 -- 172.21.15.45:0/1200018867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 msgr2=0x7fbcf8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 0x7fbcf8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 -- 172.21.15.45:0/1200018867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf819cf80 con 0x7fbcf8069dc0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 0x7fbcf8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcf659c640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e9f0 tx=0x7fbce800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbce77fe640 1 -- 172.21.15.45:0/1200018867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce800cd80 con 0x7fbcf8069dc0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbce77fe640 1 -- 172.21.15.45:0/1200018867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce8005500 con 0x7fbcf8069dc0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf819d270 con 0x7fbcf8069dc0 2024-04-25T00:29:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbce77fe640 1 -- 172.21.15.45:0/1200018867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8010690 con 0x7fbcf8069dc0 2024-04-25T00:29:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.912+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81a4b00 con 0x7fbcf8069dc0 2024-04-25T00:29:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.914+0000 7fbce77fe640 1 -- 172.21.15.45:0/1200018867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbce80046c0 con 0x7fbcf8069dc0 2024-04-25T00:29:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 Processor -- start 2024-04-25T00:29:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 -- start start 2024-04-25T00:29:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbce77fe640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 0x7fbce00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c55a42d6b0 con 0x55c55a4d7a30 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c55a3d5880 con 0x55c55a4c8530 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcff78a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c55a429670 con 0x55c55a4c8e50 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf659c640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 0x7fbce00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35188/0 (socket says 172.21.15.45:35188) 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1834343348 learned_addr learned my addr 172.21.15.45:0/1834343348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbce77fe640 1 -- 172.21.15.45:0/1200018867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7fbce80cfc70 con 0x7fbcf8069dc0 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.915+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1834343348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 msgr2=0x55c55a4d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1834343348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 msgr2=0x55c55a4c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1834343348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55a20b7d0 con 0x55c55a4d7a30 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009a60 tx=0x7fbcf00093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca93db0837dd7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbcf659c640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 0x7fbce00a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce80cd0b0 tx=0x7fbce8091260 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbce4ff9640 1 -- 172.21.15.45:0/1834343348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0031970 con 0x55c55a4d7a30 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbce4ff9640 1 -- 172.21.15.45:0/1834343348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf0036070 con 0x55c55a4d7a30 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.916+0000 7fbce4ff9640 1 -- 172.21.15.45:0/1834343348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0004610 con 0x55c55a4d7a30 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- 172.21.15.45:0/1834343348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 msgr2=0x55c55a4d9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009a60 tx=0x7fbcf00093a0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- 172.21.15.45:0/1834343348 shutdown_connections 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d9e20 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1834343348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- 172.21.15.45:0/1834343348 >> 172.21.15.45:0/1834343348 conn(0x55c55a4c4950 msgr2=0x55c55a4c4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- 172.21.15.45:0/1834343348 shutdown_connections 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- 172.21.15.45:0/1834343348 wait complete. 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- start start 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 Processor -- start 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8530 0x55c55a4d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8e50 0x55c55a4d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c55a4da790 con 0x55c55a4d7a30 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c55a3e8f50 con 0x55c55a4c8e50 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c55a4da910 con 0x55c55a4c8530 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcff78a740 10 client.0 _ll_register_callbacks cb 0x55c55a424000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35196/0 (socket says 172.21.15.45:35196) 2024-04-25T00:29:39.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/536825146 learned_addr learned my addr 172.21.15.45:0/536825146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.917+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8530 0x55c55a4d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8e50 0x55c55a4d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/536825146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8530 msgr2=0x55c55a4d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8530 0x55c55a4d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8e50 msgr2=0x55c55a4d4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8e50 0x55c55a4d4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/536825146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55a1413f0 con 0x55c55a4d7a30 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0002230 tx=0x7fbcf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ffb15611a8a027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0004e10 con 0x55c55a4d7a30 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7fbcf0004460 con 0x55c55a4d7a30 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf0036070 con 0x55c55a4d7a30 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcf005e050 con 0x55c55a4d7a30 2024-04-25T00:29:39.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcff78a740 10 client.17889 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c55a1010f0 con 0x55c55a4d7a30 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf0036070 con 0x55c55a4d7a30 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.918+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbcf0030070 con 0x55c55a4d7a30 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 10 client.17889 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 20 client.17889 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a4e7b30 0x55c55a4e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 4 client.17889 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c55a43abf0 con 0x55c55a4e7b30 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a4e7b30 0x55c55a4e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcddffb640 10 client.17889 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.919+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a4e7b30 0x55c55a4e9f20 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6353c5f9948794c0 server_cookie=346c68aba0e6bd25 in_seq=0 out_seq=0 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcddffb640 1 -- 172.21.15.45:0/536825146 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c55a43abf0 con 0x55c55a4e7b30 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcddffb640 10 client.17889 handle_command_reply: tid=0 2024-04-25T00:29:39.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 client.17889 shutdown 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 msgr2=0x55c55a4d5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0002230 tx=0x7fbcf00042f0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 shutdown_connections 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a4e7b30 0x55c55a4e9f20 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4d7a30 0x55c55a4d5450 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8e50 0x55c55a4d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 --2- 172.21.15.45:0/536825146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8530 0x55c55a4d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcddffb640 0 client.17889 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 >> 172.21.15.45:0/536825146 conn(0x55c55a4c4950 msgr2=0x55c55a4ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 shutdown_connections 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 1 -- 172.21.15.45:0/536825146 wait complete. 2024-04-25T00:29:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.920+0000 7fbcff78a740 20 client.17889 trim_cache size 0 max 16384 2024-04-25T00:29:39.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 Processor -- start 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 -- start start 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4e32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4d7a30 0x55c55a4ecf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81acac0 con 0x55c55a4c8530 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81ae600 con 0x55c55a4d7a30 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcff78a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81ad190 con 0x55c55a4c8e50 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35202/0 (socket says 172.21.15.45:35202) 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1776508529 learned_addr learned my addr 172.21.15.45:0/1776508529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1776508529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 msgr2=0x55c55a4e32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4e32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1776508529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4d7a30 msgr2=0x55c55a4ecf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4d7a30 0x55c55a4ecf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 -- 172.21.15.45:0/1776508529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55a4c8c80 con 0x55c55a4c8530 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009620 tx=0x7fbcf0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8467a0eb4d87415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.938+0000 7fbcdffff640 1 -- 172.21.15.45:0/1776508529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0009800 con 0x55c55a4c8530 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcdffff640 1 -- 172.21.15.45:0/1776508529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf003f070 con 0x55c55a4c8530 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcdffff640 1 -- 172.21.15.45:0/1776508529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0030940 con 0x55c55a4c8530 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- 172.21.15.45:0/1776508529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 msgr2=0x55c55a4d7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009620 tx=0x7fbcf0009560 comp rx=0 tx=0).stop 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- 172.21.15.45:0/1776508529 shutdown_connections 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4d7a30 0x55c55a4ecf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4c8e50 0x55c55a4e32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 --2- 172.21.15.45:0/1776508529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8530 0x55c55a4d7100 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- 172.21.15.45:0/1776508529 >> 172.21.15.45:0/1776508529 conn(0x55c55a4c4950 msgr2=0x55c55a4e2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- 172.21.15.45:0/1776508529 shutdown_connections 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- 172.21.15.45:0/1776508529 wait complete. 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.939+0000 7fbcff78a740 1 -- start start 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 Processor -- start 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4d7a30 0x55c55a4eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81adf30 con 0x55c55a4c8e50 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81ad860 con 0x55c55a4c8530 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf81aaac0 con 0x55c55a4d7a30 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 10 client.0 _ll_register_callbacks cb 0x7fbcf0031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35214/0 (socket says 172.21.15.45:35214) 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 -- 172.21.15.45:0/521791171 learned_addr learned my addr 172.21.15.45:0/521791171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:29:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 -- 172.21.15.45:0/521791171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4d7a30 msgr2=0x55c55a4eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4d7a30 0x55c55a4eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 -- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 msgr2=0x55c55a4ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4ed2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 -- 172.21.15.45:0/521791171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55a4eb3c0 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcf5d9b640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x55c55a4cb030 tx=0x7fbcec013990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b2e9dcbb667c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec023060 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf003f070 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.940+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9475+0+0 (secure 0 0 0) 0x7fbcec004230 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbcec02a050 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 10 client.17895 fetch_fsmap learned FSMap version 28 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c55a54dd80 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcf003f070 con 0x55c55a4c8e50 2024-04-25T00:29:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbcec003b90 con 0x55c55a4c8e50 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 10 client.17895 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 20 client.17895 populate_metadata read hostname 'smithi045' 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a54e4e0 0x55c55a550900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 4 client.17895 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c55a434f40 con 0x55c55a54e4e0 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.941+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a54e4e0 0x55c55a550900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcddffb640 10 client.17895 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcf6d9d640 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a54e4e0 0x55c55a550900 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719114ec11a68cf4 server_cookie=94d2eb587368b548 in_seq=0 out_seq=0 2024-04-25T00:29:39.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcddffb640 1 -- 172.21.15.45:0/521791171 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c55a434f40 con 0x55c55a54e4e0 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcddffb640 10 client.17895 handle_command_reply: tid=0 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 client.17895 shutdown 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 msgr2=0x55c55a4ea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x55c55a4cb030 tx=0x7fbcec013990 comp rx=0 tx=0).stop 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 shutdown_connections 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c55a4d7a30 0x55c55a4eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55c55a54e4e0 0x55c55a550900 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c55a4c8e50 0x55c55a4ea670 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcddffb640 0 client.17895 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 --2- 172.21.15.45:0/521791171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c55a4c8530 0x55c55a4ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 >> 172.21.15.45:0/521791171 conn(0x55c55a4c4950 msgr2=0x55c55a4e3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 shutdown_connections 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 1 -- 172.21.15.45:0/521791171 wait complete. 2024-04-25T00:29:39.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcff78a740 20 client.17895 trim_cache size 0 max 16384 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "7f6cf0ab-2b86-4342-9f66-1d99112c999c": { 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "7f6cf0ab-2b86-4342-9f66-1d99112c999c", 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:29:39.950 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:29:39.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 msgr2=0x7fbce00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 0x7fbce00a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce80cd0b0 tx=0x7fbce8091260 comp rx=0 tx=0).stop 2024-04-25T00:29:39.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 msgr2=0x7fbcf8197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:39.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.942+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbce800e9f0 tx=0x7fbce800edf0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 shutdown_connections 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbce00a2d00 0x7fbce00a51c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcf8107730 0x7fbcf819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcf806a780 0x7fbcf8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 --2- 172.21.15.45:0/1200018867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf8069dc0 0x7fbcf8197fe0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 >> 172.21.15.45:0/1200018867 conn(0x7fbcf80776a0 msgr2=0x7fbcf80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 shutdown_connections 2024-04-25T00:29:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:29:39.943+0000 7fbcfc9ab640 1 -- 172.21.15.45:0/1200018867 wait complete. 2024-04-25T00:29:39.954 INFO:tasks.cephfs.filesystem:scrub status for tag:7f6cf0ab-2b86-4342-9f66-1d99112c999c - {'path': '/', 'tag': '7f6cf0ab-2b86-4342-9f66-1d99112c999c', 'options': 'recursive,force'} 2024-04-25T00:29:40.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:40 smithi071 ceph-mon[32667]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:40.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:40 smithi045 ceph-mon[29194]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:41.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:40 smithi122 ceph-mon[32959]: pgmap v236: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:29:41.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:41 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:29:41.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:41 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:41.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:41 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:29:41.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:41 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:41.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:41 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:29:41.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:41 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:41.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:41 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:29:41.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:41 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:42.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:41 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:29:42.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:41 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:42.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:41 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:29:42.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:41 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:29:42.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:42 smithi071 ceph-mon[32667]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:42.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:42 smithi045 ceph-mon[29194]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:43.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:42 smithi122 ceph-mon[32959]: pgmap v237: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:43.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:43 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:29:43.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:43 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:29:43.957 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:43 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:29:44.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:44 smithi071 ceph-mon[32667]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:29:44.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:44 smithi045 ceph-mon[29194]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:29:45.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:44 smithi122 ceph-mon[32959]: pgmap v238: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:29:46.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:46 smithi071 ceph-mon[32667]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:46.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:46 smithi045 ceph-mon[29194]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:47.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:46 smithi122 ceph-mon[32959]: pgmap v239: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:48 smithi122 ceph-mon[32959]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:48 smithi071 ceph-mon[32667]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:48 smithi045 ceph-mon[29194]: pgmap v240: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:51.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:50 smithi122 ceph-mon[32959]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:29:51.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:50 smithi071 ceph-mon[32667]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:29:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:50 smithi045 ceph-mon[29194]: pgmap v241: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:29:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:52 smithi122 ceph-mon[32959]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:52 smithi071 ceph-mon[32667]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:52 smithi045 ceph-mon[29194]: pgmap v242: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:29:55.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:54 smithi122 ceph-mon[32959]: pgmap v243: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:29:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:54 smithi071 ceph-mon[32667]: pgmap v243: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:29:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:54 smithi045 ceph-mon[29194]: pgmap v243: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:29:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:56 smithi122 ceph-mon[32959]: pgmap v244: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:57.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:56 smithi071 ceph-mon[32667]: pgmap v244: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:56 smithi045 ceph-mon[29194]: pgmap v244: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:29:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:29:58 smithi122 ceph-mon[32959]: pgmap v245: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:29:58 smithi071 ceph-mon[32667]: pgmap v245: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:29:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:29:58 smithi045 ceph-mon[29194]: pgmap v245: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:01.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:00 smithi122 ceph-mon[32959]: pgmap v246: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:01.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:00 smithi122 ceph-mon[32959]: overall HEALTH_OK 2024-04-25T00:30:01.373 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:00 smithi071 ceph-mon[32667]: pgmap v246: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:01.373 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:00 smithi071 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T00:30:01.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:00 smithi045 ceph-mon[29194]: pgmap v246: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:01.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:00 smithi045 ceph-mon[29194]: overall HEALTH_OK 2024-04-25T00:30:03.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:02 smithi122 ceph-mon[32959]: pgmap v247: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:02 smithi071 ceph-mon[32667]: pgmap v247: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:02 smithi045 ceph-mon[29194]: pgmap v247: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:05.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:04 smithi122 ceph-mon[32959]: pgmap v248: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:30:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:04 smithi071 ceph-mon[32667]: pgmap v248: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:30:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:04 smithi045 ceph-mon[29194]: pgmap v248: 129 pgs: 129 active+clean; 259 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:30:07.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:06 smithi122 ceph-mon[32959]: pgmap v249: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:30:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:06 smithi071 ceph-mon[32667]: pgmap v249: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:30:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:06 smithi045 ceph-mon[29194]: pgmap v249: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:30:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:09 smithi122 ceph-mon[32959]: pgmap v250: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:30:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:09 smithi122 ceph-mon[32959]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:30:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:09 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:09 smithi071 ceph-mon[32667]: pgmap v250: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:30:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:09 smithi071 ceph-mon[32667]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:30:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:09 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:09 smithi045 ceph-mon[29194]: pgmap v250: 129 pgs: 129 active+clean; 259 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:30:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:09 smithi045 ceph-mon[29194]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:30:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:09 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:09.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:30:09.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:30:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 -- 172.21.15.45:0/989452377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc106630 msgr2=0x7f77bc106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 --2- 172.21.15.45:0/989452377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc106630 0x7f77bc106a10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009930 tx=0x7f77ac02a420 comp rx=0 tx=0).stop 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 -- 172.21.15.45:0/989452377 shutdown_connections 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 --2- 172.21.15.45:0/989452377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc100fe0 0x7f77bc10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 --2- 172.21.15.45:0/989452377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100620 0x7f77bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 --2- 172.21.15.45:0/989452377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc106630 0x7f77bc106a10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 -- 172.21.15.45:0/989452377 >> 172.21.15.45:0/989452377 conn(0x7f77bc0776a0 msgr2=0x7f77bc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 -- 172.21.15.45:0/989452377 shutdown_connections 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.064+0000 7f77c2c12640 1 -- 172.21.15.45:0/989452377 wait complete. 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 Processor -- start 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 -- start start 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100fe0 0x7f77bc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc106630 0x7f77bc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc110a40 con 0x7f77bc100620 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1108c0 con 0x7f77bc106630 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c2c12640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc110bc0 con 0x7f77bc100fe0 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57546/0 (socket says 172.21.15.45:57546) 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 -- 172.21.15.45:0/3357962771 learned_addr learned my addr 172.21.15.45:0/3357962771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77bbfff640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100fe0 0x7f77bc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 -- 172.21.15.45:0/3357962771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100fe0 msgr2=0x7f77bc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100fe0 0x7f77bc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c09ae640 1 -- 172.21.15.45:0/3357962771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc106630 msgr2=0x7f77bc19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.065+0000 7f77c11af640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc106630 0x7f77bc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77c09ae640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc106630 0x7f77bc19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77c09ae640 1 -- 172.21.15.45:0/3357962771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc19d0a0 con 0x7f77bc100620 2024-04-25T00:30:10.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77c09ae640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009230 tx=0x7f77ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77b9ffb640 1 -- 172.21.15.45:0/3357962771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac03f030 con 0x7f77bc100620 2024-04-25T00:30:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc19d330 con 0x7f77bc100620 2024-04-25T00:30:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77b9ffb640 1 -- 172.21.15.45:0/3357962771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac003ad0 con 0x7f77bc100620 2024-04-25T00:30:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77b9ffb640 1 -- 172.21.15.45:0/3357962771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac035070 con 0x7f77bc100620 2024-04-25T00:30:10.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.066+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc1a4c20 con 0x7f77bc100620 2024-04-25T00:30:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.068+0000 7f77b9ffb640 1 -- 172.21.15.45:0/3357962771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f77ac042030 con 0x7f77bc100620 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.068+0000 7f77c59fa740 1 Processor -- start 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.068+0000 7f77c59fa740 1 -- start start 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9b4e6b0 con 0x55bbd9be9f50 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9af6880 con 0x55bbd9be9630 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c59fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9b4a670 con 0x55bbd9bf4620 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57554/0 (socket says 172.21.15.45:57554) 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 -- 172.21.15.45:0/2330758867 learned_addr learned my addr 172.21.15.45:0/2330758867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77b9ffb640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 0x7f77a40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77bbfff640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c11af640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c11af640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 0x7f77a40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77b9ffb640 1 -- 172.21.15.45:0/3357962771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f77ac0efa60 con 0x7f77bc100620 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 -- 172.21.15.45:0/2330758867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 msgr2=0x55bbd9bf6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 -- 172.21.15.45:0/2330758867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 msgr2=0x55bbd9be9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 -- 172.21.15.45:0/2330758867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd992c7d0 con 0x55bbd9be9f50 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c11af640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.069+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0bb810 tx=0x7f77ac0b7d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a59b330849ffa33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77a2ffd640 1 -- 172.21.15.45:0/2330758867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac049070 con 0x55bbd9be9f50 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c11af640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 0x7f77a40a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099a0 tx=0x7f77b4009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77a2ffd640 1 -- 172.21.15.45:0/2330758867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac0bc210 con 0x55bbd9be9f50 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77a2ffd640 1 -- 172.21.15.45:0/2330758867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0b7860 con 0x55bbd9be9f50 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 -- 172.21.15.45:0/2330758867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 msgr2=0x55bbd9bf40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0bb810 tx=0x7f77ac0b7d10 comp rx=0 tx=0).stop 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 -- 172.21.15.45:0/2330758867 shutdown_connections 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf40e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2330758867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 -- 172.21.15.45:0/2330758867 >> 172.21.15.45:0/2330758867 conn(0x55bbd9be5950 msgr2=0x55bbd9be5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 -- 172.21.15.45:0/2330758867 shutdown_connections 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.070+0000 7f77c59fa740 1 -- 172.21.15.45:0/2330758867 wait complete. 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 -- start start 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 Processor -- start 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bf1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9b4e6b0 con 0x55bbd9be9f50 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9b09f50 con 0x55bbd9bf4620 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbd9bf71d0 con 0x55bbd9be9630 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77c59fa740 10 client.0 _ll_register_callbacks cb 0x55bbd9b45000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57566/0 (socket says 172.21.15.45:57566) 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.071+0000 7f77bbfff640 1 -- 172.21.15.45:0/2844096774 learned_addr learned my addr 172.21.15.45:0/2844096774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77c11af640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bf1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 -- 172.21.15.45:0/2844096774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 msgr2=0x55bbd9bf1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bf1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 -- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9bf4620 msgr2=0x55bbd9bf1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 -- 172.21.15.45:0/2844096774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd98623f0 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77bbfff640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f77b000bf40 tx=0x7f77b00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da79112b1acba9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f779bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b001f060 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f77b00108a0 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ac0bc210 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77b0026050 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77c59fa740 10 client.17907 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbd98220f0 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.072+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac0bc210 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77b0019030 con 0x55bbd9be9f50 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 10 client.17907 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:10.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 20 client.17907 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c04630 0x55bbd9c06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 4 client.17907 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbd9b5bbf0 con 0x55bbd9c04630 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c04630 0x55bbd9c06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f779bfff640 10 client.17907 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.073+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c04630 0x55bbd9c06a90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0fe92c9eeddcb6 server_cookie=654023d916e417d4 in_seq=0 out_seq=0 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f779bfff640 1 -- 172.21.15.45:0/2844096774 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbd9b5bbf0 con 0x55bbd9c04630 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f779bfff640 10 client.17907 handle_command_reply: tid=0 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 client.17907 shutdown 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 msgr2=0x55bbd9bf1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f77b000bf40 tx=0x7f77b00128e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 shutdown_connections 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c04630 0x55bbd9c06a90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bf1aa0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f779bfff640 0 client.17907 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2844096774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bf1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 >> 172.21.15.45:0/2844096774 conn(0x55bbd9be5950 msgr2=0x55bbd9beb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 shutdown_connections 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 1 -- 172.21.15.45:0/2844096774 wait complete. 2024-04-25T00:30:10.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.074+0000 7f77c59fa740 20 client.17907 trim_cache size 0 max 16384 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 Processor -- start 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 -- start start 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1aeda0 con 0x55bbd9be9f50 2024-04-25T00:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1b08e0 con 0x55bbd9be9630 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c59fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1af470 con 0x55bbd9bf4620 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57568/0 (socket says 172.21.15.45:57568) 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 -- 172.21.15.45:0/1634050288 learned_addr learned my addr 172.21.15.45:0/1634050288 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c09ae640 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 -- 172.21.15.45:0/1634050288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 msgr2=0x55bbd9bf3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 -- 172.21.15.45:0/1634050288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 msgr2=0x55bbd9be5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 -- 172.21.15.45:0/1634050288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd9bec2d0 con 0x55bbd9be9f50 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77c09ae640 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.092+0000 7f77bbfff640 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ba90 tx=0x7f77b00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac31e9b9d10a7b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77a27fc640 1 -- 172.21.15.45:0/1634050288 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b002e070 con 0x55bbd9be9f50 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77a27fc640 1 -- 172.21.15.45:0/1634050288 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b000d5a0 con 0x55bbd9be9f50 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77a27fc640 1 -- 172.21.15.45:0/1634050288 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0010d40 con 0x55bbd9be9f50 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 -- 172.21.15.45:0/1634050288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 msgr2=0x55bbd9bebd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f77b000ba90 tx=0x7f77b00057e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 -- 172.21.15.45:0/1634050288 shutdown_connections 2024-04-25T00:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9bf4620 0x55bbd9bf3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9be9f50 0x55bbd9bebd90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 --2- 172.21.15.45:0/1634050288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9630 0x55bbd9be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 -- 172.21.15.45:0/1634050288 >> 172.21.15.45:0/1634050288 conn(0x55bbd9be5950 msgr2=0x55bbd9bf2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 -- 172.21.15.45:0/1634050288 shutdown_connections 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.093+0000 7f77c59fa740 1 -- 172.21.15.45:0/1634050288 wait complete. 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 -- start start 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 Processor -- start 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9f50 0x55bbd9beea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1b0210 con 0x55bbd9bf4620 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1afb40 con 0x55bbd9be9f50 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc1acda0 con 0x55bbd9be9630 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c59fa740 10 client.0 _ll_register_callbacks cb 0x7f77b000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c11af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c11af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57574/0 (socket says 172.21.15.45:57574) 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c11af640 1 -- 172.21.15.45:0/2223651241 learned_addr learned my addr 172.21.15.45:0/2223651241 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77bbfff640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9f50 0x55bbd9beea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.094+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 -- 172.21.15.45:0/2223651241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 msgr2=0x55bbd9bee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 -- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9f50 msgr2=0x55bbd9beea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9f50 0x55bbd9beea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 -- 172.21.15.45:0/2223651241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd9bef300 con 0x55bbd9bf4620 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c11af640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f77b4011d80 tx=0x7f77b4009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c684c64c53dd9216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f779bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4027020 con 0x55bbd9bf4620 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f77b4023470 con 0x55bbd9bf4620 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b000d5a0 con 0x55bbd9bf4620 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77b4032050 con 0x55bbd9bf4620 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c59fa740 10 client.17913 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.095+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbd9c6b1c0 con 0x55bbd9bf4620 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b000d5a0 con 0x55bbd9bf4620 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77b4021750 con 0x55bbd9bf4620 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 10 client.17913 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 10 client.17913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 20 client.17913 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c6b9c0 0x55bbd9c6dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 4 client.17913 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbd9b55f40 con 0x55bbd9c6b9c0 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c6b9c0 0x55bbd9c6dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f779bfff640 10 client.17913 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c09ae640 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c6b9c0 0x55bbd9c6dde0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f04ee877d7eb853 server_cookie=69dfc45e9c2cd3a9 in_seq=0 out_seq=0 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f779bfff640 1 -- 172.21.15.45:0/2223651241 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbd9b55f40 con 0x55bbd9c6b9c0 2024-04-25T00:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f779bfff640 10 client.17913 handle_command_reply: tid=0 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 1 client.17913 shutdown 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 msgr2=0x55bbd9c69bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.096+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f77b4011d80 tx=0x7f77b4009910 comp rx=0 tx=0).stop 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 shutdown_connections 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55bbd9c6b9c0 0x55bbd9c6dde0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bbd9bf4620 0x55bbd9c69bd0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bbd9be9f50 0x55bbd9beea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f779bfff640 0 client.17913 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 --2- 172.21.15.45:0/2223651241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbd9be9630 0x55bbd9bee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 >> 172.21.15.45:0/2223651241 conn(0x55bbd9be5950 msgr2=0x55bbd9bf1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 shutdown_connections 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 1 -- 172.21.15.45:0/2223651241 wait complete. 2024-04-25T00:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c59fa740 20 client.17913 trim_cache size 0 max 16384 2024-04-25T00:30:10.106 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:10.106 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:30:10.106 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:30:10.106 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 msgr2=0x7f77a40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 0x7f77a40a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77b40099a0 tx=0x7f77b4009500 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 msgr2=0x7f77bc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009230 tx=0x7f77ac004270 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 shutdown_connections 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77a40a2d00 0x7f77a40a51c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77bc106630 0x7f77bc19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77bc100fe0 0x7f77bc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 --2- 172.21.15.45:0/3357962771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77bc100620 0x7f77bc1980b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 >> 172.21.15.45:0/3357962771 conn(0x7f77bc0776a0 msgr2=0x7f77bc06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 shutdown_connections 2024-04-25T00:30:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.097+0000 7f77c2c12640 1 -- 172.21.15.45:0/3357962771 wait complete. 2024-04-25T00:30:10.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:30:10.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:30:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.210+0000 7fa16d9f9640 1 -- 172.21.15.45:0/957662219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100fe0 msgr2=0x7fa16810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.210+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/957662219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100fe0 0x7fa16810d4f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa160009cd0 tx=0x7fa16002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 -- 172.21.15.45:0/957662219 shutdown_connections 2024-04-25T00:30:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/957662219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100fe0 0x7fa16810d4f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/957662219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168100620 0x7fa168100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/957662219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168106630 0x7fa168106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 -- 172.21.15.45:0/957662219 >> 172.21.15.45:0/957662219 conn(0x7fa168077740 msgr2=0x7fa168077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 -- 172.21.15.45:0/957662219 shutdown_connections 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 -- 172.21.15.45:0/957662219 wait complete. 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 Processor -- start 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.211+0000 7fa16d9f9640 1 -- start start 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 0x7fa1681984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168106630 0x7fa16819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa168110a40 con 0x7fa168100620 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1681108c0 con 0x7fa168100fe0 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa168110bc0 con 0x7fa168106630 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57600/0 (socket says 172.21.15.45:57600) 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 -- 172.21.15.45:0/3423162751 learned_addr learned my addr 172.21.15.45:0/3423162751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa1667fc640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 0x7fa1681984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa1677fe640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168106630 0x7fa16819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 -- 172.21.15.45:0/3423162751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168106630 msgr2=0x7fa16819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168106630 0x7fa16819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 -- 172.21.15.45:0/3423162751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 msgr2=0x7fa1681984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 0x7fa1681984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 -- 172.21.15.45:0/3423162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16819cf50 con 0x7fa168100620 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa1667fc640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 0x7fa1681984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa166ffd640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa15800ea60 tx=0x7fa15800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15800cde0 con 0x7fa168100620 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16819d240 con 0x7fa168100620 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1580035f0 con 0x7fa168100620 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158010670 con 0x7fa168100620 2024-04-25T00:30:10.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.212+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1681a4ad0 con 0x7fa168100620 2024-04-25T00:30:10.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.214+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa168068560 con 0x7fa168100620 2024-04-25T00:30:10.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.215+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa1580037a0 con 0x7fa168100620 2024-04-25T00:30:10.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.215+0000 7fa16c9f7640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 0x7fa1500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.215+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fa1580cfaa0 con 0x7fa168100620 2024-04-25T00:30:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.215+0000 7fa1667fc640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 0x7fa1500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.216+0000 7fa1667fc640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 0x7fa1500a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa168102750 tx=0x7fa15c00b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.220+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa15809c2b0 con 0x7fa168100620 2024-04-25T00:30:10.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:10 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:10.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:10 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:10.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:10 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:30:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:10 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:10 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:10 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:30:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:10 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:10 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:10 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:30:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.442+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1681992b0 con 0x7fa168100620 2024-04-25T00:30:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.443+0000 7fa16c9f7640 1 -- 172.21.15.45:0/3423162751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fa15809bbb0 con 0x7fa168100620 2024-04-25T00:30:10.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:30:10.444 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:30:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 msgr2=0x7fa1500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 0x7fa1500a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa168102750 tx=0x7fa15c00b390 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 msgr2=0x7fa168197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa15800ea60 tx=0x7fa15800ef20 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 shutdown_connections 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa1500a2d00 0x7fa1500a51c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa168106630 0x7fa16819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168100fe0 0x7fa1681984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 --2- 172.21.15.45:0/3423162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa168100620 0x7fa168197f60 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 >> 172.21.15.45:0/3423162751 conn(0x7fa168077740 msgr2=0x7fa168068b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 shutdown_connections 2024-04-25T00:30:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.445+0000 7fa16d9f9640 1 -- 172.21.15.45:0/3423162751 wait complete. 2024-04-25T00:30:10.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:30:10.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 -- 172.21.15.45:0/2336444664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd94101710 msgr2=0x7fcd94101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/2336444664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd94101710 0x7fcd94101b90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009930 tx=0x7fcd8402a420 comp rx=0 tx=0).stop 2024-04-25T00:30:10.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 -- 172.21.15.45:0/2336444664 shutdown_connections 2024-04-25T00:30:10.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/2336444664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd941020d0 0x7fcd9410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/2336444664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd94101710 0x7fcd94101b90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/2336444664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 0x7fcd94107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.559+0000 7fcd9bf09640 1 -- 172.21.15.45:0/2336444664 >> 172.21.15.45:0/2336444664 conn(0x7fcd940fd180 msgr2=0x7fcd940ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- 172.21.15.45:0/2336444664 shutdown_connections 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- 172.21.15.45:0/2336444664 wait complete. 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 Processor -- start 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- start start 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94101710 0x7fcd94198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 0x7fcd9419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94111af0 con 0x7fcd941020d0 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94111970 con 0x7fcd94101710 2024-04-25T00:30:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd9bf09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94111c70 con 0x7fcd94107720 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57618/0 (socket says 172.21.15.45:57618) 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.560+0000 7fcd994a4640 1 -- 172.21.15.45:0/1315284552 learned_addr learned my addr 172.21.15.45:0/1315284552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 0x7fcd9419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94101710 0x7fcd94198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 -- 172.21.15.45:0/1315284552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 msgr2=0x7fcd9419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 0x7fcd9419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 -- 172.21.15.45:0/1315284552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94101710 msgr2=0x7fcd94198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94101710 0x7fcd94198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 -- 172.21.15.45:0/1315284552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9419d080 con 0x7fcd941020d0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009230 tx=0x7fcd84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd8affd640 1 -- 172.21.15.45:0/1315284552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8403f030 con 0x7fcd941020d0 2024-04-25T00:30:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9419d310 con 0x7fcd941020d0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd8affd640 1 -- 172.21.15.45:0/1315284552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd84003ad0 con 0x7fcd941020d0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd8affd640 1 -- 172.21.15.45:0/1315284552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84049060 con 0x7fcd941020d0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.561+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd941a4c00 con 0x7fcd941020d0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.563+0000 7fcd8affd640 1 -- 172.21.15.45:0/1315284552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd84042030 con 0x7fcd941020d0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.563+0000 7fcd9ec0b740 1 Processor -- start 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 -- start start 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d428670 con 0x55ac4d4c3580 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d3d0970 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d424630 con 0x55ac4d4ce570 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd8affd640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 0x7fcd800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9a4a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9a4a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57624/0 (socket says 172.21.15.45:57624) 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9a4a6640 1 -- 172.21.15.45:0/4149980131 learned_addr learned my addr 172.21.15.45:0/4149980131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 0x7fcd800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd8affd640 1 -- 172.21.15.45:0/1315284552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcd840f0160 con 0x7fcd941020d0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd994a4640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4149980131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 msgr2=0x55ac4d4d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4149980131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 msgr2=0x55ac4d4c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4149980131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4d3f4400 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd994a4640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.564+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4ce030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000db70 tx=0x7fcd9000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895a30df5b16d78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 0x7fcd800a51c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009fd0 tx=0x7fcd8c005bd0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd7ffff640 1 -- 172.21.15.45:0/4149980131 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000de60 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd7ffff640 1 -- 172.21.15.45:0/4149980131 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd90005500 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd7ffff640 1 -- 172.21.15.45:0/4149980131 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd900116a0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4149980131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 msgr2=0x55ac4d4ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.565+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4ce030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000db70 tx=0x7fcd9000f730 comp rx=0 tx=0).stop 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4149980131 shutdown_connections 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4ce030 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4149980131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3580 0x55ac4d4c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4149980131 >> 172.21.15.45:0/4149980131 conn(0x55ac4d4bf8a0 msgr2=0x55ac4d4bfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4149980131 shutdown_connections 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4149980131 wait complete. 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- start start 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 Processor -- start 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d4d1360 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d3e4050 con 0x55ac4d4c3580 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ac4d4d14e0 con 0x55ac4d4ce570 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.566+0000 7fcd9ec0b740 10 client.0 _ll_register_callbacks cb 0x55ac4d13bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:44260/0 (socket says 172.21.15.45:44260) 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57638/0 (socket says 172.21.15.45:57638) 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4123119933 learned_addr learned my addr 172.21.15.45:0/4123119933 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 msgr2=0x55ac4d4cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4123119933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 msgr2=0x55ac4d4cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 -- 172.21.15.45:0/4123119933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4d3b5dc0 con 0x55ac4d4c3580 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd994a4640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c00 tx=0x7fcd9000d8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37c95012246612e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9002a060 con 0x55ac4d4c3580 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcd900046c0 con 0x55ac4d4c3580 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd90005500 con 0x55ac4d4c3580 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd90031050 con 0x55ac4d4c3580 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd9ec0b740 10 client.27155 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.567+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac4d13c3f0 con 0x55ac4d4c3580 2024-04-25T00:30:10.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd90005500 con 0x55ac4d4c3580 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd90011900 con 0x55ac4d4c3580 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 10 client.27155 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 10 client.27155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 20 client.27155 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d4de4d0 0x55ac4d4e08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 4 client.27155 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4d435bb0 con 0x55ac4d4de4d0 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd994a4640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d4de4d0 0x55ac4d4e08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd7d7fa640 10 client.27155 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:10.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.568+0000 7fcd994a4640 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d4de4d0 0x55ac4d4e08c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c728ea5f01c5b93d server_cookie=1d829ce35e0a242f in_seq=0 out_seq=0 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/4123119933 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac4d435bb0 con 0x55ac4d4de4d0 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd7d7fa640 10 client.27155 handle_command_reply: tid=0 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 client.27155 shutdown 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 msgr2=0x55ac4d4cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c00 tx=0x7fcd9000d8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 shutdown_connections 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d4de4d0 0x55ac4d4e08c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/4123119933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d4cdee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 >> 172.21.15.45:0/4123119933 conn(0x55ac4d4bf8a0 msgr2=0x55ac4d4beb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd7d7fa640 0 client.27155 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 shutdown_connections 2024-04-25T00:30:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.569+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/4123119933 wait complete. 2024-04-25T00:30:10.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.570+0000 7fcd9ec0b740 20 client.27155 trim_cache size 0 max 16384 2024-04-25T00:30:10.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 Processor -- start 2024-04-25T00:30:10.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 -- start start 2024-04-25T00:30:10.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4c3580 0x55ac4d4da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941aee20 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941b0960 con 0x55ac4d4ce570 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941af4f0 con 0x55ac4d4c3580 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57640/0 (socket says 172.21.15.45:57640) 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 -- 172.21.15.45:0/1061045025 learned_addr learned my addr 172.21.15.45:0/1061045025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4c3580 0x55ac4d4da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 -- 172.21.15.45:0/1061045025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4c3580 msgr2=0x55ac4d4da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4c3580 0x55ac4d4da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 -- 172.21.15.45:0/1061045025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4ce570 msgr2=0x55ac4d4cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.583+0000 7fcd994a4640 1 -- 172.21.15.45:0/1061045025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4d4cbaa0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd994a4640 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ed550 tx=0x7fcd840dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57def6d27ef67fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd7f7fe640 1 -- 172.21.15.45:0/1061045025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84035040 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd7f7fe640 1 -- 172.21.15.45:0/1061045025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd840c4ee0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd7f7fe640 1 -- 172.21.15.45:0/1061045025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b88e0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/1061045025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 msgr2=0x55ac4d4be9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcd840ed550 tx=0x7fcd840dc040 comp rx=0 tx=0).stop 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/1061045025 shutdown_connections 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4ce570 0x55ac4d4cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d4be9c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/1061045025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4c3580 0x55ac4d4da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/1061045025 >> 172.21.15.45:0/1061045025 conn(0x55ac4d4bf8a0 msgr2=0x55ac4d4be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/1061045025 shutdown_connections 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.584+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/1061045025 wait complete. 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 -- start start 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 Processor -- start 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d53c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d53d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941b0290 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941afbc0 con 0x55ac4d4c3580 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941ace20 con 0x55ac4d4ce570 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9ec0b740 10 client.0 _ll_register_callbacks cb 0x7fcd90015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57644/0 (socket says 172.21.15.45:57644) 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 -- 172.21.15.45:0/2529980959 learned_addr learned my addr 172.21.15.45:0/2529980959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d53d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d53c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 -- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 msgr2=0x55ac4d53d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d53d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 -- 172.21.15.45:0/2529980959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 msgr2=0x55ac4d53c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d53c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd994a4640 1 -- 172.21.15.45:0/2529980959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4d2067d0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd9a4a6640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d53d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.585+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d53c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd994a4640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009a50 tx=0x7fcd840c06e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4735f21132cb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84035040 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcd840ed720 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd840c4ee0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd840d5210 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 10 client.17937 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac4d5489a0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd840c4ee0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd84031bd0 con 0x55ac4d4c3ea0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 10 client.17937 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 10 client.17937 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 20 client.17937 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d549030 0x55ac4d54b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 4 client.17937 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.586+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac4d42ff00 con 0x55ac4d549030 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d549030 0x55ac4d54b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd99ca5640 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d549030 0x55ac4d54b450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9f4faf25d3e11a8 server_cookie=2687b84a968c5ec in_seq=0 out_seq=0 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd7d7fa640 10 client.17937 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:10.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd7d7fa640 1 -- 172.21.15.45:0/2529980959 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac4d42ff00 con 0x55ac4d549030 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd7d7fa640 10 client.17937 handle_command_reply: tid=0 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 client.17937 shutdown 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 msgr2=0x55ac4d53cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009a50 tx=0x7fcd840c06e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 shutdown_connections 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x55ac4d549030 0x55ac4d54b450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ac4d4ce570 0x55ac4d53d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd7d7fa640 0 client.17937 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac4d4c3ea0 0x55ac4d53cc00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 --2- 172.21.15.45:0/2529980959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac4d4c3580 0x55ac4d53c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 >> 172.21.15.45:0/2529980959 conn(0x55ac4d4bf8a0 msgr2=0x55ac4d4beb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 shutdown_connections 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.587+0000 7fcd9ec0b740 1 -- 172.21.15.45:0/2529980959 wait complete. 2024-04-25T00:30:10.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9ec0b740 20 client.17937 trim_cache size 0 max 16384 2024-04-25T00:30:10.597 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 msgr2=0x7fcd800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 0x7fcd800a51c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009fd0 tx=0x7fcd8c005bd0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 msgr2=0x7fcd941985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009230 tx=0x7fcd84004270 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 shutdown_connections 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcd800a2d00 0x7fcd800a51c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd94107720 0x7fcd9419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd941020d0 0x7fcd941985d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 --2- 172.21.15.45:0/1315284552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94101710 0x7fcd94198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 >> 172.21.15.45:0/1315284552 conn(0x7fcd940fd180 msgr2=0x7fcd940fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 shutdown_connections 2024-04-25T00:30:10.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.588+0000 7fcd9bf09640 1 -- 172.21.15.45:0/1315284552 wait complete. 2024-04-25T00:30:10.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:30:10.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 -- 172.21.15.45:0/2263449000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d806a780 msgr2=0x7f18d810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 --2- 172.21.15.45:0/2263449000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d806a780 0x7f18d810e5f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009cd0 tx=0x7f18d002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 -- 172.21.15.45:0/2263449000 shutdown_connections 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 --2- 172.21.15.45:0/2263449000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d806a780 0x7f18d810e5f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 --2- 172.21.15.45:0/2263449000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d8069dc0 0x7f18d806a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 --2- 172.21.15.45:0/2263449000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8107730 0x7f18d8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.702+0000 7f18e00bc640 1 -- 172.21.15.45:0/2263449000 >> 172.21.15.45:0/2263449000 conn(0x7f18d80776a0 msgr2=0x7f18d8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- 172.21.15.45:0/2263449000 shutdown_connections 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- 172.21.15.45:0/2263449000 wait complete. 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 Processor -- start 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- start start 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8069dc0 0x7f18d8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8111b90 con 0x7f18d8107730 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8111a10 con 0x7f18d8069dc0 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.703+0000 7f18e00bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8111d10 con 0x7f18d806a780 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18de659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:50980/0 (socket says 172.21.15.45:50980) 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 -- 172.21.15.45:0/3399350514 learned_addr learned my addr 172.21.15.45:0/3399350514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18de659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57664/0 (socket says 172.21.15.45:57664) 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dde58640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8069dc0 0x7f18d8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 -- 172.21.15.45:0/3399350514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8069dc0 msgr2=0x7f18d8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8069dc0 0x7f18d8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 -- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 msgr2=0x7f18d819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 -- 172.21.15.45:0/3399350514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d819cfc0 con 0x7f18d806a780 2024-04-25T00:30:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18de659640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18dd657640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18ceffd640 1 -- 172.21.15.45:0/3399350514 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800cd80 con 0x7f18d806a780 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18ceffd640 1 -- 172.21.15.45:0/3399350514 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8005500 con 0x7f18d806a780 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d819d2b0 con 0x7f18d806a780 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18ceffd640 1 -- 172.21.15.45:0/3399350514 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8010690 con 0x7f18d806a780 2024-04-25T00:30:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.704+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d81a4b40 con 0x7f18d806a780 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.706+0000 7f18ceffd640 1 -- 172.21.15.45:0/3399350514 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f18c80046c0 con 0x7f18d806a780 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 Processor -- start 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 -- start start 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09580 0x562b2ca09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 0x562b2ca16960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b2c96e670 con 0x562b2ca09ea0 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b2c916970 con 0x562b2ca09580 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b2c96a630 con 0x562b2ca14570 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57666/0 (socket says 172.21.15.45:57666) 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 -- 172.21.15.45:0/1257546691 learned_addr learned my addr 172.21.15.45:0/1257546691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18ceffd640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 0x7f18c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18de659640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09580 0x562b2ca09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dd657640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 0x562b2ca16960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 -- 172.21.15.45:0/1257546691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 msgr2=0x562b2ca16960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18de659640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 0x7f18c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 0x562b2ca16960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 -- 172.21.15.45:0/1257546691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09580 msgr2=0x562b2ca09960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09580 0x562b2ca09960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18dde58640 1 -- 172.21.15.45:0/1257546691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b2c93a400 con 0x562b2ca09ea0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.707+0000 7f18ceffd640 1 -- 172.21.15.45:0/3399350514 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f18c80cfd30 con 0x7f18d806a780 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18dd657640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 0x562b2ca16960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18dde58640 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c00 tx=0x7f18d40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b20bd16985f7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18c3fff640 1 -- 172.21.15.45:0/1257546691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4012450 con 0x562b2ca09ea0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18c3fff640 1 -- 172.21.15.45:0/1257546691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012ad0 con 0x562b2ca09ea0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18de659640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 0x7f18c40a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d0005e30 tx=0x7f18d0032fd0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.708+0000 7f18c3fff640 1 -- 172.21.15.45:0/1257546691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d400fb90 con 0x562b2ca09ea0 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1257546691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 msgr2=0x562b2ca14030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c00 tx=0x7f18d40117b0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1257546691 shutdown_connections 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca14570 0x562b2ca16960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca14030 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1257546691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09580 0x562b2ca09960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1257546691 >> 172.21.15.45:0/1257546691 conn(0x562b2ca058a0 msgr2=0x562b2ca05c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1257546691 shutdown_connections 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1257546691 wait complete. 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- start start 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 Processor -- start 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca13ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 0x562b2ca111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b2ca17360 con 0x562b2ca14570 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b2c92a050 con 0x562b2ca09ea0 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.709+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b2ca174e0 con 0x562b2ca09580 2024-04-25T00:30:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18e2dc5740 10 client.0 _ll_register_callbacks cb 0x562b2c681be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18de659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18de659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57680/0 (socket says 172.21.15.45:57680) 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18de659640 1 -- 172.21.15.45:0/3339145356 learned_addr learned my addr 172.21.15.45:0/3339145356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 0x562b2ca111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dde58640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca13ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 -- 172.21.15.45:0/3339145356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 msgr2=0x562b2ca13ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca13ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 -- 172.21.15.45:0/3339145356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 msgr2=0x562b2ca11720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 -- 172.21.15.45:0/3339145356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2c8fbdc0 con 0x562b2ca09ea0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18de659640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18dd657640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 0x562b2ca111c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18c80cd070 tx=0x7f18c8066f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea2ef265b7b8e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c809e070 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f18c80984b0 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d4012ad0 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18c809c540 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18e2dc5740 10 client.27173 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.710+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b2c6823f0 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4012ad0 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f18c80a4620 con 0x562b2ca09ea0 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 10 client.27173 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 10 client.27173 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 20 client.27173 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca24a40 0x562b2ca26ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 4 client.27173 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b2c97bbb0 con 0x562b2ca24a40 2024-04-25T00:30:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18dde58640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca24a40 0x562b2ca26ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18c17fa640 10 client.27173 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.711+0000 7f18dde58640 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca24a40 0x562b2ca26ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54005701f4a08957 server_cookie=a9f500b400310b8e in_seq=0 out_seq=0 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18c17fa640 1 -- 172.21.15.45:0/3339145356 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b2c97bbb0 con 0x562b2ca24a40 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18c17fa640 10 client.27173 handle_command_reply: tid=0 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 client.27173 shutdown 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 msgr2=0x562b2ca111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 0x562b2ca111c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18c80cd070 tx=0x7f18c8066f20 comp rx=0 tx=0).stop 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 shutdown_connections 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca24a40 0x562b2ca26ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca14570 0x562b2ca11720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca09ea0 0x562b2ca111c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3339145356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca13ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18c17fa640 0 client.27173 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 >> 172.21.15.45:0/3339145356 conn(0x562b2ca058a0 msgr2=0x562b2ca04b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 shutdown_connections 2024-04-25T00:30:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3339145356 wait complete. 2024-04-25T00:30:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.712+0000 7f18e2dc5740 20 client.27173 trim_cache size 0 max 16384 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 Processor -- start 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 -- start start 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 0x562b2ca13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81aed30 con 0x562b2ca09580 2024-04-25T00:30:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81b0870 con 0x562b2ca14570 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81af400 con 0x562b2ca09ea0 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57682/0 (socket says 172.21.15.45:57682) 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 -- 172.21.15.45:0/3240907302 learned_addr learned my addr 172.21.15.45:0/3240907302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18de659640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dd657640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 0x562b2ca13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 -- 172.21.15.45:0/3240907302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 msgr2=0x562b2ca13a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 0x562b2ca13a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 -- 172.21.15.45:0/3240907302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 msgr2=0x562b2ca1c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dde58640 1 -- 172.21.15.45:0/3240907302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b2ca11a90 con 0x562b2ca09580 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.726+0000 7f18dd657640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 0x562b2ca13a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18dde58640 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009b50 tx=0x7f18d4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d9f5bf63c2c36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18c37fe640 1 -- 172.21.15.45:0/3240907302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d40050b0 con 0x562b2ca09580 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18c37fe640 1 -- 172.21.15.45:0/3240907302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4029070 con 0x562b2ca09580 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18c37fe640 1 -- 172.21.15.45:0/3240907302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003850 con 0x562b2ca09580 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3240907302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 msgr2=0x562b2ca04b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009b50 tx=0x7f18d4005ab0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3240907302 shutdown_connections 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09ea0 0x562b2ca13a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/3240907302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09580 0x562b2ca04b40 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3240907302 >> 172.21.15.45:0/3240907302 conn(0x562b2ca058a0 msgr2=0x562b2ca1f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3240907302 shutdown_connections 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.727+0000 7f18e2dc5740 1 -- 172.21.15.45:0/3240907302 wait complete. 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 -- start start 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 Processor -- start 2024-04-25T00:30:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca1c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81b01a0 con 0x562b2ca09ea0 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81afad0 con 0x562b2ca14570 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d81acd30 con 0x562b2ca09580 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18e2dc5740 10 client.0 _ll_register_callbacks cb 0x7f18c800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18dd657640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18dd657640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57692/0 (socket says 172.21.15.45:57692) 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18dd657640 1 -- 172.21.15.45:0/1188570674 learned_addr learned my addr 172.21.15.45:0/1188570674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.728+0000 7f18de659640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dde58640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca1c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 -- 172.21.15.45:0/1188570674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 msgr2=0x562b2ca1c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca1c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 -- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 msgr2=0x562b2ca1a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 -- 172.21.15.45:0/1188570674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2c74c7d0 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dde58640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18dd657640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9c0 tx=0x7f18c8004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e235313c9baa6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c809e070 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f18c809c0a0 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d4029070 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18c809b7a0 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18e2dc5740 10 client.17955 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b2ca8b8e0 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4029070 con 0x562b2ca09ea0 2024-04-25T00:30:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.729+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f18c809af00 con 0x562b2ca09ea0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 10 client.17955 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 10 client.17955 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 20 client.17955 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca8bed0 0x562b2ca8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 4 client.17955 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b2c975f00 con 0x562b2ca8bed0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18dde58640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca8bed0 0x562b2ca8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18c17fa640 10 client.17955 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18dde58640 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca8bed0 0x562b2ca8e2f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35ec542b0563100a server_cookie=f11cde43f818d51d in_seq=0 out_seq=0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18c17fa640 1 -- 172.21.15.45:0/1188570674 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b2c975f00 con 0x562b2ca8bed0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18c17fa640 10 client.17955 handle_command_reply: tid=0 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 client.17955 shutdown 2024-04-25T00:30:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 msgr2=0x562b2ca1caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9c0 tx=0x7f18c8004b60 comp rx=0 tx=0).stop 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 shutdown_connections 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x562b2ca8bed0 0x562b2ca8e2f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2ca14570 0x562b2ca1a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b2ca09ea0 0x562b2ca1caa0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 --2- 172.21.15.45:0/1188570674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b2ca09580 0x562b2ca1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 >> 172.21.15.45:0/1188570674 conn(0x562b2ca058a0 msgr2=0x562b2ca049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.730+0000 7f18c17fa640 0 client.17955 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 shutdown_connections 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e2dc5740 1 -- 172.21.15.45:0/1188570674 wait complete. 2024-04-25T00:30:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e2dc5740 20 client.17955 trim_cache size 0 max 16384 2024-04-25T00:30:10.739 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 msgr2=0x7f18c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 0x7f18c40a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d0005e30 tx=0x7f18d0032fd0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 msgr2=0x7f18d8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800edf0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 shutdown_connections 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f18c40a2d00 0x7f18c40a51c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d8107730 0x7f18d819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18d806a780 0x7f18d8198510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 --2- 172.21.15.45:0/3399350514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8069dc0 0x7f18d8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 >> 172.21.15.45:0/3399350514 conn(0x7f18d80776a0 msgr2=0x7f18d80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 shutdown_connections 2024-04-25T00:30:10.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.731+0000 7f18e00bc640 1 -- 172.21.15.45:0/3399350514 wait complete. 2024-04-25T00:30:10.743 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 -- 172.21.15.45:0/1981711070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c069dc0 msgr2=0x7f1e1c06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 --2- 172.21.15.45:0/1981711070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c06a240 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009930 tx=0x7f1e1002a420 comp rx=0 tx=0).stop 2024-04-25T00:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 -- 172.21.15.45:0/1981711070 shutdown_connections 2024-04-25T00:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 --2- 172.21.15.45:0/1981711070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c06a780 0x7f1e1c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 --2- 172.21.15.45:0/1981711070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c06a240 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 --2- 172.21.15.45:0/1981711070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c107730 0x7f1e1c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 -- 172.21.15.45:0/1981711070 >> 172.21.15.45:0/1981711070 conn(0x7f1e1c0776a0 msgr2=0x7f1e1c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 -- 172.21.15.45:0/1981711070 shutdown_connections 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.846+0000 7f1e2229e640 1 -- 172.21.15.45:0/1981711070 wait complete. 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 Processor -- start 2024-04-25T00:30:10.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 -- start start 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c06a780 0x7f1e1c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c111b90 con 0x7f1e1c107730 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c111a10 con 0x7f1e1c069dc0 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2229e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c111d10 con 0x7f1e1c06a780 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2083b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2083b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57714/0 (socket says 172.21.15.45:57714) 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e2083b640 1 -- 172.21.15.45:0/2075198355 learned_addr learned my addr 172.21.15.45:0/2075198355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c06a780 0x7f1e1c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.847+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 -- 172.21.15.45:0/2075198355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c06a780 msgr2=0x7f1e1c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c06a780 0x7f1e1c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 -- 172.21.15.45:0/2075198355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c069dc0 msgr2=0x7f1e1c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 -- 172.21.15.45:0/2075198355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c19cff0 con 0x7f1e1c107730 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2083b640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400eaa0 tx=0x7f1e1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e197fa640 1 -- 172.21.15.45:0/2075198355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400cdb0 con 0x7f1e1c107730 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c19d2e0 con 0x7f1e1c107730 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e197fa640 1 -- 172.21.15.45:0/2075198355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e14005020 con 0x7f1e1c107730 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e197fa640 1 -- 172.21.15.45:0/2075198355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140116b0 con 0x7f1e1c107730 2024-04-25T00:30:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.848+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c1a4b70 con 0x7f1e1c107730 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.850+0000 7f1e197fa640 1 -- 172.21.15.45:0/2075198355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1e14005560 con 0x7f1e1c107730 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.850+0000 7f1e24fad740 1 Processor -- start 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.850+0000 7f1e24fad740 1 -- start start 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 0x556908d84030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d84570 0x556908d86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556908cde670 con 0x556908d79580 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556908c86970 con 0x556908d84570 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e24fad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556908cda630 con 0x556908d79ea0 2024-04-25T00:30:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e197fa640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 0x7f1e040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57722/0 (socket says 172.21.15.45:57722) 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/1373654718 learned_addr learned my addr 172.21.15.45:0/1373654718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 0x556908d84030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e2083b640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 0x7f1e040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d84570 0x556908d86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e197fa640 1 -- 172.21.15.45:0/2075198355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f1e140cfa60 con 0x7f1e1c107730 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/1373654718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 msgr2=0x556908d84030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 0x556908d84030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/1373654718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d84570 msgr2=0x556908d86960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d84570 0x556908d86960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/1373654718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556908caa400 con 0x556908d79580 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 0x556908d84030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e2083b640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 0x7f1e040a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ccda0 tx=0x7f1e1406db80 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.851+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009930 tx=0x7f1e10005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6373954ce7a9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e027fc640 1 -- 172.21.15.45:0/1373654718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10044020 con 0x556908d79580 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e027fc640 1 -- 172.21.15.45:0/1373654718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e100044b0 con 0x556908d79580 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e027fc640 1 -- 172.21.15.45:0/1373654718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10035040 con 0x556908d79580 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 -- 172.21.15.45:0/1373654718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 msgr2=0x556908d79960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009930 tx=0x7f1e10005b50 comp rx=0 tx=0).stop 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 -- 172.21.15.45:0/1373654718 shutdown_connections 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d84570 0x556908d86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79ea0 0x556908d84030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 --2- 172.21.15.45:0/1373654718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d79960 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 -- 172.21.15.45:0/1373654718 >> 172.21.15.45:0/1373654718 conn(0x556908d758a0 msgr2=0x556908d75c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 -- 172.21.15.45:0/1373654718 shutdown_connections 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.852+0000 7f1e24fad740 1 -- 172.21.15.45:0/1373654718 wait complete. 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 -- start start 2024-04-25T00:30:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 Processor -- start 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79580 0x556908d83ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556908d87300 con 0x556908d84570 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556908c9a050 con 0x556908d79ea0 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556908d87480 con 0x556908d79580 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e24fad740 10 client.0 _ll_register_callbacks cb 0x5569089f1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57724/0 (socket says 172.21.15.45:57724) 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/3730738461 learned_addr learned my addr 172.21.15.45:0/3730738461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.853+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79580 0x556908d83ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79580 msgr2=0x556908d83ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79580 0x556908d83ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/3730738461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 msgr2=0x556908d811d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d811d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/3730738461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556908c6bdc0 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1e10003740 tx=0x7f1e1002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eb1eea92cc5926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10051020 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f1e10005240 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e100044b0 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e10035030 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 10 client.17967 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569089f23f0 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e100044b0 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e100315b0 con 0x556908d84570 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 10 client.17967 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 10 client.17967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 20 client.17967 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908d94420 0x556908d96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 4 client.17967 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.854+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556908cebbb0 con 0x556908d94420 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.855+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908d94420 0x556908d96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.855+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908d94420 0x556908d96810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3d069478021c0fc server_cookie=11f5e22983cef092 in_seq=0 out_seq=0 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.855+0000 7f1dfb7fe640 10 client.17967 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/3730738461 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556908cebbb0 con 0x556908d94420 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1dfb7fe640 10 client.17967 handle_command_reply: tid=0 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 client.17967 shutdown 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 msgr2=0x556908d81730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1e10003740 tx=0x7f1e1002a420 comp rx=0 tx=0).stop 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 shutdown_connections 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908d94420 0x556908d96810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d84570 0x556908d81730 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d811d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1dfb7fe640 0 client.17967 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3730738461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d79580 0x556908d83ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 >> 172.21.15.45:0/3730738461 conn(0x556908d758a0 msgr2=0x556908d74b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 shutdown_connections 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 1 -- 172.21.15.45:0/3730738461 wait complete. 2024-04-25T00:30:10.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.856+0000 7f1e24fad740 20 client.17967 trim_cache size 0 max 16384 2024-04-25T00:30:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.869+0000 7f1e24fad740 1 Processor -- start 2024-04-25T00:30:10.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.869+0000 7f1e24fad740 1 -- start start 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.869+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e24fad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1aec30 con 0x556908d79580 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e24fad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b0770 con 0x556908d79ea0 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e24fad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1af300 con 0x556908d84570 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:44390/0 (socket says 172.21.15.45:44390) 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 -- 172.21.15.45:0/3826909359 learned_addr learned my addr 172.21.15.45:0/3826909359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57738/0 (socket says 172.21.15.45:57738) 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 -- 172.21.15.45:0/3826909359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 msgr2=0x556908d99830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 -- 172.21.15.45:0/3826909359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 msgr2=0x556908d80080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 -- 172.21.15.45:0/3826909359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556908abc7d0 con 0x556908d79ea0 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556908d80a30 tx=0x7f1e0c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2da058ca3def2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e01ffb640 1 -- 172.21.15.45:0/3826909359 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c024070 con 0x556908d79ea0 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e01ffb640 1 -- 172.21.15.45:0/3826909359 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c003de0 con 0x556908d79ea0 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.870+0000 7f1e01ffb640 1 -- 172.21.15.45:0/3826909359 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c010980 con 0x556908d79ea0 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- 172.21.15.45:0/3826909359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 msgr2=0x556908d83a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556908d80a30 tx=0x7f1e0c005f90 comp rx=0 tx=0).stop 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- 172.21.15.45:0/3826909359 shutdown_connections 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d99830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d83a10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 --2- 172.21.15.45:0/3826909359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d80080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- 172.21.15.45:0/3826909359 >> 172.21.15.45:0/3826909359 conn(0x556908d758a0 msgr2=0x556908d7f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- 172.21.15.45:0/3826909359 shutdown_connections 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- 172.21.15.45:0/3826909359 wait complete. 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 -- start start 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.871+0000 7f1e24fad740 1 Processor -- start 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d97860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b00a0 con 0x556908d79580 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1af9d0 con 0x556908d79ea0 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1acc30 con 0x556908d84570 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 10 client.0 _ll_register_callbacks cb 0x7f1e0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57750/0 (socket says 172.21.15.45:57750) 2024-04-25T00:30:10.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/4059036980 learned_addr learned my addr 172.21.15.45:0/4059036980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 msgr2=0x556908d97860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d97860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/4059036980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 msgr2=0x556908d97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 -- 172.21.15.45:0/4059036980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556908dfda10 con 0x556908d79580 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e1b7fe640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005940 tx=0x7f1e10005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dc0f1e835b4e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1dfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10051070 con 0x556908d79580 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7f1e10005000 con 0x556908d79580 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.872+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0c003de0 con 0x556908d79580 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e10044070 con 0x556908d79580 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 10 client.17973 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:10.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556908dfeb40 con 0x556908d79580 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e0c003de0 con 0x556908d79580 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e10003d10 con 0x556908d79580 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 10 client.17973 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 10 client.17973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 20 client.17973 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908dff0d0 0x556908e014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 4 client.17973 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556908ce5f00 con 0x556908dff0d0 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.873+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908dff0d0 0x556908e014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e1bfff640 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908dff0d0 0x556908e014c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb88aacfa4eeec9 server_cookie=66f91d00bddb5583 in_seq=0 out_seq=0 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1dfb7fe640 10 client.17973 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:10.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1dfb7fe640 1 -- 172.21.15.45:0/4059036980 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556908ce5f00 con 0x556908dff0d0 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1dfb7fe640 10 client.17973 handle_command_reply: tid=0 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 client.17973 shutdown 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 msgr2=0x556908d99d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005940 tx=0x7f1e10005bc0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 shutdown_connections 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x556908dff0d0 0x556908e014c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556908d84570 0x556908d97860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1dfb7fe640 0 client.17973 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556908d79ea0 0x556908d97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 --2- 172.21.15.45:0/4059036980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556908d79580 0x556908d99d40 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 >> 172.21.15.45:0/4059036980 conn(0x556908d758a0 msgr2=0x556908d80200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 shutdown_connections 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 1 -- 172.21.15.45:0/4059036980 wait complete. 2024-04-25T00:30:10.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.874+0000 7f1e24fad740 20 client.17973 trim_cache size 0 max 16384 2024-04-25T00:30:10.882 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:10.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 msgr2=0x7f1e040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 0x7f1e040a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ccda0 tx=0x7f1e1406db80 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 msgr2=0x7f1e1c19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400eaa0 tx=0x7f1e1400ef60 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 shutdown_connections 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1e040a2d00 0x7f1e040a51c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e1c107730 0x7f1e1c19c8c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e1c06a780 0x7f1e1c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 --2- 172.21.15.45:0/2075198355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e1c069dc0 0x7f1e1c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 >> 172.21.15.45:0/2075198355 conn(0x7f1e1c0776a0 msgr2=0x7f1e1c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 shutdown_connections 2024-04-25T00:30:10.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.875+0000 7f1e2229e640 1 -- 172.21.15.45:0/2075198355 wait complete. 2024-04-25T00:30:10.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:30:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 -- 172.21.15.45:0/2866645145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 msgr2=0x7fce3010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 --2- 172.21.15.45:0/2866645145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce3010e5f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fce28009cd0 tx=0x7fce2802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 -- 172.21.15.45:0/2866645145 shutdown_connections 2024-04-25T00:30:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 --2- 172.21.15.45:0/2866645145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce3010e5f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 --2- 172.21.15.45:0/2866645145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce3006a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 --2- 172.21.15.45:0/2866645145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 0x7fce30107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 -- 172.21.15.45:0/2866645145 >> 172.21.15.45:0/2866645145 conn(0x7fce300776a0 msgr2=0x7fce30077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 -- 172.21.15.45:0/2866645145 shutdown_connections 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.990+0000 7fce36141640 1 -- 172.21.15.45:0/2866645145 wait complete. 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 Processor -- start 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 -- start start 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce30197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 0x7fce3019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce30111b40 con 0x7fce3006a780 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce301119c0 con 0x7fce30107730 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce36141640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce30111cc0 con 0x7fce30069dc0 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57778/0 (socket says 172.21.15.45:57778) 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 -- 172.21.15.45:0/2468425976 learned_addr learned my addr 172.21.15.45:0/2468425976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2ffff640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 0x7fce3019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce30197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 -- 172.21.15.45:0/2468425976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 msgr2=0x7fce30197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce30197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 -- 172.21.15.45:0/2468425976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 msgr2=0x7fce3019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 0x7fce3019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2effd640 1 -- 172.21.15.45:0/2468425976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3019cf90 con 0x7fce3006a780 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.991+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce30197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce2effd640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e9f0 tx=0x7fce2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce2cff9640 1 -- 172.21.15.45:0/2468425976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2400cd80 con 0x7fce3006a780 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce2cff9640 1 -- 172.21.15.45:0/2468425976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce24005500 con 0x7fce3006a780 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3019d280 con 0x7fce3006a780 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce2cff9640 1 -- 172.21.15.45:0/2468425976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24010690 con 0x7fce3006a780 2024-04-25T00:30:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.992+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce301a4b10 con 0x7fce3006a780 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce2cff9640 1 -- 172.21.15.45:0/2468425976 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce240046c0 con 0x7fce3006a780 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 Processor -- start 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 -- start start 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad4989f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad497f9670 con 0x55ad4989f570 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ad497a1970 con 0x55ad49894580 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.994+0000 7fce38e45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ad497f5630 con 0x55ad49894ea0 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57788/0 (socket says 172.21.15.45:57788) 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 -- 172.21.15.45:0/2780731553 learned_addr learned my addr 172.21.15.45:0/2780731553 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:44444/0 (socket says 172.21.15.45:44444) 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad4989f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2cff9640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 0x7fce180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 -- 172.21.15.45:0/2780731553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 msgr2=0x55ad4989f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad4989f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 -- 172.21.15.45:0/2780731553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 msgr2=0x55ad49894960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2cff9640 1 -- 172.21.15.45:0/2468425976 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fce240cfd30 con 0x7fce3006a780 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 -- 172.21.15.45:0/2780731553 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad497c5400 con 0x55ad4989f570 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2ffff640 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2ffff640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 0x7fce180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce2effd640 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fce2409bb40 tx=0x7fce24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc286d45b3a1614 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce15ffb640 1 -- 172.21.15.45:0/2780731553 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2409e040 con 0x55ad4989f570 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce15ffb640 1 -- 172.21.15.45:0/2780731553 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce24059030 con 0x55ad4989f570 2024-04-25T00:30:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.995+0000 7fce15ffb640 1 -- 172.21.15.45:0/2780731553 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240a4ab0 con 0x55ad4989f570 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- 172.21.15.45:0/2780731553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 msgr2=0x55ad498a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce2ffff640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 0x7fce180a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce28004ab0 tx=0x7fce280095c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fce2409bb40 tx=0x7fce24004a40 comp rx=0 tx=0).stop 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- 172.21.15.45:0/2780731553 shutdown_connections 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad4989f570 0x55ad498a1960 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad4989f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 --2- 172.21.15.45:0/2780731553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad49894580 0x55ad49894960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- 172.21.15.45:0/2780731553 >> 172.21.15.45:0/2780731553 conn(0x55ad498908a0 msgr2=0x55ad49890c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- 172.21.15.45:0/2780731553 shutdown_connections 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- 172.21.15.45:0/2780731553 wait complete. 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.996+0000 7fce38e45740 1 -- start start 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 Processor -- start 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad4989ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad4989c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad498a2300 con 0x55ad49894ea0 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ad497b5050 con 0x55ad4989f570 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ad498a2480 con 0x55ad49894580 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce38e45740 10 client.0 _ll_register_callbacks cb 0x55ad4950cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57796/0 (socket says 172.21.15.45:57796) 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 -- 172.21.15.45:0/1092700208 learned_addr learned my addr 172.21.15.45:0/1092700208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad4989ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2ffff640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad4989c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 -- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 msgr2=0x55ad4989ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad4989ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 -- 172.21.15.45:0/1092700208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 msgr2=0x55ad4989c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad4989c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2effd640 1 -- 172.21.15.45:0/1092700208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad49786dc0 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.997+0000 7fce2ffff640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad4989c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce2effd640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fce240a03b0 tx=0x7fce240a01f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a14d028308b4ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2409e040 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fce24061150 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce24059030 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce2409c540 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 10 client.17985 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad4950d3f0 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce24059030 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce240a46b0 con 0x55ad49894ea0 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 10 client.17985 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 10 client.17985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 20 client.17985 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad498af430 0x55ad498b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 4 client.17985 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.998+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad49806bb0 con 0x55ad498af430 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.999+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad498af430 0x55ad498b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.999+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad498af430 0x55ad498b1820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9759bdc2ac46333 server_cookie=aba80ab29c0f2c5d in_seq=0 out_seq=0 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:10.999+0000 7fce0effd640 10 client.17985 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce0effd640 1 -- 172.21.15.45:0/1092700208 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad49806bb0 con 0x55ad498af430 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce0effd640 10 client.17985 handle_command_reply: tid=0 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 client.17985 shutdown 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 msgr2=0x55ad4989c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fce240a03b0 tx=0x7fce240a01f0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 shutdown_connections 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad498af430 0x55ad498b1820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad4989c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce0effd640 0 client.17985 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad4989c210 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 --2- 172.21.15.45:0/1092700208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad4989ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 >> 172.21.15.45:0/1092700208 conn(0x55ad498908a0 msgr2=0x55ad4988fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 shutdown_connections 2024-04-25T00:30:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 1 -- 172.21.15.45:0/1092700208 wait complete. 2024-04-25T00:30:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.000+0000 7fce38e45740 20 client.17985 trim_cache size 0 max 16384 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 Processor -- start 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 -- start start 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad498b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce301aeda0 con 0x55ad49894580 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce301b08e0 con 0x55ad4989f570 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce38e45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce301af470 con 0x55ad49894ea0 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57808/0 (socket says 172.21.15.45:57808) 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 -- 172.21.15.45:0/4140288190 learned_addr learned my addr 172.21.15.45:0/4140288190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2effd640 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad498b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2ffff640 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 -- 172.21.15.45:0/4140288190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 msgr2=0x55ad498b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad498b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 -- 172.21.15.45:0/4140288190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 msgr2=0x55ad498b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.014+0000 7fce2f7fe640 1 -- 172.21.15.45:0/4140288190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad495d77d0 con 0x55ad49894580 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55ad498a25b0 tx=0x7fce20014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0709cf720f5c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce157fa640 1 -- 172.21.15.45:0/4140288190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20024070 con 0x55ad49894580 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce157fa640 1 -- 172.21.15.45:0/4140288190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce2001e430 con 0x55ad49894580 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce157fa640 1 -- 172.21.15.45:0/4140288190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20003b20 con 0x55ad49894580 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 -- 172.21.15.45:0/4140288190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 msgr2=0x55ad498ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55ad498a25b0 tx=0x7fce20014840 comp rx=0 tx=0).stop 2024-04-25T00:30:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 -- 172.21.15.45:0/4140288190 shutdown_connections 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894ea0 0x55ad498b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.015+0000 7fce38e45740 1 --2- 172.21.15.45:0/4140288190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894580 0x55ad498ab1c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- 172.21.15.45:0/4140288190 >> 172.21.15.45:0/4140288190 conn(0x55ad498908a0 msgr2=0x55ad4988fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- 172.21.15.45:0/4140288190 shutdown_connections 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- 172.21.15.45:0/4140288190 wait complete. 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- start start 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 Processor -- start 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad498a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498a5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce301b0210 con 0x55ad49894ea0 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce301afb40 con 0x55ad4989f570 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce301acda0 con 0x55ad49894580 2024-04-25T00:30:11.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce38e45740 10 client.0 _ll_register_callbacks cb 0x7fce24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce2effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57814/0 (socket says 172.21.15.45:57814) 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.016+0000 7fce2effd640 1 -- 172.21.15.45:0/1017010481 learned_addr learned my addr 172.21.15.45:0/1017010481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad498a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2ffff640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 -- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 msgr2=0x55ad498a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad498a7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 -- 172.21.15.45:0/1017010481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 msgr2=0x55ad498a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 -- 172.21.15.45:0/1017010481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad498a5840 con 0x55ad49894ea0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce2effd640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fce240cd120 tx=0x7fce24005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d4e51382c3d92a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce0effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2409e070 con 0x55ad49894ea0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fce2409a140 con 0x55ad49894ea0 2024-04-25T00:30:11.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2001e430 con 0x55ad49894ea0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce240980e0 con 0x55ad49894ea0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce38e45740 10 client.17991 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad499154b0 con 0x55ad49894ea0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.017+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce2001e430 con 0x55ad49894ea0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce2409ad00 con 0x55ad49894ea0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 10 client.17991 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 10 client.17991 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 20 client.17991 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad49915980 0x55ad49917d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 4 client.17991 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad49800f00 con 0x55ad49915980 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad49915980 0x55ad49917d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce0effd640 10 client.17991 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce2f7fe640 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad49915980 0x55ad49917d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b212d8349e182086 server_cookie=82cd8cc3fe754e5f in_seq=0 out_seq=0 2024-04-25T00:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce0effd640 1 -- 172.21.15.45:0/1017010481 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad49800f00 con 0x55ad49915980 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce0effd640 10 client.17991 handle_command_reply: tid=0 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.018+0000 7fce38e45740 1 client.17991 shutdown 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 msgr2=0x55ad498a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fce240cd120 tx=0x7fce24005730 comp rx=0 tx=0).stop 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 shutdown_connections 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55ad49915980 0x55ad49917d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad4989f570 0x55ad498a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad49894ea0 0x55ad498a4cc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 --2- 172.21.15.45:0/1017010481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ad49894580 0x55ad498a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce0effd640 0 client.17991 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 >> 172.21.15.45:0/1017010481 conn(0x55ad498908a0 msgr2=0x55ad498aa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 shutdown_connections 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 1 -- 172.21.15.45:0/1017010481 wait complete. 2024-04-25T00:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce38e45740 20 client.17991 trim_cache size 0 max 16384 2024-04-25T00:30:11.026 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:11.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 msgr2=0x7fce180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 0x7fce180a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce28004ab0 tx=0x7fce280095c0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 msgr2=0x7fce301984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fce2400e9f0 tx=0x7fce2400edf0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 shutdown_connections 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fce180a2d00 0x7fce180a51c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce30107730 0x7fce3019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce3006a780 0x7fce301984e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 --2- 172.21.15.45:0/2468425976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce30069dc0 0x7fce30197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.019+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 >> 172.21.15.45:0/2468425976 conn(0x7fce300776a0 msgr2=0x7fce300fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.020+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 shutdown_connections 2024-04-25T00:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.020+0000 7fce36141640 1 -- 172.21.15.45:0/2468425976 wait complete. 2024-04-25T00:30:11.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:30:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.135+0000 7fcc0033b640 1 -- 172.21.15.45:0/134479350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf810bd00 msgr2=0x7fcbf810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.135+0000 7fcc0033b640 1 --2- 172.21.15.45:0/134479350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf810bd00 0x7fcbf810e1a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009ca0 tx=0x7fcbf002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 -- 172.21.15.45:0/134479350 shutdown_connections 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- 172.21.15.45:0/134479350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf810bd00 0x7fcbf810e1a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- 172.21.15.45:0/134479350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 0x7fcbf810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- 172.21.15.45:0/134479350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf81029c0 0x7fcbf8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 -- 172.21.15.45:0/134479350 >> 172.21.15.45:0/134479350 conn(0x7fcbf8077740 msgr2=0x7fcbf8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 -- 172.21.15.45:0/134479350 shutdown_connections 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 -- 172.21.15.45:0/134479350 wait complete. 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 Processor -- start 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 -- start start 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 0x7fcbf8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.136+0000 7fcc0033b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf810bd00 0x7fcbf819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcc0033b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8111b00 con 0x7fcbf81029c0 2024-04-25T00:30:11.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcc0033b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8111980 con 0x7fcbf810bd00 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcc0033b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8111c80 con 0x7fcbf81032e0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57830/0 (socket says 172.21.15.45:57830) 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/3881151560 learned_addr learned my addr 172.21.15.45:0/3881151560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf810bd00 0x7fcbf819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 0x7fcbf8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/3881151560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 msgr2=0x7fcbf8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 0x7fcbf8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/3881151560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf810bd00 msgr2=0x7fcbf819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf810bd00 0x7fcbf819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/3881151560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf819cfa0 con 0x7fcbf81029c0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400e9f0 tx=0x7fcbf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbef7fe640 1 -- 172.21.15.45:0/3881151560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400cd80 con 0x7fcbf81029c0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbef7fe640 1 -- 172.21.15.45:0/3881151560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf4010910 con 0x7fcbf81029c0 2024-04-25T00:30:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf819d290 con 0x7fcbf81029c0 2024-04-25T00:30:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcbef7fe640 1 -- 172.21.15.45:0/3881151560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4018990 con 0x7fcbf81029c0 2024-04-25T00:30:11.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.137+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf81a4b20 con 0x7fcbf81029c0 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.139+0000 7fcbef7fe640 1 -- 172.21.15.45:0/3881151560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcbf4010ac0 con 0x7fcbf81029c0 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 Processor -- start 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 -- start start 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b59030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcbef7fe640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 0x7fcbe40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5ab3670 con 0x55dbb5b4e580 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5a5b970 con 0x55dbb5b4eea0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.140+0000 7fcc03049740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5aaf630 con 0x55dbb5b59570 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 0x7fcbe40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b59030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57840/0 (socket says 172.21.15.45:57840) 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 -- 172.21.15.45:0/391648574 learned_addr learned my addr 172.21.15.45:0/391648574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbef7fe640 1 -- 172.21.15.45:0/3881151560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcbf40cfbe0 con 0x7fcbf81029c0 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 -- 172.21.15.45:0/391648574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 msgr2=0x55dbb5b5b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b5b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 -- 172.21.15.45:0/391648574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 msgr2=0x55dbb5b59030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b59030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 -- 172.21.15.45:0/391648574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb5a7f400 con 0x55dbb5b4e580 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 0x7fcbe40a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8005e00 tx=0x7fcbe8005c50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9156db0a75341129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbecbfe640 1 -- 172.21.15.45:0/391648574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0033030 con 0x55dbb5b4e580 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbecbfe640 1 -- 172.21.15.45:0/391648574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf0042040 con 0x55dbb5b4e580 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.141+0000 7fcbecbfe640 1 -- 172.21.15.45:0/391648574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0004290 con 0x55dbb5b4e580 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 -- 172.21.15.45:0/391648574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 msgr2=0x55dbb5b4e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf0005c50 comp rx=0 tx=0).stop 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 -- 172.21.15.45:0/391648574 shutdown_connections 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b59030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 --2- 172.21.15.45:0/391648574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b4e960 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 -- 172.21.15.45:0/391648574 >> 172.21.15.45:0/391648574 conn(0x55dbb5b4a8a0 msgr2=0x55dbb5b4ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 -- 172.21.15.45:0/391648574 shutdown_connections 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.142+0000 7fcc03049740 1 -- 172.21.15.45:0/391648574 wait complete. 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 -- start start 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 Processor -- start 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5b5c300 con 0x55dbb5b4e580 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5a6f050 con 0x55dbb5b4eea0 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dbb5b5c480 con 0x55dbb5b59570 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcc03049740 10 client.0 _ll_register_callbacks cb 0x55dbb57c6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57850/0 (socket says 172.21.15.45:57850) 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/309339466 learned_addr learned my addr 172.21.15.45:0/309339466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/309339466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 msgr2=0x55dbb5b56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b56730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 msgr2=0x55dbb5b561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.143+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/309339466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb5a40dc0 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4098660 tx=0x7fcbf409b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31c7333353020d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40cd350 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcbf40abb00 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbf0042040 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbf40abd00 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 10 client.18003 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbb57c73f0 con 0x55dbb5b4e580 2024-04-25T00:30:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf0042040 con 0x55dbb5b4e580 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcbf4018bc0 con 0x55dbb5b4e580 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 10 client.18003 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 20 client.18003 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5b69500 0x55dbb5b6b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 4 client.18003 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.144+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbb5ac0bb0 con 0x55dbb5b69500 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.145+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5b69500 0x55dbb5b6b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.145+0000 7fcbe1ffb640 10 client.18003 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.145+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5b69500 0x55dbb5b6b970 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a55b0fb567f70e server_cookie=a7f562601e484939 in_seq=0 out_seq=0 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/309339466 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbb5ac0bb0 con 0x55dbb5b69500 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcbe1ffb640 10 client.18003 handle_command_reply: tid=0 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 client.18003 shutdown 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 msgr2=0x55dbb5b58ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4098660 tx=0x7fcbf409b6b0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 shutdown_connections 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5b69500 0x55dbb5b6b970 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 --2- 172.21.15.45:0/309339466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5b58ee0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcbe1ffb640 0 client.18003 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 >> 172.21.15.45:0/309339466 conn(0x55dbb5b4a8a0 msgr2=0x55dbb5b49b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 shutdown_connections 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 1 -- 172.21.15.45:0/309339466 wait complete. 2024-04-25T00:30:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.146+0000 7fcc03049740 20 client.18003 trim_cache size 0 max 16384 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 Processor -- start 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 -- start start 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 0x55dbb5b61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81aedd0 con 0x55dbb5b4eea0 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b0910 con 0x55dbb5b4e580 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.159+0000 7fcc03049740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81af4a0 con 0x55dbb5b59570 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57864/0 (socket says 172.21.15.45:57864) 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 -- 172.21.15.45:0/255473689 learned_addr learned my addr 172.21.15.45:0/255473689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 0x55dbb5b61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 -- 172.21.15.45:0/255473689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 msgr2=0x55dbb5b58a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b58a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 -- 172.21.15.45:0/255473689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 msgr2=0x55dbb5b61ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 0x55dbb5b61ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 -- 172.21.15.45:0/255473689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb58917d0 con 0x55dbb5b4eea0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 0x55dbb5b61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8100040 tx=0x7fcbe8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193fbf52eb55228b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbe3fff640 1 -- 172.21.15.45:0/255473689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe802b070 con 0x55dbb5b4eea0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbe3fff640 1 -- 172.21.15.45:0/255473689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8011d70 con 0x55dbb5b4eea0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.160+0000 7fcbe3fff640 1 -- 172.21.15.45:0/255473689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe801e680 con 0x55dbb5b4eea0 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- 172.21.15.45:0/255473689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 msgr2=0x55dbb5b50560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8100040 tx=0x7fcbe8015000 comp rx=0 tx=0).stop 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- 172.21.15.45:0/255473689 shutdown_connections 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5b58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4eea0 0x55dbb5b50560 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 --2- 172.21.15.45:0/255473689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4e580 0x55dbb5b61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- 172.21.15.45:0/255473689 >> 172.21.15.45:0/255473689 conn(0x55dbb5b4a8a0 msgr2=0x55dbb5b50e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- 172.21.15.45:0/255473689 shutdown_connections 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- 172.21.15.45:0/255473689 wait complete. 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 -- start start 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.161+0000 7fcc03049740 1 Processor -- start 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5bcdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5bc7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81b0240 con 0x55dbb5b4e580 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81afb70 con 0x55dbb5b4eea0 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81acdd0 con 0x55dbb5b59570 2024-04-25T00:30:11.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 10 client.0 _ll_register_callbacks cb 0x7fcbf400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfd8d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5bcdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57874/0 (socket says 172.21.15.45:57874) 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/2842027672 learned_addr learned my addr 172.21.15.45:0/2842027672 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5bc7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/2842027672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 msgr2=0x55dbb5bc7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5bc7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 msgr2=0x55dbb5bcdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5bcdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 -- 172.21.15.45:0/2842027672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb5bc8340 con 0x55dbb5b4e580 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe8d8640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5bc7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:11.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbfe0d7640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4018460 tx=0x7fcbf40a4720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cff4e99470606bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40cd350 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9563+0+0 (secure 0 0 0) 0x7fcbf40abb00 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe8011d70 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbf40a4900 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.162+0000 7fcc03049740 10 client.18009 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbb5bd4ff0 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8011d70 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcbf409ab50 con 0x55dbb5b4e580 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 10 client.18009 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:11.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 20 client.18009 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5bd55d0 0x55dbb5bd79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 4 client.18009 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbb5abaf00 con 0x55dbb5bd55d0 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5bd55d0 0x55dbb5bd79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcbe1ffb640 10 client.18009 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.163+0000 7fcbfd8d6640 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5bd55d0 0x55dbb5bd79c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ecf46e559344cdf server_cookie=1180a43063349b20 in_seq=0 out_seq=0 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcbe1ffb640 1 -- 172.21.15.45:0/2842027672 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbb5abaf00 con 0x55dbb5bd55d0 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcbe1ffb640 10 client.18009 handle_command_reply: tid=0 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 client.18009 shutdown 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 msgr2=0x55dbb5bcd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4018460 tx=0x7fcbf40a4720 comp rx=0 tx=0).stop 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 shutdown_connections 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dbb5b59570 0x55dbb5bc7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dbb5b4eea0 0x55dbb5bcdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dbb5bd55d0 0x55dbb5bd79c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 --2- 172.21.15.45:0/2842027672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbb5b4e580 0x55dbb5bcd7f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcbe1ffb640 0 client.18009 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 >> 172.21.15.45:0/2842027672 conn(0x55dbb5b4a8a0 msgr2=0x55dbb5b641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 shutdown_connections 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 1 -- 172.21.15.45:0/2842027672 wait complete. 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc03049740 20 client.18009 trim_cache size 0 max 16384 2024-04-25T00:30:11.171 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 msgr2=0x7fcbe40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 0x7fcbe40a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8005e00 tx=0x7fcbe8005c50 comp rx=0 tx=0).stop 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 msgr2=0x7fcbf8198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400e9f0 tx=0x7fcbf400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 shutdown_connections 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcbe40a2d80 0x7fcbe40a5240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf810bd00 0x7fcbf819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf81032e0 0x7fcbf8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 --2- 172.21.15.45:0/3881151560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbf81029c0 0x7fcbf8198000 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 >> 172.21.15.45:0/3881151560 conn(0x7fcbf8077740 msgr2=0x7fcbf80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 shutdown_connections 2024-04-25T00:30:11.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:11.164+0000 7fcc0033b640 1 -- 172.21.15.45:0/3881151560 wait complete. 2024-04-25T00:30:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:11 smithi122 ceph-mon[32959]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:11.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:11 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3423162751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:11 smithi071 ceph-mon[32667]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:11 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3423162751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:11.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:11 smithi045 ceph-mon[29194]: pgmap v252: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:11.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:11 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3423162751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:12.176 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:30:12.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:12.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 -- 172.21.15.45:0/599537583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4106630 msgr2=0x7f14b4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 --2- 172.21.15.45:0/599537583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4106630 0x7f14b4106a10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009930 tx=0x7f14a402a420 comp rx=0 tx=0).stop 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 -- 172.21.15.45:0/599537583 shutdown_connections 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 --2- 172.21.15.45:0/599537583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4100fe0 0x7f14b410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 --2- 172.21.15.45:0/599537583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100620 0x7f14b4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 --2- 172.21.15.45:0/599537583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4106630 0x7f14b4106a10 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 -- 172.21.15.45:0/599537583 >> 172.21.15.45:0/599537583 conn(0x7f14b40776a0 msgr2=0x7f14b4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 -- 172.21.15.45:0/599537583 shutdown_connections 2024-04-25T00:30:12.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.295+0000 7f14b9612640 1 -- 172.21.15.45:0/599537583 wait complete. 2024-04-25T00:30:12.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 Processor -- start 2024-04-25T00:30:12.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 -- start start 2024-04-25T00:30:12.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100fe0 0x7f14b4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 0x7f14b41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4110a00 con 0x7f14b4100620 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4110880 con 0x7f14b4106630 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b9612640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4110b80 con 0x7f14b4100fe0 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57884/0 (socket says 172.21.15.45:57884) 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b2ffd640 1 -- 172.21.15.45:0/1702352957 learned_addr learned my addr 172.21.15.45:0/1702352957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b27fc640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100fe0 0x7f14b4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.296+0000 7f14b37fe640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 0x7f14b41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 -- 172.21.15.45:0/1702352957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100fe0 msgr2=0x7f14b4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100fe0 0x7f14b4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 -- 172.21.15.45:0/1702352957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 msgr2=0x7f14b41998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 0x7f14b41998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 -- 172.21.15.45:0/1702352957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41a4220 con 0x7f14b4100620 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b37fe640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 0x7f14b41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009230 tx=0x7f14a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f149bfff640 1 -- 172.21.15.45:0/1702352957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a403f030 con 0x7f14b4100620 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f149bfff640 1 -- 172.21.15.45:0/1702352957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4003ad0 con 0x7f14b4100620 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41a44b0 con 0x7f14b4100620 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f149bfff640 1 -- 172.21.15.45:0/1702352957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4035070 con 0x7f14b4100620 2024-04-25T00:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.297+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41a49a0 con 0x7f14b4100620 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.299+0000 7f149bfff640 1 -- 172.21.15.45:0/1702352957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14a4042030 con 0x7f14b4100620 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.299+0000 7f14bc3f9740 1 Processor -- start 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.299+0000 7f14bc3f9740 1 -- start start 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b8980 0x561ef79b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 0x561ef79b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ef791b2c0 con 0x561ef79c2010 2024-04-25T00:30:12.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561ef78c3910 con 0x561ef79b7290 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561ef7917280 con 0x561ef79b8980 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57888/0 (socket says 172.21.15.45:57888) 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 -- 172.21.15.45:0/875481002 learned_addr learned my addr 172.21.15.45:0/875481002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f149bfff640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 0x7f149c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 0x561ef79b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b37fe640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b8980 0x561ef79b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b37fe640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 0x7f149c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 -- 172.21.15.45:0/875481002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b8980 msgr2=0x561ef79b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b8980 0x561ef79b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 -- 172.21.15.45:0/875481002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 msgr2=0x561ef79b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 0x561ef79b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 -- 172.21.15.45:0/875481002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef78d4fb0 con 0x561ef79c2010 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 0x561ef79b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f149bfff640 1 -- 172.21.15.45:0/1702352957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14a40efcc0 con 0x7f14b4100620 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.300+0000 7f14b27fc640 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f14a800aca0 tx=0x7f14a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d1af79736a46bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14997fa640 1 -- 172.21.15.45:0/875481002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a801d070 con 0x561ef79c2010 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14b37fe640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 0x7f149c0a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005c40 tx=0x7f14ac009500 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14997fa640 1 -- 172.21.15.45:0/875481002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a8018b20 con 0x561ef79c2010 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14997fa640 1 -- 172.21.15.45:0/875481002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8010c20 con 0x561ef79c2010 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 -- 172.21.15.45:0/875481002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 msgr2=0x561ef79c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f14a800aca0 tx=0x7f14a800f790 comp rx=0 tx=0).stop 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 -- 172.21.15.45:0/875481002 shutdown_connections 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79c4400 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b7290 0x561ef79b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/875481002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b8980 0x561ef79b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 -- 172.21.15.45:0/875481002 >> 172.21.15.45:0/875481002 conn(0x561ef797f950 msgr2=0x561ef79b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 -- 172.21.15.45:0/875481002 shutdown_connections 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.301+0000 7f14bc3f9740 1 -- 172.21.15.45:0/875481002 wait complete. 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 -- start start 2024-04-25T00:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 Processor -- start 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79c2010 0x561ef79bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561ef79c4ea0 con 0x561ef79b8980 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561ef78d6e90 con 0x561ef79c2010 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561ef79c5020 con 0x561ef79b7290 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14bc3f9740 10 client.0 _ll_register_callbacks cb 0x561ef7912970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57898/0 (socket says 172.21.15.45:57898) 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 -- 172.21.15.45:0/649835584 learned_addr learned my addr 172.21.15.45:0/649835584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b37fe640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79c2010 0x561ef79bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 -- 172.21.15.45:0/649835584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 msgr2=0x561ef79bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 -- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79c2010 msgr2=0x561ef79bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79c2010 0x561ef79bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.302+0000 7f14b27fc640 1 -- 172.21.15.45:0/649835584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef76f77e0 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14b27fc640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f14a800a8a0 tx=0x7f14a8010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241d391be4ac3cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a801d050 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14a8018440 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a8018b20 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14a8032050 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 10 client.18021 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ef7757f70 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a8018b20 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14a80046c0 con 0x561ef79b8980 2024-04-25T00:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 10 client.18021 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 20 client.18021 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef79d1f70 0x561ef79d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 4 client.18021 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.303+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef78ef5a0 con 0x561ef79d1f70 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.304+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef79d1f70 0x561ef79d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.304+0000 7f1496ff5640 10 client.18021 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.304+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef79d1f70 0x561ef79d4360 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44281fb2bbd2cd9b server_cookie=d09cb2347c7fc726 in_seq=0 out_seq=0 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f1496ff5640 1 -- 172.21.15.45:0/649835584 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ef78ef5a0 con 0x561ef79d1f70 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f1496ff5640 10 client.18021 handle_command_reply: tid=0 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 client.18021 shutdown 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 msgr2=0x561ef79bf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f14a800a8a0 tx=0x7f14a8010660 comp rx=0 tx=0).stop 2024-04-25T00:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 shutdown_connections 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79c2010 0x561ef79bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef79d1f70 0x561ef79d4360 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79b8980 0x561ef79bf450 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/649835584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f1496ff5640 0 client.18021 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 >> 172.21.15.45:0/649835584 conn(0x561ef797f950 msgr2=0x561ef79b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 shutdown_connections 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 1 -- 172.21.15.45:0/649835584 wait complete. 2024-04-25T00:30:12.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.305+0000 7f14bc3f9740 20 client.18021 trim_cache size 0 max 16384 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 Processor -- start 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 -- start start 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41aebf0 con 0x561ef79c2010 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b0730 con 0x561ef79b8980 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41af2c0 con 0x561ef79b7290 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57902/0 (socket says 172.21.15.45:57902) 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14b37fe640 1 -- 172.21.15.45:0/667667559 learned_addr learned my addr 172.21.15.45:0/667667559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.323+0000 7f14b27fc640 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 -- 172.21.15.45:0/667667559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 msgr2=0x561ef79cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 -- 172.21.15.45:0/667667559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 msgr2=0x561ef79be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 -- 172.21.15.45:0/667667559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef75ef170 con 0x561ef79c2010 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f14b37fe640 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0125a0 tx=0x7f14ac0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bafaffd47eef9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f1498ff9640 1 -- 172.21.15.45:0/667667559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac018070 con 0x561ef79c2010 2024-04-25T00:30:12.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f1498ff9640 1 -- 172.21.15.45:0/667667559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac01c430 con 0x561ef79c2010 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.324+0000 7f1498ff9640 1 -- 172.21.15.45:0/667667559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00f680 con 0x561ef79c2010 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- 172.21.15.45:0/667667559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 msgr2=0x561ef79ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0125a0 tx=0x7f14ac0099e0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- 172.21.15.45:0/667667559 shutdown_connections 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79ca690 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/667667559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- 172.21.15.45:0/667667559 >> 172.21.15.45:0/667667559 conn(0x561ef797f950 msgr2=0x561ef79b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- 172.21.15.45:0/667667559 shutdown_connections 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- 172.21.15.45:0/667667559 wait complete. 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- start start 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 Processor -- start 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b0060 con 0x561ef79c2010 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41af990 con 0x561ef79b8980 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41acbf0 con 0x561ef79b7290 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14bc3f9740 10 client.0 _ll_register_callbacks cb 0x7f14a801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57904/0 (socket says 172.21.15.45:57904) 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.325+0000 7f14b37fe640 1 -- 172.21.15.45:0/827617146 learned_addr learned my addr 172.21.15.45:0/827617146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b27fc640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 -- 172.21.15.45:0/827617146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 msgr2=0x561ef79bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 -- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 msgr2=0x561ef79bca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79bca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 -- 172.21.15.45:0/827617146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef79bba30 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14b37fe640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01cea0 tx=0x7f14ac009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e8413bae87780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f1496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac018040 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14ac01c7f0 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ac01c430 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14ac00e570 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14bc3f9740 10 client.18027 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ef7a3ccc0 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.326+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac01c430 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14ac027070 con 0x561ef79c2010 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 10 client.18027 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 20 client.18027 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:12.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef7a3d320 0x561ef7a3f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 4 client.18027 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:30:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ef7928800 con 0x561ef7a3d320 2024-04-25T00:30:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef7a3d320 0x561ef7a3f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f14b2ffd640 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef7a3d320 0x561ef7a3f740 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57fe74c6a58cf92 server_cookie=714d1421b196b2a5 in_seq=0 out_seq=0 2024-04-25T00:30:12.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.327+0000 7f1496ff5640 10 client.18027 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.328+0000 7f1496ff5640 1 -- 172.21.15.45:0/827617146 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ef7928800 con 0x561ef7a3d320 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.328+0000 7f1496ff5640 10 client.18027 handle_command_reply: tid=0 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.328+0000 7f14bc3f9740 1 client.18027 shutdown 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 msgr2=0x561ef79bb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01cea0 tx=0x7f14ac009570 comp rx=0 tx=0).stop 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 shutdown_connections 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561ef7a3d320 0x561ef7a3f740 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561ef79c2010 0x561ef79bb200 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ef79b8980 0x561ef79bca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 --2- 172.21.15.45:0/827617146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ef79b7290 0x561ef79bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 >> 172.21.15.45:0/827617146 conn(0x561ef797f950 msgr2=0x561ef79b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f1496ff5640 0 client.18027 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 shutdown_connections 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 1 -- 172.21.15.45:0/827617146 wait complete. 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14bc3f9740 20 client.18027 trim_cache size 0 max 16384 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "2ff2f25e-01de-42d4-bd08-2f8777a5978e", 2024-04-25T00:30:12.335 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:30:12.336 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 msgr2=0x7f149c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 0x7f149c0a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005c40 tx=0x7f14ac009500 comp rx=0 tx=0).stop 2024-04-25T00:30:12.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 msgr2=0x7f14b4198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009230 tx=0x7f14a4004270 comp rx=0 tx=0).stop 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 shutdown_connections 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f149c0a2d00 0x7f149c0a51c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14b4106630 0x7f14b41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14b4100fe0 0x7f14b4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 --2- 172.21.15.45:0/1702352957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b4100620 0x7f14b4198e20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.329+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 >> 172.21.15.45:0/1702352957 conn(0x7f14b40776a0 msgr2=0x7f14b41030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.330+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 shutdown_connections 2024-04-25T00:30:12.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.330+0000 7f14b9612640 1 -- 172.21.15.45:0/1702352957 wait complete. 2024-04-25T00:30:12.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:30:12.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 -- 172.21.15.45:0/986025009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 msgr2=0x7f77f4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 --2- 172.21.15.45:0/986025009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f4106a90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009cd0 tx=0x7f77f002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 -- 172.21.15.45:0/986025009 shutdown_connections 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 --2- 172.21.15.45:0/986025009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 --2- 172.21.15.45:0/986025009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 0x7f77f4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 --2- 172.21.15.45:0/986025009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f4106a90 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 -- 172.21.15.45:0/986025009 >> 172.21.15.45:0/986025009 conn(0x7f77f4077740 msgr2=0x7f77f4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.444+0000 7f77fb815640 1 -- 172.21.15.45:0/986025009 shutdown_connections 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 -- 172.21.15.45:0/986025009 wait complete. 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 Processor -- start 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 -- start start 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 0x7f77f410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111b30 con 0x7f77f41066b0 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41119b0 con 0x7f77f4100610 2024-04-25T00:30:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77fb815640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111cb0 con 0x7f77f4100fd0 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77f9db2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77f9db2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57920/0 (socket says 172.21.15.45:57920) 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77f9db2640 1 -- 172.21.15.45:0/2804929312 learned_addr learned my addr 172.21.15.45:0/2804929312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.445+0000 7f77f95b1640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 0x7f77f410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 -- 172.21.15.45:0/2804929312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 msgr2=0x7f77f410e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 -- 172.21.15.45:0/2804929312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 msgr2=0x7f77f410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 0x7f77f410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 -- 172.21.15.45:0/2804929312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f41a64b0 con 0x7f77f41066b0 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00d750 tx=0x7f77ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77ea7fc640 1 -- 172.21.15.45:0/2804929312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec017070 con 0x7f77f41066b0 2024-04-25T00:30:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77ea7fc640 1 -- 172.21.15.45:0/2804929312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec005500 con 0x7f77f41066b0 2024-04-25T00:30:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41a67a0 con 0x7f77f41066b0 2024-04-25T00:30:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77ea7fc640 1 -- 172.21.15.45:0/2804929312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec004d90 con 0x7f77f41066b0 2024-04-25T00:30:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.446+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41a6cf0 con 0x7f77f41066b0 2024-04-25T00:30:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.448+0000 7f77ea7fc640 1 -- 172.21.15.45:0/2804929312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f77ec004f30 con 0x7f77f41066b0 2024-04-25T00:30:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 Processor -- start 2024-04-25T00:30:12.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 -- start start 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3f50 0x561b0c5fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77ea7fc640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 0x7f77e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5fe620 0x561b0c600a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561b0c5586b0 con 0x561b0c5f3630 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561b0c500880 con 0x561b0c5f3f50 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561b0c554670 con 0x561b0c5fe620 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57928/0 (socket says 172.21.15.45:57928) 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 -- 172.21.15.45:0/2078403713 learned_addr learned my addr 172.21.15.45:0/2078403713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 0x7f77e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77ea7fc640 1 -- 172.21.15.45:0/2804929312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f77ec0cf1c0 con 0x7f77f41066b0 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5fe620 0x561b0c600a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f95b1640 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3f50 0x561b0c5fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 -- 172.21.15.45:0/2078403713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5fe620 msgr2=0x561b0c600a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5fe620 0x561b0c600a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 -- 172.21.15.45:0/2078403713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3f50 msgr2=0x561b0c5fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3f50 0x561b0c5fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.449+0000 7f77f8db0640 1 -- 172.21.15.45:0/2078403713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0c3367d0 con 0x561b0c5f3630 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f77e4014c10 tx=0x7f77e40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d73ec583574af7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 0x7f77e00a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77e4011cb0 tx=0x7f77e401c000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77df7fe640 1 -- 172.21.15.45:0/2078403713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4014590 con 0x561b0c5f3630 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77df7fe640 1 -- 172.21.15.45:0/2078403713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4009a00 con 0x561b0c5f3630 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77df7fe640 1 -- 172.21.15.45:0/2078403713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40039f0 con 0x561b0c5f3630 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2078403713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 msgr2=0x561b0c5f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f77e4014c10 tx=0x7f77e40117b0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2078403713 shutdown_connections 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5fe620 0x561b0c600a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3f50 0x561b0c5fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2078403713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3630 0x561b0c5f3a10 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.450+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2078403713 >> 172.21.15.45:0/2078403713 conn(0x561b0c5ef950 msgr2=0x561b0c5efd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2078403713 shutdown_connections 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2078403713 wait complete. 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- start start 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 Processor -- start 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3630 0x561b0c5fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3f50 0x561b0c5fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561b0c6013b0 con 0x561b0c5fe620 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561b0c513f50 con 0x561b0c5f3630 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561b0c601530 con 0x561b0c5f3f50 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77fe5ff740 10 client.0 _ll_register_callbacks cb 0x561b0c54f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77f9db2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77f9db2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57940/0 (socket says 172.21.15.45:57940) 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.451+0000 7f77f9db2640 1 -- 172.21.15.45:0/2026072872 learned_addr learned my addr 172.21.15.45:0/2026072872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f8db0640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3f50 0x561b0c5fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f95b1640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3630 0x561b0c5fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 -- 172.21.15.45:0/2026072872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3f50 msgr2=0x561b0c5fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3f50 0x561b0c5fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 -- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3630 msgr2=0x561b0c5fb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3630 0x561b0c5fb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 -- 172.21.15.45:0/2026072872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0c26c3f0 con 0x561b0c5fe620 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77f9db2640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f77ec097c90 tx=0x7f77ec09b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b613ea54d585ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec020040 con 0x561b0c5fe620 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f77ec0c5cb0 con 0x561b0c5fe620 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b0c565bf0 con 0x561b0c5fe620 2024-04-25T00:30:12.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77ec0c5eb0 con 0x561b0c5fe620 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77fe5ff740 10 client.18039 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b0c22c0f0 con 0x561b0c5fe620 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.452+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561b0c565bf0 con 0x561b0c5fe620 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77ec0cc770 con 0x561b0c5fe620 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 10 client.18039 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 20 client.18039 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c60e8e0 0x561b0c610cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 4 client.18039 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0c55ff40 con 0x561b0c60e8e0 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77f95b1640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c60e8e0 0x561b0c610cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77f95b1640 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c60e8e0 0x561b0c610cf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b2493b6b0555e1 server_cookie=f36a6d15d29be51 in_seq=0 out_seq=0 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.453+0000 7f77dcff9640 10 client.18039 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77dcff9640 1 -- 172.21.15.45:0/2026072872 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b0c55ff40 con 0x561b0c60e8e0 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77dcff9640 10 client.18039 handle_command_reply: tid=0 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 client.18039 shutdown 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 msgr2=0x561b0c5fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f77ec097c90 tx=0x7f77ec09b9c0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 shutdown_connections 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c60e8e0 0x561b0c610cf0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5fe620 0x561b0c5fc0d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77dcff9640 0 client.18039 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3f50 0x561b0c5fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/2026072872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5f3630 0x561b0c5fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 >> 172.21.15.45:0/2026072872 conn(0x561b0c5ef950 msgr2=0x561b0c5f5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 shutdown_connections 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 1 -- 172.21.15.45:0/2026072872 wait complete. 2024-04-25T00:30:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.454+0000 7f77fe5ff740 20 client.18039 trim_cache size 0 max 16384 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 Processor -- start 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 -- start start 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c5fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c60a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b0e70 con 0x561b0c5f3f50 2024-04-25T00:30:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b29b0 con 0x561b0c5fe620 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b1540 con 0x561b0c5f3630 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57954/0 (socket says 172.21.15.45:57954) 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.472+0000 7f77f8db0640 1 -- 172.21.15.45:0/58405221 learned_addr learned my addr 172.21.15.45:0/58405221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 -- 172.21.15.45:0/58405221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 msgr2=0x561b0c5fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c5fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 -- 172.21.15.45:0/58405221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 msgr2=0x561b0c60a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c60a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 -- 172.21.15.45:0/58405221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0c5e1090 con 0x561b0c5f3f50 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77f8db0640 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f77e40096b0 tx=0x7f77e4009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c66b05e81f5bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77deffd640 1 -- 172.21.15.45:0/58405221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4035070 con 0x561b0c5f3f50 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77deffd640 1 -- 172.21.15.45:0/58405221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4005040 con 0x561b0c5f3f50 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77deffd640 1 -- 172.21.15.45:0/58405221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004a60 con 0x561b0c5f3f50 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 -- 172.21.15.45:0/58405221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 msgr2=0x561b0c5efd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f77e40096b0 tx=0x7f77e4009c40 comp rx=0 tx=0).stop 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 -- 172.21.15.45:0/58405221 shutdown_connections 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c60a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c5efd30 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/58405221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c5fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.473+0000 7f77fe5ff740 1 -- 172.21.15.45:0/58405221 >> 172.21.15.45:0/58405221 conn(0x561b0c5ef950 msgr2=0x561b0c5f4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- 172.21.15.45:0/58405221 shutdown_connections 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- 172.21.15.45:0/58405221 wait complete. 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- start start 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 Processor -- start 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c606850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c602200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b22e0 con 0x561b0c5f3f50 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b1c10 con 0x561b0c5fe620 2024-04-25T00:30:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41aee70 con 0x561b0c5f3630 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77fe5ff740 10 client.0 _ll_register_callbacks cb 0x7f77ec09ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77f8db0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57960/0 (socket says 172.21.15.45:57960) 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.474+0000 7f77f8db0640 1 -- 172.21.15.45:0/66849499 learned_addr learned my addr 172.21.15.45:0/66849499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 -- 172.21.15.45:0/66849499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 msgr2=0x561b0c606850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f95b1640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c606850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c606850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 -- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 msgr2=0x561b0c602200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f9db2640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c602200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c602200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 -- 172.21.15.45:0/66849499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0c602740 con 0x561b0c5f3f50 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77f8db0640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f77e4033040 tx=0x7f77e4005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c95c5adc7754866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4035030 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f77e400ce00 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e4005040 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77e4046050 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77fe5ff740 10 client.18045 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b0c674b60 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4005040 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.475+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f77e401d070 con 0x561b0c5f3f50 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 10 client.18045 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 20 client.18045 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c6750e0 0x561b0c677500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 4 client.18045 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b0c4dd6f0 con 0x561b0c6750e0 2024-04-25T00:30:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77f95b1640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c6750e0 0x561b0c677500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77f95b1640 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c6750e0 0x561b0c677500 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b02bbed951701c server_cookie=290551cc1465b658 in_seq=0 out_seq=0 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77dcff9640 10 client.18045 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77dcff9640 1 -- 172.21.15.45:0/66849499 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b0c4dd6f0 con 0x561b0c6750e0 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77dcff9640 10 client.18045 handle_command_reply: tid=0 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 client.18045 shutdown 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 msgr2=0x561b0c601cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f77e4033040 tx=0x7f77e4005a70 comp rx=0 tx=0).stop 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 shutdown_connections 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b0c5fe620 0x561b0c602200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x561b0c6750e0 0x561b0c677500 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561b0c5f3f50 0x561b0c601cc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.476+0000 7f77fe5ff740 1 --2- 172.21.15.45:0/66849499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561b0c5f3630 0x561b0c606850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77dcff9640 0 client.18045 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 >> 172.21.15.45:0/66849499 conn(0x561b0c5ef950 msgr2=0x561b0c5fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 shutdown_connections 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fe5ff740 1 -- 172.21.15.45:0/66849499 wait complete. 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fe5ff740 20 client.18045 trim_cache size 0 max 16384 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout: "2ff2f25e-01de-42d4-bd08-2f8777a5978e": { 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:30:12.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "2ff2f25e-01de-42d4-bd08-2f8777a5978e", 2024-04-25T00:30:12.485 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:30:12.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:30:12.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:30:12.485 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 msgr2=0x7f77e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 0x7f77e00a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f77e4011cb0 tx=0x7f77e401c000 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 msgr2=0x7f77f410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00d750 tx=0x7f77ec00dc10 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 shutdown_connections 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f77e00a2d00 0x7f77e00a51c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77f41066b0 0x7f77f410e9f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4100fd0 0x7f77f410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 --2- 172.21.15.45:0/2804929312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77f4100610 0x7f77f410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 >> 172.21.15.45:0/2804929312 conn(0x7f77f4077740 msgr2=0x7f77f40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 shutdown_connections 2024-04-25T00:30:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:12.477+0000 7f77fb815640 1 -- 172.21.15.45:0/2804929312 wait complete. 2024-04-25T00:30:12.488 INFO:tasks.cephfs.filesystem:scrub status for tag:2ff2f25e-01de-42d4-bd08-2f8777a5978e - {'path': '/', 'tag': '2ff2f25e-01de-42d4-bd08-2f8777a5978e', 'options': 'recursive,force'} 2024-04-25T00:30:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:30:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:30:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:30:13.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:12 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:30:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:12 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: pgmap v253: 129 pgs: 129 active+clean; 2.1 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:12 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:30:15.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:14 smithi122 ceph-mon[32959]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:30:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:14 smithi071 ceph-mon[32667]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:30:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:14 smithi045 ceph-mon[29194]: pgmap v255: 129 pgs: 129 active+clean; 2.1 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T00:30:17.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:16 smithi122 ceph-mon[32959]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T00:30:17.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:16 smithi071 ceph-mon[32667]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T00:30:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:16 smithi045 ceph-mon[29194]: pgmap v256: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-25T00:30:19.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:18 smithi122 ceph-mon[32959]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-25T00:30:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:18 smithi071 ceph-mon[32667]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-25T00:30:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:18 smithi045 ceph-mon[29194]: pgmap v257: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-25T00:30:21.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:20 smithi122 ceph-mon[32959]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:20 smithi071 ceph-mon[32667]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:20 smithi045 ceph-mon[29194]: pgmap v258: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:23.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:22 smithi122 ceph-mon[32959]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:22 smithi071 ceph-mon[32667]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:22 smithi045 ceph-mon[29194]: pgmap v259: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:30:25.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:24 smithi122 ceph-mon[32959]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:30:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:24 smithi071 ceph-mon[32667]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:30:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:24 smithi045 ceph-mon[29194]: pgmap v260: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:30:27.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:26 smithi122 ceph-mon[32959]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:27.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:26 smithi071 ceph-mon[32667]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:26 smithi045 ceph-mon[29194]: pgmap v261: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:29.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:28 smithi122 ceph-mon[32959]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:28 smithi071 ceph-mon[32667]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:28 smithi045 ceph-mon[29194]: pgmap v262: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:31.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:31 smithi122 ceph-mon[32959]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:31 smithi071 ceph-mon[32667]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:31 smithi045 ceph-mon[29194]: pgmap v263: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:33.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:33 smithi122 ceph-mon[32959]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:33.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:33 smithi071 ceph-mon[32667]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:33 smithi045 ceph-mon[29194]: pgmap v264: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:35.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:35 smithi122 ceph-mon[32959]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:35.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:35 smithi071 ceph-mon[32667]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:35 smithi045 ceph-mon[29194]: pgmap v265: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:37.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:37 smithi122 ceph-mon[32959]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:37.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:37 smithi071 ceph-mon[32667]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:37.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:37 smithi045 ceph-mon[29194]: pgmap v266: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:39.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:39 smithi122 ceph-mon[32959]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:39.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:39 smithi071 ceph-mon[32667]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:39 smithi045 ceph-mon[29194]: pgmap v267: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:41.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:41 smithi122 ceph-mon[32959]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:41.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:41 smithi071 ceph-mon[32667]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:41 smithi045 ceph-mon[29194]: pgmap v268: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:30:42.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:30:42.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:30:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 -- 172.21.15.45:0/146492379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c069dc0 msgr2=0x7f4a5c06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 --2- 172.21.15.45:0/146492379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c06a240 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009930 tx=0x7f4a5002a420 comp rx=0 tx=0).stop 2024-04-25T00:30:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 -- 172.21.15.45:0/146492379 shutdown_connections 2024-04-25T00:30:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 --2- 172.21.15.45:0/146492379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c06a780 0x7f4a5c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 --2- 172.21.15.45:0/146492379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c06a240 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 --2- 172.21.15.45:0/146492379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 -- 172.21.15.45:0/146492379 >> 172.21.15.45:0/146492379 conn(0x7f4a5c0776a0 msgr2=0x7f4a5c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 -- 172.21.15.45:0/146492379 shutdown_connections 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.607+0000 7f4a627dc640 1 -- 172.21.15.45:0/146492379 wait complete. 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 Processor -- start 2024-04-25T00:30:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 -- start start 2024-04-25T00:30:42.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c111b40 con 0x7f4a5c06a780 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1119c0 con 0x7f4a5c069dc0 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a627dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c111cc0 con 0x7f4a5c107730 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33744/0 (socket says 172.21.15.45:33744) 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a5b7fe640 1 -- 172.21.15.45:0/881633807 learned_addr learned my addr 172.21.15.45:0/881633807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.608+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 -- 172.21.15.45:0/881633807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 msgr2=0x7f4a5c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a60d79640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 -- 172.21.15.45:0/881633807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 msgr2=0x7f4a5c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 -- 172.21.15.45:0/881633807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c19cff0 con 0x7f4a5c06a780 2024-04-25T00:30:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a60d79640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009230 tx=0x7f4a50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a597fa640 1 -- 172.21.15.45:0/881633807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5003f030 con 0x7f4a5c06a780 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a597fa640 1 -- 172.21.15.45:0/881633807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50003ad0 con 0x7f4a5c06a780 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c19d280 con 0x7f4a5c06a780 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a597fa640 1 -- 172.21.15.45:0/881633807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50035070 con 0x7f4a5c06a780 2024-04-25T00:30:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.609+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a4b70 con 0x7f4a5c06a780 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.611+0000 7f4a597fa640 1 -- 172.21.15.45:0/881633807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a50042030 con 0x7f4a5c06a780 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 Processor -- start 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 -- start start 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a597fa640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 0x7f4a440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd8234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd823a30 0x5558cd825e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5558cd7796b0 con 0x5558cd814530 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5558cd721880 con 0x5558cd823a30 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a655c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558cd775670 con 0x5558cd814e50 2024-04-25T00:30:42.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 0x7f4a440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33758/0 (socket says 172.21.15.45:33758) 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 -- 172.21.15.45:0/1742700978 learned_addr learned my addr 172.21.15.45:0/1742700978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a597fa640 1 -- 172.21.15.45:0/881633807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f4a500efe20 con 0x7f4a5c06a780 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd823a30 0x5558cd825e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd8234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 -- 172.21.15.45:0/1742700978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 msgr2=0x5558cd8234f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd8234f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 -- 172.21.15.45:0/1742700978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd823a30 msgr2=0x5558cd825e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd823a30 0x5558cd825e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.612+0000 7f4a60d79640 1 -- 172.21.15.45:0/1742700978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558cd5577d0 con 0x5558cd814530 2024-04-25T00:30:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 0x7f4a440a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c00b5d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a60d79640 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4a54008f30 tx=0x7f4a540087d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1c2206b98bdbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a427fc640 1 -- 172.21.15.45:0/1742700978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400fa60 con 0x5558cd814530 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a427fc640 1 -- 172.21.15.45:0/1742700978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5401a070 con 0x5558cd814530 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a427fc640 1 -- 172.21.15.45:0/1742700978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54012cf0 con 0x5558cd814530 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a655c8740 1 -- 172.21.15.45:0/1742700978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 msgr2=0x5558cd814910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.613+0000 7f4a655c8740 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4a54008f30 tx=0x7f4a540087d0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 -- 172.21.15.45:0/1742700978 shutdown_connections 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd823a30 0x5558cd825e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd8234f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- 172.21.15.45:0/1742700978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd814910 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 -- 172.21.15.45:0/1742700978 >> 172.21.15.45:0/1742700978 conn(0x5558cd810950 msgr2=0x5558cd810d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 -- 172.21.15.45:0/1742700978 shutdown_connections 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 -- 172.21.15.45:0/1742700978 wait complete. 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 -- start start 2024-04-25T00:30:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 Processor -- start 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd820e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.614+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd823a30 0x5558cd821350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a655c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5558cd826790 con 0x5558cd814530 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a655c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5558cd734f50 con 0x5558cd814e50 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a655c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558cd826910 con 0x5558cd823a30 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a655c8740 10 client.0 _ll_register_callbacks cb 0x5558cd770000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33762/0 (socket says 172.21.15.45:33762) 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 -- 172.21.15.45:0/3135495503 learned_addr learned my addr 172.21.15.45:0/3135495503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a60d79640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd823a30 0x5558cd821350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd820e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 -- 172.21.15.45:0/3135495503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd823a30 msgr2=0x5558cd821350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd823a30 0x5558cd821350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 -- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 msgr2=0x5558cd820e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd820e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 -- 172.21.15.45:0/3135495503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558cd48d3f0 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c011490 tx=0x7f4a4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9233588c89603d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c022070 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f4a4c01e860 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5401a070 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.615+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c02e050 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 10 client.18057 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558cd44d0f0 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5401a070 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a4c010c10 con 0x5558cd814530 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 10 client.18057 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 20 client.18057 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd833bd0 0x5558cd836020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 4 client.18057 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558cd786bf0 con 0x5558cd833bd0 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.616+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd833bd0 0x5558cd836020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.617+0000 7f4a3b7fe640 10 client.18057 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.617+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd833bd0 0x5558cd836020 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40086ff8d3f2e4e server_cookie=1101ed1de6e537c5 in_seq=0 out_seq=0 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.617+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3135495503 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558cd786bf0 con 0x5558cd833bd0 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.617+0000 7f4a3b7fe640 10 client.18057 handle_command_reply: tid=0 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.617+0000 7f4a655c8740 1 client.18057 shutdown 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 msgr2=0x5558cd8208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c011490 tx=0x7f4a4c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 shutdown_connections 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd823a30 0x5558cd821350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd820e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd833bd0 0x5558cd836020 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3135495503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd814530 0x5558cd8208d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a3b7fe640 0 client.18057 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 >> 172.21.15.45:0/3135495503 conn(0x5558cd810950 msgr2=0x5558cd816780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 shutdown_connections 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 1 -- 172.21.15.45:0/3135495503 wait complete. 2024-04-25T00:30:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.618+0000 7f4a655c8740 20 client.18057 trim_cache size 0 max 16384 2024-04-25T00:30:42.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.635+0000 7f4a655c8740 1 Processor -- start 2024-04-25T00:30:42.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 -- start start 2024-04-25T00:30:42.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814530 0x5558cd82f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd8201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ac9e0 con 0x5558cd823a30 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ae520 con 0x5558cd814e50 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a655c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ad0b0 con 0x5558cd814530 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33770/0 (socket says 172.21.15.45:33770) 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 -- 172.21.15.45:0/35104360 learned_addr learned my addr 172.21.15.45:0/35104360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814530 0x5558cd82f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd8201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 -- 172.21.15.45:0/35104360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814530 msgr2=0x5558cd82f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814530 0x5558cd82f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 -- 172.21.15.45:0/35104360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 msgr2=0x5558cd8201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd8201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 -- 172.21.15.45:0/35104360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558cd810780 con 0x5558cd823a30 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd8201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a60d79640 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400f730 tx=0x7f4a54008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd17660c22340b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.636+0000 7f4a41ffb640 1 -- 172.21.15.45:0/35104360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400f860 con 0x5558cd823a30 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a41ffb640 1 -- 172.21.15.45:0/35104360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5401a050 con 0x5558cd823a30 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a41ffb640 1 -- 172.21.15.45:0/35104360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011440 con 0x5558cd823a30 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- 172.21.15.45:0/35104360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 msgr2=0x5558cd820fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400f730 tx=0x7f4a54008400 comp rx=0 tx=0).stop 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- 172.21.15.45:0/35104360 shutdown_connections 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd820fa0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814e50 0x5558cd8201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 --2- 172.21.15.45:0/35104360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814530 0x5558cd82f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- 172.21.15.45:0/35104360 >> 172.21.15.45:0/35104360 conn(0x5558cd810950 msgr2=0x5558cd820890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- 172.21.15.45:0/35104360 shutdown_connections 2024-04-25T00:30:42.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- 172.21.15.45:0/35104360 wait complete. 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.637+0000 7f4a655c8740 1 -- start start 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 Processor -- start 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 0x5558cd82bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd81d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ade50 con 0x5558cd823a30 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ad780 con 0x5558cd814530 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1aa9e0 con 0x5558cd814e50 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a655c8740 10 client.0 _ll_register_callbacks cb 0x7f4a4c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33786/0 (socket says 172.21.15.45:33786) 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 -- 172.21.15.45:0/3372944952 learned_addr learned my addr 172.21.15.45:0/3372944952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a5b7fe640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd81d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 0x5558cd82bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 -- 172.21.15.45:0/3372944952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 msgr2=0x5558cd81d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd81d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 -- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 msgr2=0x5558cd82bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 0x5558cd82bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a60d79640 1 -- 172.21.15.45:0/3372944952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558cd81b6c0 con 0x5558cd823a30 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.638+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 0x5558cd82bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a60d79640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005f60 tx=0x7f4a54012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fa7b7f3d9ee64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54004d40 con 0x5558cd823a30 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5401a050 con 0x5558cd823a30 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f4a5400f760 con 0x5558cd823a30 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a5403b050 con 0x5558cd823a30 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 10 client.18063 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:42.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558cd899d00 con 0x5558cd823a30 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5401a050 con 0x5558cd823a30 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a54026070 con 0x5558cd823a30 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 10 client.18063 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 20 client.18063 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd89a230 0x5558cd89c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 4 client.18063 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.639+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558cd780f40 con 0x5558cd89a230 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd89a230 0x5558cd89c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a3b7fe640 10 client.18063 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:42.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a5bfff640 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd89a230 0x5558cd89c650 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a367cb12d8eddfa6 server_cookie=7399d666a3302bf3 in_seq=0 out_seq=0 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a3b7fe640 1 -- 172.21.15.45:0/3372944952 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5558cd780f40 con 0x5558cd89a230 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a3b7fe640 10 client.18063 handle_command_reply: tid=0 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.640+0000 7f4a655c8740 1 client.18063 shutdown 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 msgr2=0x5558cd81d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005f60 tx=0x7f4a54012920 comp rx=0 tx=0).stop 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 shutdown_connections 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5558cd89a230 0x5558cd89c650 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a3b7fe640 0 client.18063 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5558cd823a30 0x5558cd81d660 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558cd814e50 0x5558cd81d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 --2- 172.21.15.45:0/3372944952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558cd814530 0x5558cd82bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 >> 172.21.15.45:0/3372944952 conn(0x5558cd810950 msgr2=0x5558cd821100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 shutdown_connections 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 1 -- 172.21.15.45:0/3372944952 wait complete. 2024-04-25T00:30:42.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a655c8740 20 client.18063 trim_cache size 0 max 16384 2024-04-25T00:30:42.649 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:30:42.649 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:30:42.649 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 msgr2=0x7f4a440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 0x7f4a440a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c00b5d0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 msgr2=0x7f4a5c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.641+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009230 tx=0x7f4a50004270 comp rx=0 tx=0).stop 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 shutdown_connections 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a440a2d00 0x7f4a440a51c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c107730 0x7f4a5c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a5c06a780 0x7f4a5c198540 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 --2- 172.21.15.45:0/881633807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a5c069dc0 0x7f4a5c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 >> 172.21.15.45:0/881633807 conn(0x7f4a5c0776a0 msgr2=0x7f4a5c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 shutdown_connections 2024-04-25T00:30:42.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.642+0000 7f4a627dc640 1 -- 172.21.15.45:0/881633807 wait complete. 2024-04-25T00:30:42.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:30:42.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 -- 172.21.15.45:0/3306403904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01029d0 msgr2=0x7ff7e0102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/3306403904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01029d0 0x7ff7e0102db0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009900 tx=0x7ff7dc02a510 comp rx=0 tx=0).stop 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 -- 172.21.15.45:0/3306403904 shutdown_connections 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/3306403904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e010bd10 0x7ff7e010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/3306403904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e01032f0 0x7ff7e010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/3306403904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01029d0 0x7ff7e0102db0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 -- 172.21.15.45:0/3306403904 >> 172.21.15.45:0/3306403904 conn(0x7ff7e0077740 msgr2=0x7ff7e0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 -- 172.21.15.45:0/3306403904 shutdown_connections 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.755+0000 7ff7e81ee640 1 -- 172.21.15.45:0/3306403904 wait complete. 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 Processor -- start 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 -- start start 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e01029d0 0x7ff7e01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 0x7ff7e019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0111b60 con 0x7ff7e01032f0 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01119e0 con 0x7ff7e010bd10 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e81ee640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0111ce0 con 0x7ff7e01029d0 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e5789640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e5789640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33808/0 (socket says 172.21.15.45:33808) 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e5789640 1 -- 172.21.15.45:0/1358890060 learned_addr learned my addr 172.21.15.45:0/1358890060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:42.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.756+0000 7ff7e5f8a640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e01029d0 0x7ff7e01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e678b640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 0x7ff7e019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 -- 172.21.15.45:0/1358890060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e01029d0 msgr2=0x7ff7e01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e01029d0 0x7ff7e01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 -- 172.21.15.45:0/1358890060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 msgr2=0x7ff7e019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 0x7ff7e019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 -- 172.21.15.45:0/1358890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e019d060 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e678b640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 0x7ff7e019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e5789640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c890 tx=0x7ff7d000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0004220 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d00043c0 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e019d350 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00049e0 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.757+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e01a4be0 con 0x7ff7e01032f0 2024-04-25T00:30:42.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.758+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e0068560 con 0x7ff7e01032f0 2024-04-25T00:30:42.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.759+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff7d00079b0 con 0x7ff7e01032f0 2024-04-25T00:30:42.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.760+0000 7ff7d6ffd640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 0x7ff7cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:42.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.760+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7ff7d00cf870 con 0x7ff7e01032f0 2024-04-25T00:30:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.760+0000 7ff7e5f8a640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 0x7ff7cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.760+0000 7ff7e5f8a640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 0x7ff7cc0a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000c00 tx=0x7ff7dc005b50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:42.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.764+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7d0014030 con 0x7ff7e01032f0 2024-04-25T00:30:42.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.987+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7e01993c0 con 0x7ff7e01032f0 2024-04-25T00:30:42.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.987+0000 7ff7d6ffd640 1 -- 172.21.15.45:0/1358890060 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7ff7e01993c0 con 0x7ff7e01032f0 2024-04-25T00:30:42.988 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:30:42.988 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:30:42.989 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:30:42.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.989+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 msgr2=0x7ff7cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.989+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 0x7ff7cc0a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000c00 tx=0x7ff7dc005b50 comp rx=0 tx=0).stop 2024-04-25T00:30:42.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.989+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 msgr2=0x7ff7e0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.989+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c890 tx=0x7ff7d000cd50 comp rx=0 tx=0).stop 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 shutdown_connections 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff7cc0a2d00 0x7ff7cc0a51c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7e010bd10 0x7ff7e019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7e01032f0 0x7ff7e0198600 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 --2- 172.21.15.45:0/1358890060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7e01029d0 0x7ff7e01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 >> 172.21.15.45:0/1358890060 conn(0x7ff7e0077740 msgr2=0x7ff7e00fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 shutdown_connections 2024-04-25T00:30:42.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:42.990+0000 7ff7e81ee640 1 -- 172.21.15.45:0/1358890060 wait complete. 2024-04-25T00:30:43.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:30:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 -- 172.21.15.45:0/1453446916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c10bd00 msgr2=0x7f201c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 --2- 172.21.15.45:0/1453446916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c10bd00 0x7f201c10e1a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2014009ca0 tx=0x7f201402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 -- 172.21.15.45:0/1453446916 shutdown_connections 2024-04-25T00:30:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 --2- 172.21.15.45:0/1453446916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c10bd00 0x7f201c10e1a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 --2- 172.21.15.45:0/1453446916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 --2- 172.21.15.45:0/1453446916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c1029c0 0x7f201c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 -- 172.21.15.45:0/1453446916 >> 172.21.15.45:0/1453446916 conn(0x7f201c077740 msgr2=0x7f201c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 -- 172.21.15.45:0/1453446916 shutdown_connections 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.105+0000 7f2022a38640 1 -- 172.21.15.45:0/1453446916 wait complete. 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 Processor -- start 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 -- start start 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c10bd00 0x7f201c101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f201c111b00 con 0x7f201c1029c0 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f201c111980 con 0x7f201c10bd00 2024-04-25T00:30:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2022a38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f201c111c80 con 0x7f201c1032e0 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f201bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f201bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33818/0 (socket says 172.21.15.45:33818) 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f201bfff640 1 -- 172.21.15.45:0/3605803042 learned_addr learned my addr 172.21.15.45:0/3605803042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f201b7fe640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f201bfff640 1 -- 172.21.15.45:0/3605803042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 msgr2=0x7f201c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201bfff640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.106+0000 7f2020fd5640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c10bd00 0x7f201c101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201bfff640 1 -- 172.21.15.45:0/3605803042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c10bd00 msgr2=0x7f201c101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201bfff640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c10bd00 0x7f201c101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201bfff640 1 -- 172.21.15.45:0/3605803042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c1a6480 con 0x7f201c1029c0 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201b7fe640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f201bfff640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d950 tx=0x7f200c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f20197fa640 1 -- 172.21.15.45:0/3605803042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c017070 con 0x7f201c1029c0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f20197fa640 1 -- 172.21.15.45:0/3605803042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c009c30 con 0x7f201c1029c0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c1a6770 con 0x7f201c1029c0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f20197fa640 1 -- 172.21.15.45:0/3605803042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00f6b0 con 0x7f201c1029c0 2024-04-25T00:30:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.107+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c1a6cc0 con 0x7f201c1029c0 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f20197fa640 1 -- 172.21.15.45:0/3605803042 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f200c0109d0 con 0x7f201c1029c0 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 Processor -- start 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 -- start start 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c87960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.109+0000 7f2025824740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582b6bdf670 con 0x5582b6c7a580 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2025824740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5582b6b87970 con 0x5582b6c7aea0 2024-04-25T00:30:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2025824740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582b6bdb630 con 0x5582b6c85570 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33828/0 (socket says 172.21.15.45:33828) 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 -- 172.21.15.45:0/89387605 learned_addr learned my addr 172.21.15.45:0/89387605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f201bfff640 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f201b7fe640 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c87960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f20197fa640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 0x7f20040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 -- 172.21.15.45:0/89387605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 msgr2=0x5582b6c87960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c87960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 -- 172.21.15.45:0/89387605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 msgr2=0x5582b6c85030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c85030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 -- 172.21.15.45:0/89387605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b6bab400 con 0x5582b6c7a580 2024-04-25T00:30:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f20197fa640 1 -- 172.21.15.45:0/3605803042 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f200c0cf4a0 con 0x7f201c1029c0 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 0x7f20040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f2020fd5640 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f201402afd0 tx=0x7f2014005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcec8851a6242caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.110+0000 7f20027fc640 1 -- 172.21.15.45:0/89387605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014033050 con 0x5582b6c7a580 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f20027fc640 1 -- 172.21.15.45:0/89387605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2014003e00 con 0x5582b6c7a580 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f20027fc640 1 -- 172.21.15.45:0/89387605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20140042a0 con 0x5582b6c7a580 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 -- 172.21.15.45:0/89387605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 msgr2=0x5582b6c7a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2020fd5640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 0x7f20040a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2014049000 tx=0x7f201404a000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f201402afd0 tx=0x7f2014005ef0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 -- 172.21.15.45:0/89387605 shutdown_connections 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c87960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c85030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 --2- 172.21.15.45:0/89387605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c7a960 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 -- 172.21.15.45:0/89387605 >> 172.21.15.45:0/89387605 conn(0x5582b6c768a0 msgr2=0x5582b6c76c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 -- 172.21.15.45:0/89387605 shutdown_connections 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.111+0000 7f2025824740 1 -- 172.21.15.45:0/89387605 wait complete. 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 -- start start 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 Processor -- start 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7a580 0x5582b6c84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582b6c88360 con 0x5582b6c7aea0 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5582b6b9b050 con 0x5582b6c7a580 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5582b6c884e0 con 0x5582b6c85570 2024-04-25T00:30:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2025824740 10 client.0 _ll_register_callbacks cb 0x5582b68f2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33844/0 (socket says 172.21.15.45:33844) 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 -- 172.21.15.45:0/1479992338 learned_addr learned my addr 172.21.15.45:0/1479992338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201bfff640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7a580 0x5582b6c84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f2020fd5640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 -- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 msgr2=0x5582b6c827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.112+0000 7f201b7fe640 1 -- 172.21.15.45:0/1479992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7a580 msgr2=0x5582b6c84f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f201b7fe640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7a580 0x5582b6c84f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f201b7fe640 1 -- 172.21.15.45:0/1479992338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b6b6cdc0 con 0x5582b6c7aea0 2024-04-25T00:30:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2020fd5640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f201b7fe640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2010013710 tx=0x7f2010013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3ea6d5577d44fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010011dd0 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f2010004020 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2014003e00 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2010004220 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 10 client.18081 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582b68f33f0 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2014003e00 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f201001a070 con 0x5582b6c7aea0 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 10 client.18081 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 10 client.18081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:30:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.113+0000 7f2025824740 20 client.18081 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6c95700 0x5582b6c97b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f2025824740 4 client.18081 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b6becbb0 con 0x5582b6c95700 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f201bfff640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6c95700 0x5582b6c97b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f201bfff640 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6c95700 0x5582b6c97b20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16675f12df888d20 server_cookie=2b72e1005858fb83 in_seq=0 out_seq=0 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.114+0000 7f1ffb7fe640 10 client.18081 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/1479992338 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582b6becbb0 con 0x5582b6c95700 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f1ffb7fe640 10 client.18081 handle_command_reply: tid=0 2024-04-25T00:30:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 client.18081 shutdown 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 msgr2=0x5582b6c82200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2010013710 tx=0x7f2010013bd0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 shutdown_connections 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6c95700 0x5582b6c97b20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c82200 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 --2- 172.21.15.45:0/1479992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7a580 0x5582b6c84f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f1ffb7fe640 0 client.18081 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 >> 172.21.15.45:0/1479992338 conn(0x5582b6c768a0 msgr2=0x5582b6c75b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 shutdown_connections 2024-04-25T00:30:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 1 -- 172.21.15.45:0/1479992338 wait complete. 2024-04-25T00:30:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.115+0000 7f2025824740 20 client.18081 trim_cache size 0 max 16384 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.128+0000 7f2025824740 1 Processor -- start 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.128+0000 7f2025824740 1 -- start start 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c84a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1b0e50 con 0x5582b6c7a580 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1b2990 con 0x5582b6c7aea0 2024-04-25T00:30:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2025824740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1b1520 con 0x5582b6c85570 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33850/0 (socket says 172.21.15.45:33850) 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 -- 172.21.15.45:0/1765652491 learned_addr learned my addr 172.21.15.45:0/1765652491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201b7fe640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c84a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2020fd5640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 -- 172.21.15.45:0/1765652491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 msgr2=0x5582b6c9d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c9d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 -- 172.21.15.45:0/1765652491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 msgr2=0x5582b6c84a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c84a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 -- 172.21.15.45:0/1765652491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b69bd7d0 con 0x5582b6c7a580 2024-04-25T00:30:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201b7fe640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c84a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2020fd5640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c9d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f201bfff640 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f200c097a60 tx=0x7f200c0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab02f2ebf14c8be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2001ffb640 1 -- 172.21.15.45:0/1765652491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c020040 con 0x5582b6c7a580 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2001ffb640 1 -- 172.21.15.45:0/1765652491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c009de0 con 0x5582b6c7a580 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.129+0000 7f2001ffb640 1 -- 172.21.15.45:0/1765652491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c097040 con 0x5582b6c7a580 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- 172.21.15.45:0/1765652491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 msgr2=0x5582b6c91220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f200c097a60 tx=0x7f200c0d1040 comp rx=0 tx=0).stop 2024-04-25T00:30:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- 172.21.15.45:0/1765652491 shutdown_connections 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c85570 0x5582b6c9d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c7aea0 0x5582b6c84a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 --2- 172.21.15.45:0/1765652491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7a580 0x5582b6c91220 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- 172.21.15.45:0/1765652491 >> 172.21.15.45:0/1765652491 conn(0x5582b6c768a0 msgr2=0x5582b6c7d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- 172.21.15.45:0/1765652491 shutdown_connections 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- 172.21.15.45:0/1765652491 wait complete. 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.130+0000 7f2025824740 1 -- start start 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 Processor -- start 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c7a580 0x5582b6c891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 0x5582b6c89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1b22c0 con 0x5582b6c7aea0 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1b1bf0 con 0x5582b6c85570 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1aee50 con 0x5582b6c7a580 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2025824740 10 client.0 _ll_register_callbacks cb 0x7f2010009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33854/0 (socket says 172.21.15.45:33854) 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 -- 172.21.15.45:0/2222343773 learned_addr learned my addr 172.21.15.45:0/2222343773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201bfff640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c7a580 0x5582b6c891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2020fd5640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 0x5582b6c89c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 -- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c7a580 msgr2=0x5582b6c891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c7a580 0x5582b6c891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 -- 172.21.15.45:0/2222343773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 msgr2=0x5582b6c89c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 0x5582b6c89c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 -- 172.21.15.45:0/2222343773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b6cfacb0 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f2020fd5640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 0x5582b6c89c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f201b7fe640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f201001b8c0 tx=0x7f2010004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c2da9fb8341aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.131+0000 7f1ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201002d070 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f2010011bd0 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c009de0 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2010005640 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 10 client.18087 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582b6cfbf20 con 0x5582b6c7aea0 2024-04-25T00:30:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c009de0 con 0x5582b6c7aea0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f201001a040 con 0x5582b6c7aea0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 10 client.18087 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 10 client.18087 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 20 client.18087 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6cfc6c0 0x5582b6cfeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 4 client.18087 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582b6be6f00 con 0x5582b6cfc6c0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.132+0000 7f201bfff640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6cfc6c0 0x5582b6cfeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f201bfff640 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6cfc6c0 0x5582b6cfeae0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9496bfef17cd2c0b server_cookie=8c74bdebd6f105d6 in_seq=0 out_seq=0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f1ffb7fe640 10 client.18087 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f1ffb7fe640 1 -- 172.21.15.45:0/2222343773 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582b6be6f00 con 0x5582b6cfc6c0 2024-04-25T00:30:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f1ffb7fe640 10 client.18087 handle_command_reply: tid=0 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 client.18087 shutdown 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 msgr2=0x5582b6c896e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f201001b8c0 tx=0x7f2010004140 comp rx=0 tx=0).stop 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 shutdown_connections 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5582b6cfc6c0 0x5582b6cfeae0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582b6c85570 0x5582b6c89c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582b6c7aea0 0x5582b6c896e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f1ffb7fe640 0 client.18087 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 --2- 172.21.15.45:0/2222343773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5582b6c7a580 0x5582b6c891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 >> 172.21.15.45:0/2222343773 conn(0x5582b6c768a0 msgr2=0x5582b6c90210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 shutdown_connections 2024-04-25T00:30:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 1 -- 172.21.15.45:0/2222343773 wait complete. 2024-04-25T00:30:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.133+0000 7f2025824740 20 client.18087 trim_cache size 0 max 16384 2024-04-25T00:30:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 msgr2=0x7f20040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 0x7f20040a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2014049000 tx=0x7f201404a000 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 msgr2=0x7f201c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d950 tx=0x7f200c00de10 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 shutdown_connections 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20040a2d30 0x7f20040a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f201c10bd00 0x7f201c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201c1032e0 0x7f201c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 --2- 172.21.15.45:0/3605803042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f201c1029c0 0x7f201c100610 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 >> 172.21.15.45:0/3605803042 conn(0x7f201c077740 msgr2=0x7f201c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 shutdown_connections 2024-04-25T00:30:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.134+0000 7f2022a38640 1 -- 172.21.15.45:0/3605803042 wait complete. 2024-04-25T00:30:43.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:30:43.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 -- 172.21.15.45:0/2662828964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1764106aa0 msgr2=0x7f176410e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 --2- 172.21.15.45:0/2662828964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1764106aa0 0x7f176410e6b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f175c009ca0 tx=0x7f175c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 -- 172.21.15.45:0/2662828964 shutdown_connections 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 --2- 172.21.15.45:0/2662828964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1764106aa0 0x7f176410e6b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 --2- 172.21.15.45:0/2662828964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764100c90 0x7f17641063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 --2- 172.21.15.45:0/2662828964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 -- 172.21.15.45:0/2662828964 >> 172.21.15.45:0/2662828964 conn(0x7f17640758d0 msgr2=0x7f1764075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 -- 172.21.15.45:0/2662828964 shutdown_connections 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.253+0000 7f1769858640 1 -- 172.21.15.45:0/2662828964 wait complete. 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 Processor -- start 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 -- start start 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764106aa0 0x7f17641985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1764111b90 con 0x7f176419c970 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1764111a10 con 0x7f1764106aa0 2024-04-25T00:30:43.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f1769858640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1764111d10 con 0x7f17641002e0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.254+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33868/0 (socket says 172.21.15.45:33868) 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f17637fe640 1 -- 172.21.15.45:0/9125263 learned_addr learned my addr 172.21.15.45:0/9125263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f17627fc640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764106aa0 0x7f17641985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 -- 172.21.15.45:0/9125263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764106aa0 msgr2=0x7f17641985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764106aa0 0x7f17641985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 -- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 msgr2=0x7f176419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 -- 172.21.15.45:0/9125263 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176419d460 con 0x7f17641002e0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f17637fe640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1762ffd640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641980b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f175400e9f0 tx=0x7f175400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1768856640 1 -- 172.21.15.45:0/9125263 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175400ccc0 con 0x7f17641002e0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176419d750 con 0x7f17641002e0 2024-04-25T00:30:43.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17641a4f40 con 0x7f17641002e0 2024-04-25T00:30:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1768856640 1 -- 172.21.15.45:0/9125263 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1754005020 con 0x7f17641002e0 2024-04-25T00:30:43.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.255+0000 7f1768856640 1 -- 172.21.15.45:0/9125263 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17540049e0 con 0x7f17641002e0 2024-04-25T00:30:43.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 Processor -- start 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 -- start start 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fd580 0x5586732fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 0x558673308030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558673262670 con 0x558673308570 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55867320a970 con 0x5586732fdea0 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f176c566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55867325e630 con 0x5586732fd580 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f1768856640 1 -- 172.21.15.45:0/9125263 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1754005560 con 0x7f17641002e0 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33880/0 (socket says 172.21.15.45:33880) 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 -- 172.21.15.45:0/2949892782 learned_addr learned my addr 172.21.15.45:0/2949892782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fd580 0x5586732fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f1762ffd640 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 0x558673308030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 -- 172.21.15.45:0/2949892782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fd580 msgr2=0x5586732fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fd580 0x5586732fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 -- 172.21.15.45:0/2949892782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 msgr2=0x558673308030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 0x558673308030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f17627fc640 1 -- 172.21.15.45:0/2949892782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867322e400 con 0x558673308570 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.258+0000 7f1762ffd640 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 0x558673308030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f17627fc640 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1758009c20 tx=0x7f1758011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec359accb5b9290a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f1768856640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 0x7f174c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f1749ffb640 1 -- 172.21.15.45:0/2949892782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758004050 con 0x558673308570 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f1749ffb640 1 -- 172.21.15.45:0/2949892782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1758005500 con 0x558673308570 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f1749ffb640 1 -- 172.21.15.45:0/2949892782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758004d00 con 0x558673308570 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f17637fe640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 0x7f174c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f1768856640 1 -- 172.21.15.45:0/9125263 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f17540cf770 con 0x7f17641002e0 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 -- 172.21.15.45:0/2949892782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 msgr2=0x55867330a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1758009c20 tx=0x7f1758011810 comp rx=0 tx=0).stop 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f17637fe640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 0x7f174c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f175c031f80 tx=0x7f175c004a60 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 -- 172.21.15.45:0/2949892782 shutdown_connections 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330a960 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fdea0 0x558673308030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 --2- 172.21.15.45:0/2949892782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fd580 0x5586732fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 -- 172.21.15.45:0/2949892782 >> 172.21.15.45:0/2949892782 conn(0x5586732f98a0 msgr2=0x5586732f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 -- 172.21.15.45:0/2949892782 shutdown_connections 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.259+0000 7f176c566740 1 -- 172.21.15.45:0/2949892782 wait complete. 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 -- start start 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 Processor -- start 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673307e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673305170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55867330b360 con 0x558673308570 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55867321e050 con 0x5586732fd580 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55867330b4e0 con 0x5586732fdea0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f176c566740 10 client.0 _ll_register_callbacks cb 0x558672f75be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33882/0 (socket says 172.21.15.45:33882) 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f17637fe640 1 -- 172.21.15.45:0/3687437617 learned_addr learned my addr 172.21.15.45:0/3687437617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.260+0000 7f17627fc640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673305170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1762ffd640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673307e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 -- 172.21.15.45:0/3687437617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 msgr2=0x558673305170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673305170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 -- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 msgr2=0x558673307e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673307e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 -- 172.21.15.45:0/3687437617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586731efdc0 con 0x558673308570 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f17637fe640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f175c003f20 tx=0x7f175c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b95a82e0e5ed46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c0045c0 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f175c004760 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1758005500 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f175c02f610 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f176c566740 10 client.18099 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558672f763f0 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1758005500 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.261+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f175c0309d0 con 0x558673308570 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 10 client.18099 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 10 client.18099 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 20 client.18099 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733185c0 0x55867331aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 4 client.18099 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55867326fbb0 con 0x5586733185c0 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f1762ffd640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733185c0 0x55867331aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f1762ffd640 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733185c0 0x55867331aa10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a52a42785e3d554b server_cookie=738bf1e0033b4984 in_seq=0 out_seq=0 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.262+0000 7f1742ffd640 10 client.18099 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f1742ffd640 1 -- 172.21.15.45:0/3687437617 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55867326fbb0 con 0x5586733185c0 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f1742ffd640 10 client.18099 handle_command_reply: tid=0 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 client.18099 shutdown 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 msgr2=0x5586733056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f175c003f20 tx=0x7f175c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 shutdown_connections 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733185c0 0x55867331aa10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733056b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673305170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 --2- 172.21.15.45:0/3687437617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673307e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 >> 172.21.15.45:0/3687437617 conn(0x5586732f98a0 msgr2=0x5586732f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f1742ffd640 0 client.18099 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 shutdown_connections 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 1 -- 172.21.15.45:0/3687437617 wait complete. 2024-04-25T00:30:43.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.263+0000 7f176c566740 20 client.18099 trim_cache size 0 max 16384 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 Processor -- start 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 -- start start 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673314220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673300650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17641aebf0 con 0x558673308570 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17641b0730 con 0x5586732fd580 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f176c566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17641af2c0 con 0x5586732fdea0 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33896/0 (socket says 172.21.15.45:33896) 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 -- 172.21.15.45:0/1385017969 learned_addr learned my addr 172.21.15.45:0/1385017969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 -- 172.21.15.45:0/1385017969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 msgr2=0x558673300650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673300650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f1762ffd640 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673314220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 -- 172.21.15.45:0/1385017969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 msgr2=0x558673314220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673314220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 -- 172.21.15.45:0/1385017969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586730407d0 con 0x558673308570 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17637fe640 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f175c043040 tx=0x7f175c003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abe8b6f5c6f0be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17497fa640 1 -- 172.21.15.45:0/1385017969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c004d40 con 0x558673308570 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17497fa640 1 -- 172.21.15.45:0/1385017969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c04e070 con 0x558673308570 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.277+0000 7f17497fa640 1 -- 172.21.15.45:0/1385017969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c02f7b0 con 0x558673308570 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- 172.21.15.45:0/1385017969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 msgr2=0x5586733079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f175c043040 tx=0x7f175c003fd0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- 172.21.15.45:0/1385017969 shutdown_connections 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x5586733079b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x558673300650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 --2- 172.21.15.45:0/1385017969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x558673314220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- 172.21.15.45:0/1385017969 >> 172.21.15.45:0/1385017969 conn(0x5586732f98a0 msgr2=0x5586732fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- 172.21.15.45:0/1385017969 shutdown_connections 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- 172.21.15.45:0/1385017969 wait complete. 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.278+0000 7f176c566740 1 -- start start 2024-04-25T00:30:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 Processor -- start 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x5586733107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x55867330bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17641b0060 con 0x558673308570 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17641af990 con 0x5586732fd580 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17641acbf0 con 0x5586732fdea0 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f176c566740 10 client.0 _ll_register_callbacks cb 0x7f175c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33900/0 (socket says 172.21.15.45:33900) 2024-04-25T00:30:43.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 -- 172.21.15.45:0/811911412 learned_addr learned my addr 172.21.15.45:0/811911412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f1762ffd640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x5586733107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17627fc640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x55867330bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 -- 172.21.15.45:0/811911412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 msgr2=0x55867330bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x55867330bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 -- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 msgr2=0x5586733107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x5586733107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 -- 172.21.15.45:0/811911412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867330c790 con 0x558673308570 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f17637fe640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f175c02a900 tx=0x7f175c030a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4399b560323fd9b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f1742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.279+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c06d060 con 0x558673308570 2024-04-25T00:30:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f175c004410 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175c04e070 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f175c004610 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 10 client.18105 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558673382c50 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c04e070 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f175c0537f0 con 0x558673308570 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 10 client.18105 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 10 client.18105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 20 client.18105 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733830d0 0x5586733854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 4 client.18105 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558673305a40 con 0x5586733830d0 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.280+0000 7f1762ffd640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733830d0 0x5586733854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f1742ffd640 10 client.18105 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f1762ffd640 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733830d0 0x5586733854c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec90dcc702dd6b51 server_cookie=a85c7a7f7d6d613e in_seq=0 out_seq=0 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f1742ffd640 1 -- 172.21.15.45:0/811911412 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558673305a40 con 0x5586733830d0 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f1742ffd640 10 client.18105 handle_command_reply: tid=0 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 client.18105 shutdown 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 msgr2=0x55867330c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f175c02a900 tx=0x7f175c030a50 comp rx=0 tx=0).stop 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 shutdown_connections 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x5586733830d0 0x5586733854c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558673308570 0x55867330c250 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586732fdea0 0x55867330bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f1742ffd640 0 client.18105 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 --2- 172.21.15.45:0/811911412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5586732fd580 0x5586733107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 >> 172.21.15.45:0/811911412 conn(0x5586732f98a0 msgr2=0x5586732f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 shutdown_connections 2024-04-25T00:30:43.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 1 -- 172.21.15.45:0/811911412 wait complete. 2024-04-25T00:30:43.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.281+0000 7f176c566740 20 client.18105 trim_cache size 0 max 16384 2024-04-25T00:30:43.289 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 msgr2=0x7f174c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 0x7f174c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f175c031f80 tx=0x7f175c004a60 comp rx=0 tx=0).stop 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 msgr2=0x7f17641980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641980b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f175400e9f0 tx=0x7f175400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 shutdown_connections 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f174c0a2d00 0x7f174c0a51c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f176419c970 0x7f176419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764106aa0 0x7f17641985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 --2- 172.21.15.45:0/9125263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17641002e0 0x7f17641980b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 >> 172.21.15.45:0/9125263 conn(0x7f17640758d0 msgr2=0x7f17640fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 shutdown_connections 2024-04-25T00:30:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.282+0000 7f1769858640 1 -- 172.21.15.45:0/9125263 wait complete. 2024-04-25T00:30:43.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:43 smithi045 ceph-mon[29194]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:43.292 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:43 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1358890060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:43.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:30:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:43 smithi122 ceph-mon[32959]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:43.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:43 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1358890060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:43.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:43 smithi071 ceph-mon[32667]: pgmap v269: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:43.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:43 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1358890060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:30:43.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 -- 172.21.15.45:0/1956252679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d4101710 msgr2=0x7fc4d4101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 --2- 172.21.15.45:0/1956252679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d4101710 0x7fc4d4101b90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009930 tx=0x7fc4c802a420 comp rx=0 tx=0).stop 2024-04-25T00:30:43.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 -- 172.21.15.45:0/1956252679 shutdown_connections 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 --2- 172.21.15.45:0/1956252679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d41020d0 0x7fc4d410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 --2- 172.21.15.45:0/1956252679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d4101710 0x7fc4d4101b90 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 --2- 172.21.15.45:0/1956252679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 0x7fc4d4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 -- 172.21.15.45:0/1956252679 >> 172.21.15.45:0/1956252679 conn(0x7fc4d40fd180 msgr2=0x7fc4d40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 -- 172.21.15.45:0/1956252679 shutdown_connections 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.396+0000 7fc4d962a640 1 -- 172.21.15.45:0/1956252679 wait complete. 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 Processor -- start 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 -- start start 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d4101710 0x7fc4d4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 0x7fc4d419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4111af0 con 0x7fc4d41020d0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4111970 con 0x7fc4d4107720 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d962a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4111c70 con 0x7fc4d4101710 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.397+0000 7fc4d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33918/0 (socket says 172.21.15.45:33918) 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 -- 172.21.15.45:0/643131137 learned_addr learned my addr 172.21.15.45:0/643131137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 0x7fc4d419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d4101710 0x7fc4d4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 -- 172.21.15.45:0/643131137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d4101710 msgr2=0x7fc4d4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d4101710 0x7fc4d4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 -- 172.21.15.45:0/643131137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 msgr2=0x7fc4d419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 0x7fc4d419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 -- 172.21.15.45:0/643131137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d419d080 con 0x7fc4d41020d0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009230 tx=0x7fc4c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4bbfff640 1 -- 172.21.15.45:0/643131137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c803f030 con 0x7fc4d41020d0 2024-04-25T00:30:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d419d310 con 0x7fc4d41020d0 2024-04-25T00:30:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4bbfff640 1 -- 172.21.15.45:0/643131137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c8003ad0 con 0x7fc4d41020d0 2024-04-25T00:30:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4bbfff640 1 -- 172.21.15.45:0/643131137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8035070 con 0x7fc4d41020d0 2024-04-25T00:30:43.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.398+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d41a4c00 con 0x7fc4d41020d0 2024-04-25T00:30:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.400+0000 7fc4bbfff640 1 -- 172.21.15.45:0/643131137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc4c8042030 con 0x7fc4d41020d0 2024-04-25T00:30:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 Processor -- start 2024-04-25T00:30:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 -- start start 2024-04-25T00:30:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4bbfff640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 0x7fc4bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 0x55d2395fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395fa570 0x55d2395fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d239554670 con 0x55d2395ef580 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d2394fc970 con 0x55d2395fa570 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4dc414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d239550630 con 0x55d2395efea0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 0x7fc4bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33930/0 (socket says 172.21.15.45:33930) 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4bbfff640 1 -- 172.21.15.45:0/643131137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7fc4c80efe00 con 0x7fc4d41020d0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3978777409 learned_addr learned my addr 172.21.15.45:0/3978777409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395fa570 0x55d2395fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.401+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 0x55d2395fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3978777409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 msgr2=0x55d2395fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 0x55d2395fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3978777409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395fa570 msgr2=0x55d2395fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395fa570 0x55d2395fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3978777409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d239520400 con 0x55d2395ef580 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 0x55d2395fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 0x7fc4bc0a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc004410 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009c70 tx=0x7fc4c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb896436c435d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4b97fa640 1 -- 172.21.15.45:0/3978777409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004230 con 0x55d2395ef580 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4b97fa640 1 -- 172.21.15.45:0/3978777409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c40043d0 con 0x55d2395ef580 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.402+0000 7fc4b97fa640 1 -- 172.21.15.45:0/3978777409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4019760 con 0x55d2395ef580 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- 172.21.15.45:0/3978777409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 msgr2=0x55d2395ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009c70 tx=0x7fc4c4011810 comp rx=0 tx=0).stop 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- 172.21.15.45:0/3978777409 shutdown_connections 2024-04-25T00:30:43.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395fa570 0x55d2395fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395efea0 0x55d2395fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3978777409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395ef960 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- 172.21.15.45:0/3978777409 >> 172.21.15.45:0/3978777409 conn(0x55d2395eb8a0 msgr2=0x55d2395ebc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- 172.21.15.45:0/3978777409 shutdown_connections 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- 172.21.15.45:0/3978777409 wait complete. 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- start start 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 Processor -- start 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d2395fd360 con 0x55d2395ef580 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d239510050 con 0x55d2395efea0 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d2395fd4e0 con 0x55d2395fa570 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4dc414740 10 client.0 _ll_register_callbacks cb 0x55d239267be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33934/0 (socket says 172.21.15.45:33934) 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.403+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/4252188009 learned_addr learned my addr 172.21.15.45:0/4252188009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 msgr2=0x55d2395f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/4252188009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 msgr2=0x55d2395f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/4252188009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2394e1dc0 con 0x55d2395ef580 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005890 tx=0x7fc4c40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ed8ac6a2b008e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4027070 con 0x55d2395ef580 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7fc4c4003800 con 0x55d2395ef580 2024-04-25T00:30:43.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c40043d0 con 0x55d2395ef580 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4c4033050 con 0x55d2395ef580 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4dc414740 10 client.18117 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2392683f0 con 0x55d2395ef580 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c40043d0 con 0x55d2395ef580 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4c4018070 con 0x55d2395ef580 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.404+0000 7fc4dc414740 10 client.18117 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4dc414740 10 client.18117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4dc414740 20 client.18117 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d23960a470 0x55d23960c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4dc414740 4 client.18117 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d239561bb0 con 0x55d23960a470 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d23960a470 0x55d23960c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4b6ff5640 10 client.18117 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:43.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.405+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d23960a470 0x55d23960c860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d26dd3cb07238204 server_cookie=939b347e16bc095f in_seq=0 out_seq=0 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/4252188009 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d239561bb0 con 0x55d23960a470 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4b6ff5640 10 client.18117 handle_command_reply: tid=0 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 client.18117 shutdown 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 msgr2=0x55d2395f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4005890 tx=0x7fc4c40098d0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 shutdown_connections 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d23960a470 0x55d23960c860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4b6ff5640 0 client.18117 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 --2- 172.21.15.45:0/4252188009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d2395f9e70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 >> 172.21.15.45:0/4252188009 conn(0x55d2395eb8a0 msgr2=0x55d2395eab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 shutdown_connections 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 1 -- 172.21.15.45:0/4252188009 wait complete. 2024-04-25T00:30:43.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.406+0000 7fc4dc414740 20 client.18117 trim_cache size 0 max 16384 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 Processor -- start 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 -- start start 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41aec50 con 0x55d2395ef580 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41b0790 con 0x55d2395efea0 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4dc414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41af320 con 0x55d2395fa570 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33946/0 (socket says 172.21.15.45:33946) 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3332638125 learned_addr learned my addr 172.21.15.45:0/3332638125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3332638125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 msgr2=0x55d2395f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3332638125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 msgr2=0x55d2395f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 -- 172.21.15.45:0/3332638125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2393327d0 con 0x55d2395ef580 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.420+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011850 tx=0x7fc4c4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787bbd55a5a980b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4b8ff9640 1 -- 172.21.15.45:0/3332638125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4027070 con 0x55d2395ef580 2024-04-25T00:30:43.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4b8ff9640 1 -- 172.21.15.45:0/3332638125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4015030 con 0x55d2395ef580 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4b8ff9640 1 -- 172.21.15.45:0/3332638125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4003800 con 0x55d2395ef580 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 -- 172.21.15.45:0/3332638125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 msgr2=0x55d239606280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4011850 tx=0x7fc4c4009ee0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 -- 172.21.15.45:0/3332638125 shutdown_connections 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395efea0 0x55d2395f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 --2- 172.21.15.45:0/3332638125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395ef580 0x55d239606280 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 -- 172.21.15.45:0/3332638125 >> 172.21.15.45:0/3332638125 conn(0x55d2395eb8a0 msgr2=0x55d2395f6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 -- 172.21.15.45:0/3332638125 shutdown_connections 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.421+0000 7fc4dc414740 1 -- 172.21.15.45:0/3332638125 wait complete. 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 -- start start 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 Processor -- start 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395ef580 0x55d2396028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41b00c0 con 0x55d2395efea0 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41af9f0 con 0x55d2395ef580 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d41acc50 con 0x55d2395fa570 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4dc414740 10 client.0 _ll_register_callbacks cb 0x7fc4c4009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33962/0 (socket says 172.21.15.45:33962) 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 -- 172.21.15.45:0/2330421369 learned_addr learned my addr 172.21.15.45:0/2330421369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395ef580 0x55d2396028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 -- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 msgr2=0x55d2395fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395fe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 -- 172.21.15.45:0/2330421369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395ef580 msgr2=0x55d2396028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395ef580 0x55d2396028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 -- 172.21.15.45:0/2330421369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2395fe850 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d37fe640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4d27fc640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8032c00 tx=0x7fc4c80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0333efe88298c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.422+0000 7fc4b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8049040 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7fc4c803bd60 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c4015030 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4c8003c80 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 10 client.18123 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d239674b00 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4015030 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4c80bc210 con 0x55d2395efea0 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 10 client.18123 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 10 client.18123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:30:43.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 20 client.18123 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d239675200 0x55d239677620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 4 client.18123 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2395f2d60 con 0x55d239675200 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.423+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d239675200 0x55d239677620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4b6ff5640 10 client.18123 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4d2ffd640 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d239675200 0x55d239677620 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b85dcf45cc31868a server_cookie=edf13a583899aba4 in_seq=0 out_seq=0 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4b6ff5640 1 -- 172.21.15.45:0/2330421369 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2395f2d60 con 0x55d239675200 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4b6ff5640 10 client.18123 handle_command_reply: tid=0 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 client.18123 shutdown 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 msgr2=0x55d2395fdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8032c00 tx=0x7fc4c80043e0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 shutdown_connections 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55d239675200 0x55d239677620 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2395fa570 0x55d2395fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2395efea0 0x55d2395fdd50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 --2- 172.21.15.45:0/2330421369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2395ef580 0x55d2396028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4b6ff5640 0 client.18123 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 >> 172.21.15.45:0/2330421369 conn(0x55d2395eb8a0 msgr2=0x55d2395eab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 shutdown_connections 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 1 -- 172.21.15.45:0/2330421369 wait complete. 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.424+0000 7fc4dc414740 20 client.18123 trim_cache size 0 max 16384 2024-04-25T00:30:43.432 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 msgr2=0x7fc4bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 0x7fc4bc0a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc004410 comp rx=0 tx=0).stop 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 msgr2=0x7fc4d41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009230 tx=0x7fc4c8004270 comp rx=0 tx=0).stop 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 shutdown_connections 2024-04-25T00:30:43.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc4bc0a2d00 0x7fc4bc0a51c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d4107720 0x7fc4d419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d41020d0 0x7fc4d41985d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 --2- 172.21.15.45:0/643131137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4d4101710 0x7fc4d4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 >> 172.21.15.45:0/643131137 conn(0x7fc4d40fd180 msgr2=0x7fc4d40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 shutdown_connections 2024-04-25T00:30:43.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.425+0000 7fc4d962a640 1 -- 172.21.15.45:0/643131137 wait complete. 2024-04-25T00:30:43.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- 172.21.15.45:0/1354708847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 msgr2=0x7f8e7c10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 --2- 172.21.15.45:0/1354708847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10d4e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009cd0 tx=0x7f8e7402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- 172.21.15.45:0/1354708847 shutdown_connections 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 --2- 172.21.15.45:0/1354708847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10d4e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 --2- 172.21.15.45:0/1354708847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c100610 0x7f8e7c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 --2- 172.21.15.45:0/1354708847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- 172.21.15.45:0/1354708847 >> 172.21.15.45:0/1354708847 conn(0x7f8e7c077740 msgr2=0x7f8e7c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- 172.21.15.45:0/1354708847 shutdown_connections 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- 172.21.15.45:0/1354708847 wait complete. 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 Processor -- start 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.542+0000 7f8e829c8640 1 -- start start 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c100610 0x7f8e7c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c10e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c111b30 con 0x7f8e7c100fd0 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1119b0 con 0x7f8e7c100610 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e829c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c111cb0 con 0x7f8e7c1066b0 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33972/0 (socket says 172.21.15.45:33972) 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3234622395 learned_addr learned my addr 172.21.15.45:0/3234622395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c100610 0x7f8e7c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c10e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3234622395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 msgr2=0x7f8e7c10e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c10e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3234622395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c100610 msgr2=0x7f8e7c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c100610 0x7f8e7c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3234622395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c1a6490 con 0x7f8e7c100fd0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000e9f0 tx=0x7f8e7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.543+0000 7f8e797fa640 1 -- 172.21.15.45:0/3234622395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000cd80 con 0x7f8e7c100fd0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.544+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c1a6780 con 0x7f8e7c100fd0 2024-04-25T00:30:43.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.544+0000 7f8e797fa640 1 -- 172.21.15.45:0/3234622395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e70005500 con 0x7f8e7c100fd0 2024-04-25T00:30:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.544+0000 7f8e797fa640 1 -- 172.21.15.45:0/3234622395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70010690 con 0x7f8e7c100fd0 2024-04-25T00:30:43.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.544+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c1a6c80 con 0x7f8e7c100fd0 2024-04-25T00:30:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.546+0000 7f8e797fa640 1 -- 172.21.15.45:0/3234622395 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8e700046c0 con 0x7f8e7c100fd0 2024-04-25T00:30:43.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 Processor -- start 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 -- start start 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 0x55bc14516960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14521030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc1447b670 con 0x55bc14521570 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc14423970 con 0x55bc14516580 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e857b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc14477630 con 0x55bc14516ea0 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33982/0 (socket says 172.21.15.45:33982) 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/4010289217 learned_addr learned my addr 172.21.15.45:0/4010289217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e80f65640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 0x55bc14516960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14521030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e797fa640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 0x7f8e640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/4010289217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 msgr2=0x55bc14521030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14521030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/4010289217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 msgr2=0x55bc14516960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 0x55bc14516960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/4010289217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc14447400 con 0x55bc14521570 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e80f65640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 0x55bc14516960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.547+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 0x7f8e640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e797fa640 1 -- 172.21.15.45:0/3234622395 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8e700cfd00 con 0x7f8e7c100fd0 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8e700ccf70 tx=0x7f8e70021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a27e9955c135ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e627fc640 1 -- 172.21.15.45:0/4010289217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7009b940 con 0x55bc14521570 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e627fc640 1 -- 172.21.15.45:0/4010289217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7000dda0 con 0x55bc14521570 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 0x7f8e640a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e74004af0 tx=0x7f8e740095c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e627fc640 1 -- 172.21.15.45:0/4010289217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000e180 con 0x55bc14521570 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e857b3740 1 -- 172.21.15.45:0/4010289217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 msgr2=0x55bc14523960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.548+0000 7f8e857b3740 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8e700ccf70 tx=0x7f8e70021000 comp rx=0 tx=0).stop 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- 172.21.15.45:0/4010289217 shutdown_connections 2024-04-25T00:30:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14523960 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14521030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- 172.21.15.45:0/4010289217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516580 0x55bc14516960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- 172.21.15.45:0/4010289217 >> 172.21.15.45:0/4010289217 conn(0x55bc145128a0 msgr2=0x55bc14512c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- 172.21.15.45:0/4010289217 shutdown_connections 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- 172.21.15.45:0/4010289217 wait complete. 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- start start 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 Processor -- start 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc14520ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc1451e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc14524300 con 0x55bc14516ea0 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc14437050 con 0x55bc14521570 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc14524480 con 0x55bc14516580 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.549+0000 7f8e857b3740 10 client.0 _ll_register_callbacks cb 0x55bc1418ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33996/0 (socket says 172.21.15.45:33996) 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3376859391 learned_addr learned my addr 172.21.15.45:0/3376859391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc14520ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc1451e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 msgr2=0x55bc14520ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc14520ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3376859391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 msgr2=0x55bc1451e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc1451e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 -- 172.21.15.45:0/3376859391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc14408dc0 con 0x55bc14516ea0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8e700049d0 tx=0x7f8e7009bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff0eaa0c3b60105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7009b760 con 0x55bc14516ea0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8e700affe0 con 0x55bc14516ea0 2024-04-25T00:30:43.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc14488bb0 con 0x55bc14516ea0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e700b9050 con 0x55bc14516ea0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e857b3740 10 client.18135 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.550+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc1418f3f0 con 0x55bc14516ea0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bc14488bb0 con 0x55bc14516ea0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e70098470 con 0x55bc14516ea0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 10 client.18135 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 10 client.18135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 20 client.18135 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145313f0 0x55bc145337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 4 client.18135 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc14482f00 con 0x55bc145313f0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145313f0 0x55bc145337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145313f0 0x55bc145337e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e33b4d4007afcfb8 server_cookie=d044bb2c5fa9467c in_seq=0 out_seq=0 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.551+0000 7f8e5b7fe640 10 client.18135 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:43.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.552+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/3376859391 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc14482f00 con 0x55bc145313f0 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.552+0000 7f8e5b7fe640 10 client.18135 handle_command_reply: tid=0 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.552+0000 7f8e857b3740 1 client.18135 shutdown 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 msgr2=0x55bc1451e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8e700049d0 tx=0x7f8e7009bcc0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 shutdown_connections 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145313f0 0x55bc145337e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc1451e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516ea0 0x55bc1451e1b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e5b7fe640 0 client.18135 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3376859391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc14520ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 >> 172.21.15.45:0/3376859391 conn(0x55bc145128a0 msgr2=0x55bc14511b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 shutdown_connections 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 1 -- 172.21.15.45:0/3376859391 wait complete. 2024-04-25T00:30:43.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.553+0000 7f8e857b3740 20 client.18135 trim_cache size 0 max 16384 2024-04-25T00:30:43.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 Processor -- start 2024-04-25T00:30:43.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 -- start start 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14538d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc145392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b0f00 con 0x55bc14516580 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b2a40 con 0x55bc14521570 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e857b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b15d0 con 0x55bc14516ea0 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e7bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34000/0 (socket says 172.21.15.45:34000) 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.566+0000 7f8e7bfff640 1 -- 172.21.15.45:0/3598767593 learned_addr learned my addr 172.21.15.45:0/3598767593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14538d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc145392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 -- 172.21.15.45:0/3598767593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 msgr2=0x55bc14538d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14538d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 -- 172.21.15.45:0/3598767593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 msgr2=0x55bc145392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc145392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 -- 172.21.15.45:0/3598767593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc142597d0 con 0x55bc14516580 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14538d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e80f65640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc145392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009dd0 tx=0x7f8e6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2183b15e13d504d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e61ffb640 1 -- 172.21.15.45:0/3598767593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c024070 con 0x55bc14516580 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e61ffb640 1 -- 172.21.15.45:0/3598767593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c01f810 con 0x55bc14516580 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e61ffb640 1 -- 172.21.15.45:0/3598767593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c01ee60 con 0x55bc14516580 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e857b3740 1 -- 172.21.15.45:0/3598767593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 msgr2=0x55bc145209f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.567+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009dd0 tx=0x7f8e6c005f90 comp rx=0 tx=0).stop 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- 172.21.15.45:0/3598767593 shutdown_connections 2024-04-25T00:30:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14521570 0x55bc145392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516ea0 0x55bc14538d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- 172.21.15.45:0/3598767593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14516580 0x55bc145209f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- 172.21.15.45:0/3598767593 >> 172.21.15.45:0/3598767593 conn(0x55bc145128a0 msgr2=0x55bc145112b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- 172.21.15.45:0/3598767593 shutdown_connections 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- 172.21.15.45:0/3598767593 wait complete. 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- start start 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 Processor -- start 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc145253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516ea0 0x55bc14525910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b2370 con 0x55bc14521570 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1b1ca0 con 0x55bc14516ea0 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c1aef00 con 0x55bc14516580 2024-04-25T00:30:43.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e857b3740 10 client.0 _ll_register_callbacks cb 0x7f8e6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.568+0000 7f8e80f65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34004/0 (socket says 172.21.15.45:34004) 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 -- 172.21.15.45:0/1611927361 learned_addr learned my addr 172.21.15.45:0/1611927361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc145253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e7b7fe640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516ea0 0x55bc14525910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 -- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 msgr2=0x55bc145253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc145253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 -- 172.21.15.45:0/1611927361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516ea0 msgr2=0x55bc14525910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516ea0 0x55bc14525910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 -- 172.21.15.45:0/1611927361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc145967c0 con 0x55bc14521570 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e80f65640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009cd0 tx=0x7f8e7402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9a948ca8b47e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74004250 con 0x55bc14521570 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8e740043f0 con 0x55bc14521570 2024-04-25T00:30:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e6c01f810 con 0x55bc14521570 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e740045f0 con 0x55bc14521570 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e857b3740 10 client.18141 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.569+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc14597a10 con 0x55bc14521570 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c01f810 con 0x55bc14521570 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e74030070 con 0x55bc14521570 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 10 client.18141 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 10 client.18141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 20 client.18141 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145981f0 0x55bc1459a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 4 client.18141 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc14400820 con 0x55bc145981f0 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145981f0 0x55bc1459a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e5b7fe640 10 client.18141 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.570+0000 7f8e7bfff640 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145981f0 0x55bc1459a610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e77e12726de5ac30 server_cookie=75c8d6d95201286f in_seq=0 out_seq=0 2024-04-25T00:30:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e5b7fe640 1 -- 172.21.15.45:0/1611927361 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc14400820 con 0x55bc145981f0 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e5b7fe640 10 client.18141 handle_command_reply: tid=0 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 client.18141 shutdown 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 msgr2=0x55bc14525e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009cd0 tx=0x7f8e7402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 shutdown_connections 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55bc145981f0 0x55bc1459a610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc14521570 0x55bc14525e50 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc14516ea0 0x55bc14525910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e5b7fe640 0 client.18141 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 --2- 172.21.15.45:0/1611927361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc14516580 0x55bc145253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 >> 172.21.15.45:0/1611927361 conn(0x55bc145128a0 msgr2=0x55bc14511b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 shutdown_connections 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 1 -- 172.21.15.45:0/1611927361 wait complete. 2024-04-25T00:30:43.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.571+0000 7f8e857b3740 20 client.18141 trim_cache size 0 max 16384 2024-04-25T00:30:43.579 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:43.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 msgr2=0x7f8e640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 0x7f8e640a51c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e74004af0 tx=0x7f8e740095c0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 msgr2=0x7f8e7c10e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000e9f0 tx=0x7f8e7000edf0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 shutdown_connections 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8e640a2d00 0x7f8e640a51c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e7c1066b0 0x7f8e7c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e7c100fd0 0x7f8e7c10e4b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 --2- 172.21.15.45:0/3234622395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e7c100610 0x7f8e7c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 >> 172.21.15.45:0/3234622395 conn(0x7f8e7c077740 msgr2=0x7f8e7c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 shutdown_connections 2024-04-25T00:30:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.572+0000 7f8e829c8640 1 -- 172.21.15.45:0/3234622395 wait complete. 2024-04-25T00:30:43.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 -- 172.21.15.45:0/1789092640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 msgr2=0x7f8f7c06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/1789092640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c06a240 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009930 tx=0x7f8f6402a420 comp rx=0 tx=0).stop 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 -- 172.21.15.45:0/1789092640 shutdown_connections 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/1789092640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c06a780 0x7f8f7c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/1789092640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c06a240 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/1789092640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c107730 0x7f8f7c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 -- 172.21.15.45:0/1789092640 >> 172.21.15.45:0/1789092640 conn(0x7f8f7c0776a0 msgr2=0x7f8f7c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 -- 172.21.15.45:0/1789092640 shutdown_connections 2024-04-25T00:30:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.687+0000 7f8f80b3b640 1 -- 172.21.15.45:0/1789092640 wait complete. 2024-04-25T00:30:43.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 Processor -- start 2024-04-25T00:30:43.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 -- start start 2024-04-25T00:30:43.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 0x7f8f7c100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 0x7f8f7c1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c111b40 con 0x7f8f7c069dc0 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1119c0 con 0x7f8f7c107730 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f80b3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c111cc0 con 0x7f8f7c06a780 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34026/0 (socket says 172.21.15.45:34026) 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 -- 172.21.15.45:0/2995149858 learned_addr learned my addr 172.21.15.45:0/2995149858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f73fff640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 0x7f8f7c100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 0x7f8f7c1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 -- 172.21.15.45:0/2995149858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 msgr2=0x7f8f7c100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 0x7f8f7c100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 -- 172.21.15.45:0/2995149858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 msgr2=0x7f8f7c1010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 0x7f8f7c1010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 -- 172.21.15.45:0/2995149858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1a64e0 con 0x7f8f7c069dc0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f73fff640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 0x7f8f7c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 0x7f8f7c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.688+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d950 tx=0x7f8f6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.689+0000 7f8f737fe640 1 -- 172.21.15.45:0/2995149858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017070 con 0x7f8f7c069dc0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.689+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1a67d0 con 0x7f8f7c069dc0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.689+0000 7f8f737fe640 1 -- 172.21.15.45:0/2995149858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f6c009c30 con 0x7f8f7c069dc0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.689+0000 7f8f737fe640 1 -- 172.21.15.45:0/2995149858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00f6b0 con 0x7f8f7c069dc0 2024-04-25T00:30:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.689+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a6d70 con 0x7f8f7c069dc0 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f737fe640 1 -- 172.21.15.45:0/2995149858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8f6c0109d0 con 0x7f8f7c069dc0 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 Processor -- start 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 -- start start 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 0x56346f2e9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9ea0 0x56346f2f4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56346f24e670 con 0x56346f2f4570 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56346f1f6970 con 0x56346f2e9580 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f8384a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56346f24a630 con 0x56346f2e9ea0 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34040/0 (socket says 172.21.15.45:34040) 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f73fff640 1 -- 172.21.15.45:0/2038050955 learned_addr learned my addr 172.21.15.45:0/2038050955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9ea0 0x56346f2f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 0x56346f2e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.692+0000 7f8f737fe640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 0x7f8f680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 -- 172.21.15.45:0/2038050955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9ea0 msgr2=0x56346f2f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9ea0 0x56346f2f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 -- 172.21.15.45:0/2038050955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 msgr2=0x56346f2e9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 0x56346f2e9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 -- 172.21.15.45:0/2038050955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56346f21a400 con 0x56346f2f4570 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 0x56346f2e9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 0x7f8f680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f737fe640 1 -- 172.21.15.45:0/2995149858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8f6c0cf4c0 con 0x7f8f7c069dc0 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f73fff640 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f64009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0978df079748fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f70ff9640 1 -- 172.21.15.45:0/2038050955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6403f070 con 0x56346f2f4570 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f70ff9640 1 -- 172.21.15.45:0/2038050955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f640317b0 con 0x56346f2f4570 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.693+0000 7f8f70ff9640 1 -- 172.21.15.45:0/2038050955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64035070 con 0x56346f2f4570 2024-04-25T00:30:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- 172.21.15.45:0/2038050955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 msgr2=0x56346f2f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 0x7f8f680a51c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005fd0 tx=0x7f8f74004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f64009680 comp rx=0 tx=0).stop 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- 172.21.15.45:0/2038050955 shutdown_connections 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2f6960 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9ea0 0x56346f2f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 --2- 172.21.15.45:0/2038050955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9580 0x56346f2e9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- 172.21.15.45:0/2038050955 >> 172.21.15.45:0/2038050955 conn(0x56346f2e58a0 msgr2=0x56346f2e5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- 172.21.15.45:0/2038050955 shutdown_connections 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- 172.21.15.45:0/2038050955 wait complete. 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 -- start start 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.694+0000 7f8f8384a740 1 Processor -- start 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2f4570 0x56346f2f1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56346f2f7300 con 0x56346f2e9ea0 2024-04-25T00:30:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56346f20a050 con 0x56346f2f4570 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56346f2f7480 con 0x56346f2e9580 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 10 client.0 _ll_register_callbacks cb 0x56346ef61be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34052/0 (socket says 172.21.15.45:34052) 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 -- 172.21.15.45:0/458343696 learned_addr learned my addr 172.21.15.45:0/458343696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2f4570 0x56346f2f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 -- 172.21.15.45:0/458343696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 msgr2=0x56346f2f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 -- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2f4570 msgr2=0x56346f2f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2f4570 0x56346f2f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 -- 172.21.15.45:0/458343696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56346f1dbdc0 con 0x56346f2e9ea0 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f73fff640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000c00 tx=0x7f8f64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86faf1df6e7c30c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6403f070 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f640317b0 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8f64004050 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f64055050 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.695+0000 7f8f8384a740 10 client.18153 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56346ef623f0 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f640317b0 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f640035f0 con 0x56346f2e9ea0 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 10 client.18153 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 20 client.18153 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f304360 0x56346f306750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 4 client.18153 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56346f25bbb0 con 0x56346f304360 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.696+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f304360 0x56346f306750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.697+0000 7f8f61ffb640 10 client.18153 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.697+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f304360 0x56346f306750 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509b9695eb09a2a4 server_cookie=15dd8ddab1a26a63 in_seq=0 out_seq=0 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.697+0000 7f8f61ffb640 1 -- 172.21.15.45:0/458343696 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56346f25bbb0 con 0x56346f304360 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.697+0000 7f8f61ffb640 10 client.18153 handle_command_reply: tid=0 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 client.18153 shutdown 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 msgr2=0x56346f2f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f8f64000c00 tx=0x7f8f64009260 comp rx=0 tx=0).stop 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 shutdown_connections 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2f4570 0x56346f2f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f304360 0x56346f306750 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2e9ea0 0x56346f2f11c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 --2- 172.21.15.45:0/458343696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f61ffb640 0 client.18153 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 >> 172.21.15.45:0/458343696 conn(0x56346f2e58a0 msgr2=0x56346f2eb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 shutdown_connections 2024-04-25T00:30:43.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 1 -- 172.21.15.45:0/458343696 wait complete. 2024-04-25T00:30:43.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.698+0000 7f8f8384a740 20 client.18153 trim_cache size 0 max 16384 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 Processor -- start 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 -- start start 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f3001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f30bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1b0dd0 con 0x56346f2f4570 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1b2910 con 0x56346f2e9ea0 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f8384a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1b14a0 con 0x56346f2e9580 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34068/0 (socket says 172.21.15.45:34068) 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/578259747 learned_addr learned my addr 172.21.15.45:0/578259747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f3001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f73fff640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f30bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/578259747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 msgr2=0x56346f3001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f3001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/578259747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 msgr2=0x56346f30bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f30bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/578259747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56346f02c7d0 con 0x56346f2f4570 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f73fff640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f30bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400fbc0 tx=0x7f8f74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7eb884d70d87cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.712+0000 7f8f63fff640 1 -- 172.21.15.45:0/578259747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74004d20 con 0x56346f2f4570 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f63fff640 1 -- 172.21.15.45:0/578259747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f74022070 con 0x56346f2f4570 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f63fff640 1 -- 172.21.15.45:0/578259747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f740055f0 con 0x56346f2f4570 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- 172.21.15.45:0/578259747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 msgr2=0x56346f30c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400fbc0 tx=0x7f8f74012000 comp rx=0 tx=0).stop 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- 172.21.15.45:0/578259747 shutdown_connections 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f30c2c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f30bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 --2- 172.21.15.45:0/578259747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f3001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- 172.21.15.45:0/578259747 >> 172.21.15.45:0/578259747 conn(0x56346f2e58a0 msgr2=0x56346f2efa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- 172.21.15.45:0/578259747 shutdown_connections 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- 172.21.15.45:0/578259747 wait complete. 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 -- start start 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.713+0000 7f8f8384a740 1 Processor -- start 2024-04-25T00:30:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f2f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1b2240 con 0x56346f2f4570 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1b1b70 con 0x56346f2e9ea0 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1aedd0 con 0x56346f2e9580 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f8384a740 10 client.0 _ll_register_callbacks cb 0x7f8f6c0978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59006/0 (socket says 172.21.15.45:59006) 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/1005537220 learned_addr learned my addr 172.21.15.45:0/1005537220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f73fff640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f2f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/1005537220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 msgr2=0x56346f2fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2fcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 msgr2=0x56346f2f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:43.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f2f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 -- 172.21.15.45:0/1005537220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56346f2fad10 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f7ad9d640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8f740052d0 tx=0x7f8f7400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d824f08fe1caa7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.714+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74035060 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f8f74005570 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f74022070 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f7403c050 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 10 client.18159 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56346f36a610 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f74022070 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f7402f070 con 0x56346f2f4570 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 10 client.18159 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:43.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 20 client.18159 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f36ac50 0x56346f36d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 4 client.18159 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56346f255f00 con 0x56346f36ac50 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.715+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f36ac50 0x56346f36d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f7a59c640 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f36ac50 0x56346f36d070 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a1231323882562 server_cookie=6c8ac3b86245b46a in_seq=0 out_seq=0 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f61ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f61ffb640 1 -- 172.21.15.45:0/1005537220 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56346f255f00 con 0x56346f36ac50 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f61ffb640 10 client.18159 handle_command_reply: tid=0 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 client.18159 shutdown 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 msgr2=0x56346f2fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8f740052d0 tx=0x7f8f7400fda0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 shutdown_connections 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56346f36ac50 0x56346f36d070 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56346f2f4570 0x56346f2fa4e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56346f2e9ea0 0x56346f2f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 --2- 172.21.15.45:0/1005537220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56346f2e9580 0x56346f2fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 >> 172.21.15.45:0/1005537220 conn(0x56346f2e58a0 msgr2=0x56346f2ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f61ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 shutdown_connections 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 1 -- 172.21.15.45:0/1005537220 wait complete. 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.716+0000 7f8f8384a740 20 client.18159 trim_cache size 0 max 16384 2024-04-25T00:30:43.724 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 msgr2=0x7f8f680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 0x7f8f680a51c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005fd0 tx=0x7f8f74004a40 comp rx=0 tx=0).stop 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 msgr2=0x7f8f7c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d950 tx=0x7f8f6c00de10 comp rx=0 tx=0).stop 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 shutdown_connections 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8f680a2d00 0x7f8f680a51c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f7c107730 0x7f8f7c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c06a780 0x7f8f7c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 --2- 172.21.15.45:0/2995149858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f7c069dc0 0x7f8f7c100670 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 >> 172.21.15.45:0/2995149858 conn(0x7f8f7c0776a0 msgr2=0x7f8f7c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 shutdown_connections 2024-04-25T00:30:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:43.717+0000 7f8f80b3b640 1 -- 172.21.15.45:0/2995149858 wait complete. 2024-04-25T00:30:44.729 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:30:44.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:30:44.729 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/459903813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3810e150 msgr2=0x7f7c38110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/459903813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3810e150 0x7f7c38110540 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009ca0 tx=0x7f7c3002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/459903813 shutdown_connections 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/459903813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c3810e150 0x7f7c38110540 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/459903813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3806a6e0 0x7f7c3810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/459903813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c38069dc0 0x7f7c3806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/459903813 >> 172.21.15.45:0/459903813 conn(0x7f7c380776a0 msgr2=0x7f7c38077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/459903813 shutdown_connections 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.848+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/459903813 wait complete. 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 Processor -- start 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 -- start start 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c3806a6e0 0x7f7c3819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3810e150 0x7f7c3819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38113d10 con 0x7f7c38069dc0 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38113b90 con 0x7f7c3810e150 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3fa2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c38113e90 con 0x7f7c3806a6e0 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3d7c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3d7c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59016/0 (socket says 172.21.15.45:59016) 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/573286608 learned_addr learned my addr 172.21.15.45:0/573286608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:44.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3810e150 0x7f7c3819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.849+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c3806a6e0 0x7f7c3819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/573286608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c3806a6e0 msgr2=0x7f7c3819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c3806a6e0 0x7f7c3819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/573286608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3810e150 msgr2=0x7f7c3819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3810e150 0x7f7c3819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/573286608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3819f1b0 con 0x7f7c38069dc0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d950 tx=0x7f7c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c2e7fc640 1 -- 172.21.15.45:0/573286608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34017070 con 0x7f7c38069dc0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c2e7fc640 1 -- 172.21.15.45:0/573286608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34009c30 con 0x7f7c38069dc0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3819f4a0 con 0x7f7c38069dc0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c2e7fc640 1 -- 172.21.15.45:0/573286608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400f6b0 con 0x7f7c38069dc0 2024-04-25T00:30:44.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.850+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c381a6d30 con 0x7f7c38069dc0 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.852+0000 7f7c2e7fc640 1 -- 172.21.15.45:0/573286608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c340109d0 con 0x7f7c38069dc0 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 Processor -- start 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 -- start start 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c2e7fc640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 0x7f7c240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f496d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1f48d12c0 con 0x55d1f4978010 2024-04-25T00:30:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d1f4879910 con 0x55d1f496e980 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c4280e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d1f48cd280 con 0x55d1f496d290 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 0x7f7c240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.853+0000 7f7c2e7fc640 1 -- 172.21.15.45:0/573286608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f7c340cf3b0 con 0x7f7c38069dc0 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:58216/0 (socket says 172.21.15.45:58216) 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3dfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/2175086709 learned_addr learned my addr 172.21.15.45:0/2175086709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f496d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/2175086709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 msgr2=0x55d1f496d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f496d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/2175086709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 msgr2=0x55d1f497a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 -- 172.21.15.45:0/2175086709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f488afb0 con 0x55d1f496e980 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f496d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c34097a20 tx=0x7f7c3400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6545d38401951fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 0x7f7c240a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005e30 tx=0x7f7c3002f4b0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c237fe640 1 -- 172.21.15.45:0/2175086709 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34020070 con 0x55d1f496e980 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c237fe640 1 -- 172.21.15.45:0/2175086709 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34009de0 con 0x55d1f496e980 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.854+0000 7f7c237fe640 1 -- 172.21.15.45:0/2175086709 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3409ca70 con 0x55d1f496e980 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- 172.21.15.45:0/2175086709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 msgr2=0x55d1f496ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c34097a20 tx=0x7f7c3400bf60 comp rx=0 tx=0).stop 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- 172.21.15.45:0/2175086709 shutdown_connections 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f496d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 --2- 172.21.15.45:0/2175086709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f496ccc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- 172.21.15.45:0/2175086709 >> 172.21.15.45:0/2175086709 conn(0x55d1f4935950 msgr2=0x55d1f49697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- 172.21.15.45:0/2175086709 shutdown_connections 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- 172.21.15.45:0/2175086709 wait complete. 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 -- start start 2024-04-25T00:30:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.855+0000 7f7c4280e740 1 Processor -- start 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4974ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 0x55d1f4975430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1f497ade0 con 0x55d1f4978010 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d1f488ce90 con 0x55d1f496d290 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d1f497af60 con 0x55d1f496e980 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c4280e740 10 client.0 _ll_register_callbacks cb 0x55d1f48c8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59036/0 (socket says 172.21.15.45:59036) 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/4246874343 learned_addr learned my addr 172.21.15.45:0/4246874343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4974ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 0x55d1f4975430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/4246874343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 msgr2=0x55d1f4975430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 0x55d1f4975430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 msgr2=0x55d1f4974ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4974ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/4246874343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f46ad7e0 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 0x55d1f4975430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.856+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4974ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009ca0 tx=0x7f7c30004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ff19c491eebf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30051070 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f7c300307d0 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c34009de0 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c3005d050 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 10 client.18171 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1f470df70 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34009de0 con 0x55d1f4978010 2024-04-25T00:30:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c3004ba30 con 0x55d1f4978010 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 10 client.18171 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 20 client.18171 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f4988010 0x55d1f498a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 4 client.18171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.857+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1f48a55a0 con 0x55d1f4988010 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.858+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f4988010 0x55d1f498a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.858+0000 7f7c20ff9640 10 client.18171 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.858+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f4988010 0x55d1f498a4b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be91ec1241fe8584 server_cookie=2d00f2abfe74db2e in_seq=0 out_seq=0 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c20ff9640 1 -- 172.21.15.45:0/4246874343 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1f48a55a0 con 0x55d1f4988010 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c20ff9640 10 client.18171 handle_command_reply: tid=0 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 client.18171 shutdown 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 msgr2=0x55d1f4975970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009ca0 tx=0x7f7c30004a80 comp rx=0 tx=0).stop 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 shutdown_connections 2024-04-25T00:30:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f4988010 0x55d1f498a4b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f4975970 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496e980 0x55d1f4975430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c20ff9640 0 client.18171 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4246874343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4974ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 >> 172.21.15.45:0/4246874343 conn(0x55d1f4935950 msgr2=0x55d1f496f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 shutdown_connections 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 1 -- 172.21.15.45:0/4246874343 wait complete. 2024-04-25T00:30:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.859+0000 7f7c4280e740 20 client.18171 trim_cache size 0 max 16384 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.877+0000 7f7c4280e740 1 Processor -- start 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.877+0000 7f7c4280e740 1 -- start start 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4983ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 0x55d1f498fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b0e60 con 0x55d1f496e980 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b29a0 con 0x55d1f496d290 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c4280e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b1530 con 0x55d1f4978010 2024-04-25T00:30:44.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59044/0 (socket says 172.21.15.45:59044) 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 -- 172.21.15.45:0/4145191535 learned_addr learned my addr 172.21.15.45:0/4145191535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4983ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 0x55d1f498fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 -- 172.21.15.45:0/4145191535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 msgr2=0x55d1f498fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 0x55d1f498fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 -- 172.21.15.45:0/4145191535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 msgr2=0x55d1f4983ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4983ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 -- 172.21.15.45:0/4145191535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f45a5170 con 0x55d1f496e980 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 0x55d1f498fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7c28004580 tx=0x7f7c2800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e481e45905694941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c22ffd640 1 -- 172.21.15.45:0/4145191535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28003890 con 0x55d1f496e980 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c22ffd640 1 -- 172.21.15.45:0/4145191535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28011440 con 0x55d1f496e980 2024-04-25T00:30:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.878+0000 7f7c22ffd640 1 -- 172.21.15.45:0/4145191535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28010690 con 0x55d1f496e980 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- 172.21.15.45:0/4145191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 msgr2=0x55d1f4976e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7c28004580 tx=0x7f7c2800ba60 comp rx=0 tx=0).stop 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- 172.21.15.45:0/4145191535 shutdown_connections 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f4978010 0x55d1f498fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f496e980 0x55d1f4976e30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 --2- 172.21.15.45:0/4145191535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496d290 0x55d1f4983ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- 172.21.15.45:0/4145191535 >> 172.21.15.45:0/4145191535 conn(0x55d1f4935950 msgr2=0x55d1f4976720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- 172.21.15.45:0/4145191535 shutdown_connections 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- 172.21.15.45:0/4145191535 wait complete. 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.879+0000 7f7c4280e740 1 -- start start 2024-04-25T00:30:44.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 Processor -- start 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f49803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f497d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b22d0 con 0x55d1f4978010 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381b1c00 con 0x55d1f496e980 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c381aee60 con 0x55d1f496d290 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c4280e740 10 client.0 _ll_register_callbacks cb 0x7f7c30031a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59060/0 (socket says 172.21.15.45:59060) 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/719570755 learned_addr learned my addr 172.21.15.45:0/719570755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f49803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3cfc7640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f497d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/719570755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 msgr2=0x55d1f49803e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f49803e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 msgr2=0x55d1f497d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f497d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.880+0000 7f7c3dfc9640 1 -- 172.21.15.45:0/719570755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f497dc40 con 0x55d1f4978010 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c3dfc9640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005d50 tx=0x7f7c30031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134f9d53d285cbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30051070 con 0x55d1f4978010 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f7c30003b80 con 0x55d1f4978010 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c28011440 con 0x55d1f4978010 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c30003d80 con 0x55d1f4978010 2024-04-25T00:30:44.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 10 client.18177 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1f49ef260 con 0x55d1f4978010 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28011440 con 0x55d1f4978010 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c300307d0 con 0x55d1f4978010 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 10 client.18177 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 20 client.18177 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f49efa00 0x55d1f49f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 4 client.18177 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.881+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d1f48de800 con 0x55d1f49efa00 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.882+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f49efa00 0x55d1f49f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.882+0000 7f7c3d7c8640 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f49efa00 0x55d1f49f1e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4cbac61bc3abb7 server_cookie=4863ac6fee118d1 in_seq=0 out_seq=0 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.882+0000 7f7c20ff9640 10 client.18177 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c20ff9640 1 -- 172.21.15.45:0/719570755 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d1f48de800 con 0x55d1f49efa00 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c20ff9640 10 client.18177 handle_command_reply: tid=0 2024-04-25T00:30:44.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 client.18177 shutdown 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 msgr2=0x55d1f497d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005d50 tx=0x7f7c30031f40 comp rx=0 tx=0).stop 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 shutdown_connections 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d1f49efa00 0x55d1f49f1e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1f4978010 0x55d1f497d700 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d1f496e980 0x55d1f497d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c20ff9640 0 client.18177 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 --2- 172.21.15.45:0/719570755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d1f496d290 0x55d1f49803e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.883+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 >> 172.21.15.45:0/719570755 conn(0x55d1f4935950 msgr2=0x55d1f4973f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 shutdown_connections 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c4280e740 1 -- 172.21.15.45:0/719570755 wait complete. 2024-04-25T00:30:44.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c4280e740 20 client.18177 trim_cache size 0 max 16384 2024-04-25T00:30:44.890 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:44.890 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:30:44.890 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "6b0ff554-1850-4e6e-876a-57bcbc3ba8e2", 2024-04-25T00:30:44.890 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:30:44.890 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 msgr2=0x7f7c240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 0x7f7c240a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005e30 tx=0x7f7c3002f4b0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 msgr2=0x7f7c3819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d950 tx=0x7f7c3400de10 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 shutdown_connections 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7c240a2d00 0x7f7c240a51c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c3810e150 0x7f7c3819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c3806a6e0 0x7f7c3819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 --2- 172.21.15.45:0/573286608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c38069dc0 0x7f7c3819a210 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 >> 172.21.15.45:0/573286608 conn(0x7f7c380776a0 msgr2=0x7f7c3810f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 shutdown_connections 2024-04-25T00:30:44.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:44.884+0000 7f7c3fa2c640 1 -- 172.21.15.45:0/573286608 wait complete. 2024-04-25T00:30:44.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:30:44.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 -- 172.21.15.45:0/489365052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81020d0 msgr2=0x7f14e810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/489365052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81020d0 0x7f14e810e5e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009cd0 tx=0x7f14e002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 -- 172.21.15.45:0/489365052 shutdown_connections 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/489365052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e81020d0 0x7f14e810e5e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/489365052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e8101710 0x7f14e8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/489365052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8107720 0x7f14e8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 -- 172.21.15.45:0/489365052 >> 172.21.15.45:0/489365052 conn(0x7f14e80fd180 msgr2=0x7f14e80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 -- 172.21.15.45:0/489365052 shutdown_connections 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.000+0000 7f14ef3c8640 1 -- 172.21.15.45:0/489365052 wait complete. 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 Processor -- start 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 -- start start 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8101710 0x7f14e8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 0x7f14e81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111b30 con 0x7f14e8107720 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81119b0 con 0x7f14e8101710 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ef3c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8111cb0 con 0x7f14e81020d0 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ed965640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ed965640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59066/0 (socket says 172.21.15.45:59066) 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ed965640 1 -- 172.21.15.45:0/496919070 learned_addr learned my addr 172.21.15.45:0/496919070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ec963640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 0x7f14e81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.001+0000 7f14ed164640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8101710 0x7f14e8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 -- 172.21.15.45:0/496919070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 msgr2=0x7f14e81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 0x7f14e81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 -- 172.21.15.45:0/496919070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8101710 msgr2=0x7f14e8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8101710 0x7f14e8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 -- 172.21.15.45:0/496919070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819d060 con 0x7f14e8107720 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ec963640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 0x7f14e81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ed965640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005cc0 tx=0x7f14e0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14de7fc640 1 -- 172.21.15.45:0/496919070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004040 con 0x7f14e8107720 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14de7fc640 1 -- 172.21.15.45:0/496919070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e00046c0 con 0x7f14e8107720 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819d2f0 con 0x7f14e8107720 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14de7fc640 1 -- 172.21.15.45:0/496919070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0030e00 con 0x7f14e8107720 2024-04-25T00:30:45.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.002+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a4be0 con 0x7f14e8107720 2024-04-25T00:30:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.004+0000 7f14de7fc640 1 -- 172.21.15.45:0/496919070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14e00305a0 con 0x7f14e8107720 2024-04-25T00:30:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 Processor -- start 2024-04-25T00:30:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 -- start start 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6de9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6de9dd10 0x560e6deac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e6de026b0 con 0x560e6deaca30 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e6ddaa880 con 0x560e6de9dd10 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14f21b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e6ddfe670 con 0x560e6de9d360 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14de7fc640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 0x7f14d40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59082/0 (socket says 172.21.15.45:59082) 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 -- 172.21.15.45:0/1681314331 learned_addr learned my addr 172.21.15.45:0/1681314331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6de9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ed164640 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6de9dd10 0x560e6deac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ed965640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 0x7f14d40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 -- 172.21.15.45:0/1681314331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 msgr2=0x560e6de9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6de9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14de7fc640 1 -- 172.21.15.45:0/496919070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14e002e030 con 0x7f14e8107720 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 -- 172.21.15.45:0/1681314331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6de9dd10 msgr2=0x560e6deac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6de9dd10 0x560e6deac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.005+0000 7f14ec963640 1 -- 172.21.15.45:0/1681314331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6dbe07d0 con 0x560e6deaca30 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14ec963640 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f14d8012820 tx=0x7f14d8009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224fd4cdc4f95265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14ed965640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 0x7f14d40a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14e00bcf10 tx=0x7f14e00308d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14d37fe640 1 -- 172.21.15.45:0/1681314331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8012e60 con 0x560e6deaca30 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14d37fe640 1 -- 172.21.15.45:0/1681314331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8010cb0 con 0x560e6deaca30 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14d37fe640 1 -- 172.21.15.45:0/1681314331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d801ebf0 con 0x560e6deaca30 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14f21b1740 1 -- 172.21.15.45:0/1681314331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 msgr2=0x560e6deaee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f14d8012820 tx=0x7f14d8009720 comp rx=0 tx=0).stop 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14f21b1740 1 -- 172.21.15.45:0/1681314331 shutdown_connections 2024-04-25T00:30:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.006+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6deaca30 0x560e6deaee20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6de9dd10 0x560e6deac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1681314331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6de9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- 172.21.15.45:0/1681314331 >> 172.21.15.45:0/1681314331 conn(0x560e6de99950 msgr2=0x560e6de99d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- 172.21.15.45:0/1681314331 shutdown_connections 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- 172.21.15.45:0/1681314331 wait complete. 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- start start 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 Processor -- start 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6dea9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deaa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560e6deaf7f0 con 0x560e6de9d360 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e6ddbdf50 con 0x560e6deaca30 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e6deaf970 con 0x560e6de9dd10 2024-04-25T00:30:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14f21b1740 10 client.0 _ll_register_callbacks cb 0x560e6ddf9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59092/0 (socket says 172.21.15.45:59092) 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.007+0000 7f14ec963640 1 -- 172.21.15.45:0/446477386 learned_addr learned my addr 172.21.15.45:0/446477386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ed164640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6dea9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deaa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 -- 172.21.15.45:0/446477386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 msgr2=0x560e6dea9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6dea9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 -- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 msgr2=0x560e6deaa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deaa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 -- 172.21.15.45:0/446477386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6db163f0 con 0x560e6de9d360 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14ec963640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009d90 tx=0x7f14d801e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43326b6239877bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8012e60 con 0x560e6de9d360 2024-04-25T00:30:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14d801eb80 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8010cb0 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d801e3c0 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14f21b1740 10 client.18189 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e6dad60f0 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.008+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8010cb0 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14d8004750 con 0x560e6de9d360 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 10 client.18189 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 20 client.18189 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6debcc40 0x560e6debf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 4 client.18189 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6de0fbf0 con 0x560e6debcc40 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14ed164640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6debcc40 0x560e6debf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14d0ff9640 10 client.18189 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.009+0000 7f14ed164640 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6debcc40 0x560e6debf030 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94b9576bd58eb40 server_cookie=1d4d873424995263 in_seq=0 out_seq=0 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14d0ff9640 1 -- 172.21.15.45:0/446477386 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e6de0fbf0 con 0x560e6debcc40 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14d0ff9640 10 client.18189 handle_command_reply: tid=0 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 client.18189 shutdown 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 msgr2=0x560e6dea9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009d90 tx=0x7f14d801e7d0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 shutdown_connections 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deaa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6dea9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6debcc40 0x560e6debf030 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 --2- 172.21.15.45:0/446477386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6dea9990 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14d0ff9640 0 client.18189 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.010+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 >> 172.21.15.45:0/446477386 conn(0x560e6de99950 msgr2=0x560e6de9f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.011+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 shutdown_connections 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.011+0000 7f14f21b1740 1 -- 172.21.15.45:0/446477386 wait complete. 2024-04-25T00:30:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.011+0000 7f14f21b1740 20 client.18189 trim_cache size 0 max 16384 2024-04-25T00:30:45.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.030+0000 7f14f21b1740 1 Processor -- start 2024-04-25T00:30:45.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.030+0000 7f14f21b1740 1 -- start start 2024-04-25T00:30:45.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6deb8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6de99d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81ac970 con 0x560e6de9dd10 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81ae4b0 con 0x560e6deaca30 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14f21b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81ad040 con 0x560e6de9d360 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59106/0 (socket says 172.21.15.45:59106) 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 -- 172.21.15.45:0/1743947772 learned_addr learned my addr 172.21.15.45:0/1743947772 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ec963640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6de99d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ec963640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6deb8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 -- 172.21.15.45:0/1743947772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 msgr2=0x560e6deb8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6deb8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 -- 172.21.15.45:0/1743947772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 msgr2=0x560e6de99d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6de99d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 -- 172.21.15.45:0/1743947772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6de99780 con 0x560e6de9dd10 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ec963640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6deb8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14ed164640 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005d00 tx=0x7f14e400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d9b099764d5a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14d2ffd640 1 -- 172.21.15.45:0/1743947772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4023070 con 0x560e6de9dd10 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14d2ffd640 1 -- 172.21.15.45:0/1743947772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e401d910 con 0x560e6de9dd10 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.031+0000 7f14d2ffd640 1 -- 172.21.15.45:0/1743947772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011da0 con 0x560e6de9dd10 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- 172.21.15.45:0/1743947772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 msgr2=0x560e6deac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005d00 tx=0x7f14e400dda0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- 172.21.15.45:0/1743947772 shutdown_connections 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6de99d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9dd10 0x560e6deac050 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 --2- 172.21.15.45:0/1743947772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9d360 0x560e6deb8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- 172.21.15.45:0/1743947772 >> 172.21.15.45:0/1743947772 conn(0x560e6de99950 msgr2=0x560e6de98260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- 172.21.15.45:0/1743947772 shutdown_connections 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- 172.21.15.45:0/1743947772 wait complete. 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.032+0000 7f14f21b1740 1 -- start start 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 Processor -- start 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6deb51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deb0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81adde0 con 0x560e6de9d360 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81ad710 con 0x560e6deaca30 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81aa970 con 0x560e6de9dd10 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14f21b1740 10 client.0 _ll_register_callbacks cb 0x7f14e400d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59120/0 (socket says 172.21.15.45:59120) 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 -- 172.21.15.45:0/3478361766 learned_addr learned my addr 172.21.15.45:0/3478361766 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:30:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ed164640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6deb51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 -- 172.21.15.45:0/3478361766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 msgr2=0x560e6deb51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6deb51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 -- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 msgr2=0x560e6deb0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deb0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 -- 172.21.15.45:0/3478361766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6deb0c20 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14ec963640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004ce0 tx=0x7f14d801f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e104d41e83f6b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.033+0000 7f14d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d803c060 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9475+0+0 (secure 0 0 0) 0x7f14d80095c0 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e401d910 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d8004d60 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 10 client.18195 fetch_fsmap learned FSMap version 28 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e6df22c80 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e401d910 con 0x560e6de9d360 2024-04-25T00:30:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14d8010b80 con 0x560e6de9d360 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 10 client.18195 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 20 client.18195 populate_metadata read hostname 'smithi045' 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6df23310 0x560e6df25730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 4 client.18195 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e6de09f40 con 0x560e6df23310 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.034+0000 7f14ed164640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6df23310 0x560e6df25730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14d0ff9640 10 client.18195 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14ed164640 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6df23310 0x560e6df25730 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8806453087cee5 server_cookie=a9669bd8796d5ca in_seq=0 out_seq=0 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14d0ff9640 1 -- 172.21.15.45:0/3478361766 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e6de09f40 con 0x560e6df23310 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14d0ff9640 10 client.18195 handle_command_reply: tid=0 2024-04-25T00:30:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 client.18195 shutdown 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 msgr2=0x560e6deb4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004ce0 tx=0x7f14d801f420 comp rx=0 tx=0).stop 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 shutdown_connections 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e6deaca30 0x560e6deb0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e6de9dd10 0x560e6deb51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560e6df23310 0x560e6df25730 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 --2- 172.21.15.45:0/3478361766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560e6de9d360 0x560e6deb4c70 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14d0ff9640 0 client.18195 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 >> 172.21.15.45:0/3478361766 conn(0x560e6de99950 msgr2=0x560e6deaa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 shutdown_connections 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 1 -- 172.21.15.45:0/3478361766 wait complete. 2024-04-25T00:30:45.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.035+0000 7f14f21b1740 20 client.18195 trim_cache size 0 max 16384 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "6b0ff554-1850-4e6e-876a-57bcbc3ba8e2": { 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "6b0ff554-1850-4e6e-876a-57bcbc3ba8e2", 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:30:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 msgr2=0x7f14d40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 0x7f14d40a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14e00bcf10 tx=0x7f14e00308d0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 msgr2=0x7f14e819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f14e0005cc0 tx=0x7f14e0032f40 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 shutdown_connections 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14d40a2d00 0x7f14d40a51c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14e8107720 0x7f14e819c930 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14e81020d0 0x7f14e81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 --2- 172.21.15.45:0/496919070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14e8101710 0x7f14e8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 >> 172.21.15.45:0/496919070 conn(0x7f14e80fd180 msgr2=0x7f14e80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 shutdown_connections 2024-04-25T00:30:45.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:30:45.036+0000 7f14ef3c8640 1 -- 172.21.15.45:0/496919070 wait complete. 2024-04-25T00:30:45.047 INFO:tasks.cephfs.filesystem:scrub status for tag:6b0ff554-1850-4e6e-876a-57bcbc3ba8e2 - {'path': '/', 'tag': '6b0ff554-1850-4e6e-876a-57bcbc3ba8e2', 'options': 'recursive,force'} 2024-04-25T00:30:45.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:45 smithi122 ceph-mon[32959]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:45.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:45 smithi071 ceph-mon[32667]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:45 smithi045 ceph-mon[29194]: pgmap v270: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:30:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:46 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:30:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:46 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:46 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:30:46.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:46 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:46.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:46 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:30:46.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:46 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:46.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:46 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:30:46.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:46 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:46 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:30:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:46 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:46 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:30:46.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:46 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:30:47.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:47 smithi122 ceph-mon[32959]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:47.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:47 smithi071 ceph-mon[32667]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:47 smithi045 ceph-mon[29194]: pgmap v271: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:30:48.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:48 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:30:48.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:48 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:30:48.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:48 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:30:49.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:49 smithi122 ceph-mon[32959]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:49.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:49 smithi071 ceph-mon[32667]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:49 smithi045 ceph-mon[29194]: pgmap v272: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:30:51.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:51 smithi122 ceph-mon[32959]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:30:51.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:51 smithi071 ceph-mon[32667]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:30:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:51 smithi045 ceph-mon[29194]: pgmap v273: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:30:53.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:53 smithi122 ceph-mon[32959]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:30:53.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:53 smithi071 ceph-mon[32667]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:30:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:53 smithi045 ceph-mon[29194]: pgmap v274: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:30:55.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:55 smithi122 ceph-mon[32959]: pgmap v275: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:30:55.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:55 smithi071 ceph-mon[32667]: pgmap v275: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:30:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:55 smithi045 ceph-mon[29194]: pgmap v275: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:30:57.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:57 smithi122 ceph-mon[32959]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:57.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:57 smithi071 ceph-mon[32667]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:57.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:57 smithi045 ceph-mon[29194]: pgmap v276: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:30:59.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:30:59 smithi122 ceph-mon[32959]: pgmap v277: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:30:59.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:30:59 smithi071 ceph-mon[32667]: pgmap v277: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:30:59.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:30:59 smithi045 ceph-mon[29194]: pgmap v277: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:01.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:01 smithi071 ceph-mon[32667]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:31:01.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:01 smithi045 ceph-mon[29194]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:31:01.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:01 smithi122 ceph-mon[32959]: pgmap v278: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:31:03.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:03 smithi071 ceph-mon[32667]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:03.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:03 smithi045 ceph-mon[29194]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:03.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:03 smithi122 ceph-mon[32959]: pgmap v279: 129 pgs: 129 active+clean; 2.1 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:05.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:05 smithi071 ceph-mon[32667]: pgmap v280: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:31:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:05 smithi045 ceph-mon[29194]: pgmap v280: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:31:05.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:05 smithi122 ceph-mon[32959]: pgmap v280: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:31:07.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:07 smithi071 ceph-mon[32667]: pgmap v281: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:31:07.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:07 smithi045 ceph-mon[29194]: pgmap v281: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:31:07.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:07 smithi122 ceph-mon[32959]: pgmap v281: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:31:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:09 smithi071 ceph-mon[32667]: pgmap v282: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:31:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:09 smithi071 ceph-mon[32667]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:31:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:09 smithi045 ceph-mon[29194]: pgmap v282: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:31:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:09 smithi045 ceph-mon[29194]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:31:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:09 smithi122 ceph-mon[32959]: pgmap v282: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:31:09.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:09 smithi122 ceph-mon[32959]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:31:10.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:10 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:10.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:10 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:10.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:10 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:11.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:11 smithi071 ceph-mon[32667]: pgmap v284: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:11 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:11 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:11.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:11 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:31:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:11 smithi045 ceph-mon[29194]: pgmap v284: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:11 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:11 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:11.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:11 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:31:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:11 smithi122 ceph-mon[32959]: pgmap v284: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:11 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:11 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:11.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:11 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:31:13.358 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:13 smithi122 ceph-mon[32959]: pgmap v285: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:13 smithi071 ceph-mon[32667]: pgmap v285: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:13 smithi045 ceph-mon[29194]: pgmap v285: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:15.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:31:15.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 -- 172.21.15.45:0/3860611345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 msgr2=0x7f40b8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 --2- 172.21.15.45:0/3860611345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8100a90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009930 tx=0x7f40a002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 -- 172.21.15.45:0/3860611345 shutdown_connections 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 --2- 172.21.15.45:0/3860611345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b8100fd0 0x7f40b810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 --2- 172.21.15.45:0/3860611345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8100a90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 --2- 172.21.15.45:0/3860611345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b81066b0 0x7f40b8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 -- 172.21.15.45:0/3860611345 >> 172.21.15.45:0/3860611345 conn(0x7f40b8077740 msgr2=0x7f40b8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 -- 172.21.15.45:0/3860611345 shutdown_connections 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 -- 172.21.15.45:0/3860611345 wait complete. 2024-04-25T00:31:15.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.170+0000 7f40be109640 1 Processor -- start 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 -- start start 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 0x7f40b81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b81066b0 0x7f40b819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8110a30 con 0x7f40b8100610 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81108b0 con 0x7f40b8100fd0 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40be109640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8110bb0 con 0x7f40b81066b0 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57636/0 (socket says 172.21.15.45:57636) 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 -- 172.21.15.45:0/2402691517 learned_addr learned my addr 172.21.15.45:0/2402691517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b7fff640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b81066b0 0x7f40b819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 0x7f40b81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 -- 172.21.15.45:0/2402691517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b81066b0 msgr2=0x7f40b819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b81066b0 0x7f40b819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 -- 172.21.15.45:0/2402691517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 msgr2=0x7f40b81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 0x7f40b81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 -- 172.21.15.45:0/2402691517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b819cfa0 con 0x7f40b8100610 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 0x7f40b81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.171+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f40a800ebc0 tx=0x7f40a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.172+0000 7f40b4ff9640 1 -- 172.21.15.45:0/2402691517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003b70 con 0x7f40b8100610 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.172+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b819d290 con 0x7f40b8100610 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.172+0000 7f40b4ff9640 1 -- 172.21.15.45:0/2402691517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a8003d10 con 0x7f40b8100610 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.172+0000 7f40b4ff9640 1 -- 172.21.15.45:0/2402691517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8019980 con 0x7f40b8100610 2024-04-25T00:31:15.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.172+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a4b20 con 0x7f40b8100610 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b4ff9640 1 -- 172.21.15.45:0/2402691517 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f40a8019b20 con 0x7f40b8100610 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 Processor -- start 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 -- start start 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e00814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0083e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5572dffd76b0 con 0x5572e0072360 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5572dff7f880 con 0x5572e0081a30 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572dffd3670 con 0x5572e0072d10 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57642/0 (socket says 172.21.15.45:57642) 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b7fff640 1 -- 172.21.15.45:0/1011612957 learned_addr learned my addr 172.21.15.45:0/1011612957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b77fe640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e00814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b4ff9640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 0x7f40a40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.175+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0083e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 -- 172.21.15.45:0/1011612957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 msgr2=0x5572e00814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e00814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 -- 172.21.15.45:0/1011612957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 msgr2=0x5572e0083e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0083e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 -- 172.21.15.45:0/1011612957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dfdb57d0 con 0x5572e0072360 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b77fe640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e00814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 0x7f40a40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b4ff9640 1 -- 172.21.15.45:0/2402691517 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f40a80d0120 con 0x7f40b8100610 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009e20 tx=0x7f40b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd71d48602b5123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40adffb640 1 -- 172.21.15.45:0/1011612957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0017050 con 0x5572e0072360 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40adffb640 1 -- 172.21.15.45:0/1011612957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0003c40 con 0x5572e0072360 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40adffb640 1 -- 172.21.15.45:0/1011612957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000f620 con 0x5572e0072360 2024-04-25T00:31:15.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.176+0000 7f40b7fff640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 0x7f40a40a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f40b0023000 tx=0x7f40b0024000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- 172.21.15.45:0/1011612957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 msgr2=0x5572e0072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009e20 tx=0x7f40b0011810 comp rx=0 tx=0).stop 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- 172.21.15.45:0/1011612957 shutdown_connections 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0083e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e00814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/1011612957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e0072740 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- 172.21.15.45:0/1011612957 >> 172.21.15.45:0/1011612957 conn(0x5572e006e950 msgr2=0x5572e006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- 172.21.15.45:0/1011612957 shutdown_connections 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- 172.21.15.45:0/1011612957 wait complete. 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 -- start start 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.177+0000 7f40c0e0f740 1 Processor -- start 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e007e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e007f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5572e00847f0 con 0x5572e0072d10 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5572dff92f50 con 0x5572e0081a30 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572e0084970 con 0x5572e0072360 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 10 client.0 _ll_register_callbacks cb 0x5572dffce000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57656/0 (socket says 172.21.15.45:57656) 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 -- 172.21.15.45:0/3339444391 learned_addr learned my addr 172.21.15.45:0/3339444391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b7fff640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e007f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b77fe640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e007e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 -- 172.21.15.45:0/3339444391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 msgr2=0x5572e007e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e007e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 -- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 msgr2=0x5572e007f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e007f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 -- 172.21.15.45:0/3339444391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dfceb3f0 con 0x5572e0072d10 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c760880b878892d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f4096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0035030 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0003c40 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.178+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f40a0004460 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40a004b050 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 10 client.18207 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572dfcab0f0 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0003c40 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40a0004a60 con 0x5572e0072d10 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 10 client.18207 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 20 client.18207 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e0091d50 0x5572e00941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 4 client.18207 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572dffe4bf0 con 0x5572e0091d50 2024-04-25T00:31:15.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.179+0000 7f40b77fe640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e0091d50 0x5572e00941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.180+0000 7f40b77fe640 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e0091d50 0x5572e00941b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559885afca811346 server_cookie=675d4ab09e319aca in_seq=0 out_seq=0 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.180+0000 7f4096ffd640 10 client.18207 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f4096ffd640 1 -- 172.21.15.45:0/3339444391 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572dffe4bf0 con 0x5572e0091d50 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f4096ffd640 10 client.18207 handle_command_reply: tid=0 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 client.18207 shutdown 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 msgr2=0x5572e007ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0031860 comp rx=0 tx=0).stop 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 shutdown_connections 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e007f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e0091d50 0x5572e00941b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072d10 0x5572e007ee80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3339444391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e007e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 >> 172.21.15.45:0/3339444391 conn(0x5572e006e950 msgr2=0x5572e00747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f4096ffd640 0 client.18207 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 shutdown_connections 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3339444391 wait complete. 2024-04-25T00:31:15.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.181+0000 7f40c0e0f740 20 client.18207 trim_cache size 0 max 16384 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 Processor -- start 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 -- start start 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e008d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0072d10 0x5572e0074870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81acb60 con 0x5572e0081a30 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81ae6a0 con 0x5572e0072d10 2024-04-25T00:31:15.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81ad230 con 0x5572e0072360 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57668/0 (socket says 172.21.15.45:57668) 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 -- 172.21.15.45:0/3703795691 learned_addr learned my addr 172.21.15.45:0/3703795691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0072d10 0x5572e0074870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b77fe640 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e008d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 -- 172.21.15.45:0/3703795691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 msgr2=0x5572e008d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e008d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 -- 172.21.15.45:0/3703795691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0072d10 msgr2=0x5572e0074870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0072d10 0x5572e0074870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.199+0000 7f40b7fff640 1 -- 172.21.15.45:0/3703795691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e006e780 con 0x5572e0081a30 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40b7fff640 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009dc0 tx=0x7f40b001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de0f1285e6e2191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40ad7fa640 1 -- 172.21.15.45:0/3703795691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0037070 con 0x5572e0081a30 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40ad7fa640 1 -- 172.21.15.45:0/3703795691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0009780 con 0x5572e0081a30 2024-04-25T00:31:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40ad7fa640 1 -- 172.21.15.45:0/3703795691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000fbb0 con 0x5572e0081a30 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3703795691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 msgr2=0x5572e00808f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.200+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009dc0 tx=0x7f40b001cc70 comp rx=0 tx=0).stop 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3703795691 shutdown_connections 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0081a30 0x5572e00808f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0072d10 0x5572e0074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/3703795691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072360 0x5572e008d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3703795691 >> 172.21.15.45:0/3703795691 conn(0x5572e006e950 msgr2=0x5572e007d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3703795691 shutdown_connections 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- 172.21.15.45:0/3703795691 wait complete. 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- start start 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 Processor -- start 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e0085d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0086270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81adfd0 con 0x5572e0072360 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81ad900 con 0x5572e0081a30 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81aab60 con 0x5572e0072d10 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40c0e0f740 10 client.0 _ll_register_callbacks cb 0x7f40a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.201+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57682/0 (socket says 172.21.15.45:57682) 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 -- 172.21.15.45:0/2380594931 learned_addr learned my addr 172.21.15.45:0/2380594931 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b7fff640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0086270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e0085d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 -- 172.21.15.45:0/2380594931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 msgr2=0x5572e0085d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e0085d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 -- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 msgr2=0x5572e0086270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0086270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 -- 172.21.15.45:0/2380594931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e00796c0 con 0x5572e0072360 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40b77fe640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f40a809c1c0 tx=0x7f40a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2debf9c18704e8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f4096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a809c230 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f40a80988a0 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0009780 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40a80a4ea0 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40c0e0f740 10 client.18213 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572e00f85a0 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.202+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0009780 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f40a80a01b0 con 0x5572e0072360 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 10 client.18213 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 20 client.18213 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e00f8c20 0x5572e00fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 4 client.18213 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572dffdef40 con 0x5572e00f8c20 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e00f8c20 0x5572e00fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f40b6ffd640 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e00f8c20 0x5572e00fb010 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187bbc28e50db99e server_cookie=a053e0b2443cbb19 in_seq=0 out_seq=0 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.203+0000 7f4096ffd640 10 client.18213 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f4096ffd640 1 -- 172.21.15.45:0/2380594931 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5572dffdef40 con 0x5572e00f8c20 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f4096ffd640 10 client.18213 handle_command_reply: tid=0 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 client.18213 shutdown 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 msgr2=0x5572e00857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f40a809c1c0 tx=0x7f40a800cf90 comp rx=0 tx=0).stop 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 shutdown_connections 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572e0081a30 0x5572e0086270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572e0072d10 0x5572e0085d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5572e00f8c20 0x5572e00fb010 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 --2- 172.21.15.45:0/2380594931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5572e0072360 0x5572e00857f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f4096ffd640 0 client.18213 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 >> 172.21.15.45:0/2380594931 conn(0x5572e006e950 msgr2=0x5572e006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 shutdown_connections 2024-04-25T00:31:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 1 -- 172.21.15.45:0/2380594931 wait complete. 2024-04-25T00:31:15.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40c0e0f740 20 client.18213 trim_cache size 0 max 16384 2024-04-25T00:31:15.215 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:31:15.215 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:31:15.215 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:15.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 msgr2=0x7f40a40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 0x7f40a40a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f40b0023000 tx=0x7f40b0024000 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 msgr2=0x7f40b8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.204+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f40a800ebc0 tx=0x7f40a800c6a0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 shutdown_connections 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f40a40a2d00 0x7f40a40a51c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40b81066b0 0x7f40b819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40b8100fd0 0x7f40b81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 --2- 172.21.15.45:0/2402691517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40b8100610 0x7f40b8197fb0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 >> 172.21.15.45:0/2402691517 conn(0x7f40b8077740 msgr2=0x7f40b80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 shutdown_connections 2024-04-25T00:31:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.205+0000 7f40be109640 1 -- 172.21.15.45:0/2402691517 wait complete. 2024-04-25T00:31:15.218 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:31:15.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.319+0000 7f86377ce640 1 -- 172.21.15.45:0/3604119316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 msgr2=0x7f8630106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.319+0000 7f86377ce640 1 --2- 172.21.15.45:0/3604119316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f8630106a10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f8620009930 tx=0x7f862002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.319+0000 7f86377ce640 1 -- 172.21.15.45:0/3604119316 shutdown_connections 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- 172.21.15.45:0/3604119316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 0x7f863010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- 172.21.15.45:0/3604119316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- 172.21.15.45:0/3604119316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f8630106a10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- 172.21.15.45:0/3604119316 >> 172.21.15.45:0/3604119316 conn(0x7f8630077740 msgr2=0x7f8630077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- 172.21.15.45:0/3604119316 shutdown_connections 2024-04-25T00:31:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- 172.21.15.45:0/3604119316 wait complete. 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 Processor -- start 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- start start 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 0x7f8630197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8630110a40 con 0x7f8630106630 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86301108c0 con 0x7f8630100620 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.320+0000 7f86377ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8630110bc0 con 0x7f8630100fe0 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57708/0 (socket says 172.21.15.45:57708) 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 -- 172.21.15.45:0/3567921560 learned_addr learned my addr 172.21.15.45:0/3567921560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8634d69640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 0x7f8630197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f863556a640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 -- 172.21.15.45:0/3567921560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 msgr2=0x7f8630197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 0x7f8630197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 -- 172.21.15.45:0/3567921560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 msgr2=0x7f8630196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 -- 172.21.15.45:0/3567921560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863019bea0 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f863556a640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f8635d6b640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f862800cad0 tx=0x7f862800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862800ad60 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8628005020 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862800f6b0 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019c190 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.321+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a3a20 con 0x7f8630106630 2024-04-25T00:31:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.322+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8630068560 con 0x7f8630106630 2024-04-25T00:31:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.323+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8628005560 con 0x7f8630106630 2024-04-25T00:31:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.324+0000 7f86267fc640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 0x7f861c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.324+0000 7f863556a640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 0x7f861c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.324+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f86280cfb50 con 0x7f8630106630 2024-04-25T00:31:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.325+0000 7f863556a640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 0x7f861c0a5240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8620000c00 tx=0x7f8620005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.328+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8628014030 con 0x7f8630106630 2024-04-25T00:31:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:15 smithi071 ceph-mon[32667]: pgmap v286: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:15 smithi045 ceph-mon[29194]: pgmap v286: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.551+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863002d170 con 0x7f8630106630 2024-04-25T00:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.552+0000 7f86267fc640 1 -- 172.21.15.45:0/3567921560 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f862809c3e0 con 0x7f8630106630 2024-04-25T00:31:15.552 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:31:15.553 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:31:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 msgr2=0x7f861c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 0x7f861c0a5240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8620000c00 tx=0x7f8620005a20 comp rx=0 tx=0).stop 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 msgr2=0x7f863019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f862800cad0 tx=0x7f862800ced0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 shutdown_connections 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f861c0a2d80 0x7f861c0a5240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8630106630 0x7f863019b770 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8630100fe0 0x7f8630197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 --2- 172.21.15.45:0/3567921560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630100620 0x7f8630196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 >> 172.21.15.45:0/3567921560 conn(0x7f8630077740 msgr2=0x7f8630102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 shutdown_connections 2024-04-25T00:31:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.554+0000 7f86377ce640 1 -- 172.21.15.45:0/3567921560 wait complete. 2024-04-25T00:31:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:15 smithi122 ceph-mon[32959]: pgmap v286: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T00:31:15.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:31:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/2681051370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3806a780 msgr2=0x7f8a3810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/2681051370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3806a780 0x7f8a3810e5f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009cd0 tx=0x7f8a3002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/2681051370 shutdown_connections 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/2681051370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3806a780 0x7f8a3810e5f0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/2681051370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38069dc0 0x7f8a3806a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/2681051370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a38107730 0x7f8a38107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/2681051370 >> 172.21.15.45:0/2681051370 conn(0x7f8a380776a0 msgr2=0x7f8a38077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/2681051370 shutdown_connections 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/2681051370 wait complete. 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.669+0000 7f8a3fe4d640 1 Processor -- start 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 -- start start 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3806a780 0x7f8a38198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38107730 0x7f8a3819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38111b40 con 0x7f8a38069dc0 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381119c0 con 0x7f8a3806a780 2024-04-25T00:31:15.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3fe4d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38111cc0 con 0x7f8a38107730 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57732/0 (socket says 172.21.15.45:57732) 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1539001261 learned_addr learned my addr 172.21.15.45:0/1539001261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3806a780 0x7f8a38198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38107730 0x7f8a3819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1539001261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38107730 msgr2=0x7f8a3819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38107730 0x7f8a3819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1539001261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3806a780 msgr2=0x7f8a38198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3806a780 0x7f8a38198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1539001261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3819cfd0 con 0x7f8a38069dc0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.670+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9f0 tx=0x7f8a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.671+0000 7f8a2effd640 1 -- 172.21.15.45:0/1539001261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400cd80 con 0x7f8a38069dc0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.671+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3819d2c0 con 0x7f8a38069dc0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.671+0000 7f8a2effd640 1 -- 172.21.15.45:0/1539001261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a34010910 con 0x7f8a38069dc0 2024-04-25T00:31:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.671+0000 7f8a2effd640 1 -- 172.21.15.45:0/1539001261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340188f0 con 0x7f8a38069dc0 2024-04-25T00:31:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.671+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a4b50 con 0x7f8a38069dc0 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a2effd640 1 -- 172.21.15.45:0/1539001261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8a34010ac0 con 0x7f8a38069dc0 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 Processor -- start 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 -- start start 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec059960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5628ebfb1670 con 0x5628ec057570 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5628ebf59970 con 0x5628ec04cea0 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628ebfad630 con 0x5628ec04c580 2024-04-25T00:31:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a3d3e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec059960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a3e3ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a3e3ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35268/0 (socket says 172.21.15.45:35268) 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.674+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3731923670 learned_addr learned my addr 172.21.15.45:0/3731923670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a2effd640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 0x7f8a240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3731923670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 msgr2=0x5628ec057030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec057030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3731923670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 msgr2=0x5628ec059960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec059960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3731923670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ebf7d400 con 0x5628ec04c580 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a2effd640 1 -- 172.21.15.45:0/1539001261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f8a340cfce0 con 0x7f8a38069dc0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 0x7f8a240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009a60 tx=0x7f8a30005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5ca49162fe2bdbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a23fff640 1 -- 172.21.15.45:0/3731923670 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300315b0 con 0x5628ec04c580 2024-04-25T00:31:15.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 0x7f8a240a51c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009430 tx=0x7f8a30048000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a23fff640 1 -- 172.21.15.45:0/3731923670 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30036040 con 0x5628ec04c580 2024-04-25T00:31:15.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.675+0000 7f8a23fff640 1 -- 172.21.15.45:0/3731923670 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300307e0 con 0x5628ec04c580 2024-04-25T00:31:15.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3731923670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 msgr2=0x5628ec04c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009a60 tx=0x7f8a30005f70 comp rx=0 tx=0).stop 2024-04-25T00:31:15.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3731923670 shutdown_connections 2024-04-25T00:31:15.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec057030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3731923670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec04c960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3731923670 >> 172.21.15.45:0/3731923670 conn(0x5628ec0488a0 msgr2=0x5628ec048c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3731923670 shutdown_connections 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3731923670 wait complete. 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.676+0000 7f8a42c3a740 1 -- start start 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 Processor -- start 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec054200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec0547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5628ec05a360 con 0x5628ec04c580 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5628ebf6d050 con 0x5628ec057570 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628ec05a4e0 con 0x5628ec04cea0 2024-04-25T00:31:15.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a42c3a740 10 client.0 _ll_register_callbacks cb 0x5628ebcc4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57738/0 (socket says 172.21.15.45:57738) 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1216356670 learned_addr learned my addr 172.21.15.45:0/1216356670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec0547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec054200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 msgr2=0x5628ec054200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec054200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1216356670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 msgr2=0x5628ec0547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec0547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/1216356670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ebf3edc0 con 0x5628ec04c580 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec0547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cd620 tx=0x7f8a34009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d137db1a8f474c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.677+0000 7f8a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3409b750 con 0x5628ec04c580 2024-04-25T00:31:15.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f8a34018d50 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a30036040 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a3409c4f0 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 10 client.18231 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628ebcc53f0 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30036040 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3409b360 con 0x5628ec04c580 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 10 client.18231 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 10 client.18231 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 20 client.18231 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0676d0 0x5628ec069b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 4 client.18231 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628ebfbebb0 con 0x5628ec0676d0 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.678+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0676d0 0x5628ec069b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.679+0000 7f8a217fa640 10 client.18231 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:15.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.679+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0676d0 0x5628ec069b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc41dc121ea9e325 server_cookie=4ababb12a7b5de19 in_seq=0 out_seq=0 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a217fa640 1 -- 172.21.15.45:0/1216356670 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628ebfbebb0 con 0x5628ec0676d0 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a217fa640 10 client.18231 handle_command_reply: tid=0 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 client.18231 shutdown 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 msgr2=0x5628ec056f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cd620 tx=0x7f8a34009310 comp rx=0 tx=0).stop 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 shutdown_connections 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0676d0 0x5628ec069b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec0547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec054200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a217fa640 0 client.18231 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/1216356670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec056f10 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 >> 172.21.15.45:0/1216356670 conn(0x5628ec0488a0 msgr2=0x5628ec04e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 shutdown_connections 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 1 -- 172.21.15.45:0/1216356670 wait complete. 2024-04-25T00:31:15.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.680+0000 7f8a42c3a740 20 client.18231 trim_cache size 0 max 16384 2024-04-25T00:31:15.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.693+0000 7f8a42c3a740 1 Processor -- start 2024-04-25T00:31:15.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.693+0000 7f8a42c3a740 1 -- start start 2024-04-25T00:31:15.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.693+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.693+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec056a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec06f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381aecf0 con 0x5628ec04c580 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381b0830 con 0x5628ec057570 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381af3c0 con 0x5628ec04cea0 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57740/0 (socket says 172.21.15.45:57740) 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/3592816890 learned_addr learned my addr 172.21.15.45:0/3592816890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec056a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/3592816890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 msgr2=0x5628ec056a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec056a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/3592816890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 msgr2=0x5628ec06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec06f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 -- 172.21.15.45:0/3592816890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ebd8f7d0 con 0x5628ec04c580 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8a3409baf0 tx=0x7f8a34009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c86fbd07216006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a237fe640 1 -- 172.21.15.45:0/3592816890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3409bbc0 con 0x5628ec04c580 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a237fe640 1 -- 172.21.15.45:0/3592816890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a340a01a0 con 0x5628ec04c580 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.694+0000 7f8a237fe640 1 -- 172.21.15.45:0/3592816890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3409ad80 con 0x5628ec04c580 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3592816890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 msgr2=0x5628ec063250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8a3409baf0 tx=0x7f8a34009700 comp rx=0 tx=0).stop 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3592816890 shutdown_connections 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec057570 0x5628ec06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04cea0 0x5628ec056a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3592816890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec04c580 0x5628ec063250 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3592816890 >> 172.21.15.45:0/3592816890 conn(0x5628ec0488a0 msgr2=0x5628ec04f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3592816890 shutdown_connections 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3592816890 wait complete. 2024-04-25T00:31:15.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 -- start start 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.695+0000 7f8a42c3a740 1 Processor -- start 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec05b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec05b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381b0160 con 0x5628ec057570 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381afa90 con 0x5628ec04cea0 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381accf0 con 0x5628ec04c580 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 10 client.0 _ll_register_callbacks cb 0x7f8a340d3520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57744/0 (socket says 172.21.15.45:57744) 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3102144743 learned_addr learned my addr 172.21.15.45:0/3102144743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3d3e8640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec05b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec05b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 msgr2=0x5628ec05b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec05b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3102144743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 msgr2=0x5628ec05b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec05b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 -- 172.21.15.45:0/3102144743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ec0ccc80 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a3e3ea640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009a60 tx=0x7f8a3002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3af29e6733ab2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a217fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30058020 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a340a01a0 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.696+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f8a30004250 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a30063050 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 10 client.18237 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628ec0cdf10 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a340a01a0 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8a30030070 con 0x5628ec057570 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 10 client.18237 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 10 client.18237 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 20 client.18237 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0ce2b0 0x5628ec0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 4 client.18237 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628ebfb8f00 con 0x5628ec0ce2b0 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.697+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0ce2b0 0x5628ec0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a217fa640 10 client.18237 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a3dbe9640 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0ce2b0 0x5628ec0d06a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7dd501285f2bd886 server_cookie=b219228bd7f46458 in_seq=0 out_seq=0 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a217fa640 1 -- 172.21.15.45:0/3102144743 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628ebfb8f00 con 0x5628ec0ce2b0 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a217fa640 10 client.18237 handle_command_reply: tid=0 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 client.18237 shutdown 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 msgr2=0x5628ec05bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009a60 tx=0x7f8a3002bf30 comp rx=0 tx=0).stop 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 shutdown_connections 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5628ec0ce2b0 0x5628ec0d06a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628ec057570 0x5628ec05bbd0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5628ec04cea0 0x5628ec05b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a217fa640 0 client.18237 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 --2- 172.21.15.45:0/3102144743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628ec04c580 0x5628ec05b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 >> 172.21.15.45:0/3102144743 conn(0x5628ec0488a0 msgr2=0x5628ec062240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 shutdown_connections 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 1 -- 172.21.15.45:0/3102144743 wait complete. 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.698+0000 7f8a42c3a740 20 client.18237 trim_cache size 0 max 16384 2024-04-25T00:31:15.705 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 msgr2=0x7f8a240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 0x7f8a240a51c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009430 tx=0x7f8a30048000 comp rx=0 tx=0).stop 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 msgr2=0x7f8a38197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9f0 tx=0x7f8a3400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 shutdown_connections 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f8a240a2d00 0x7f8a240a51c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a38107730 0x7f8a3819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3806a780 0x7f8a38198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 --2- 172.21.15.45:0/1539001261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a38069dc0 0x7f8a38197fe0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 >> 172.21.15.45:0/1539001261 conn(0x7f8a380776a0 msgr2=0x7f8a380fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 shutdown_connections 2024-04-25T00:31:15.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.699+0000 7f8a3fe4d640 1 -- 172.21.15.45:0/1539001261 wait complete. 2024-04-25T00:31:15.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:31:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 -- 172.21.15.45:0/2348099821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 msgr2=0x7f6b4c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 --2- 172.21.15.45:0/2348099821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c100aa0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009930 tx=0x7f6b4002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 -- 172.21.15.45:0/2348099821 shutdown_connections 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 --2- 172.21.15.45:0/2348099821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 --2- 172.21.15.45:0/2348099821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c100aa0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 --2- 172.21.15.45:0/2348099821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 0x7f6b4c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 -- 172.21.15.45:0/2348099821 >> 172.21.15.45:0/2348099821 conn(0x7f6b4c0776a0 msgr2=0x7f6b4c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 -- 172.21.15.45:0/2348099821 shutdown_connections 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.813+0000 7f6b51709640 1 -- 172.21.15.45:0/2348099821 wait complete. 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 Processor -- start 2024-04-25T00:31:15.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 -- start start 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 0x7f6b4c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c111b40 con 0x7f6b4c100620 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1119c0 con 0x7f6b4c100fe0 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b51709640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c111cc0 con 0x7f6b4c106630 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57766/0 (socket says 172.21.15.45:57766) 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b4affd640 1 -- 172.21.15.45:0/604718957 learned_addr learned my addr 172.21.15.45:0/604718957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.814+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 0x7f6b4c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 -- 172.21.15.45:0/604718957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 msgr2=0x7f6b4c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 0x7f6b4c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 -- 172.21.15.45:0/604718957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 msgr2=0x7f6b4c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 -- 172.21.15.45:0/604718957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c19d050 con 0x7f6b4c100620 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b4affd640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00e9f0 tx=0x7f6b3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b33fff640 1 -- 172.21.15.45:0/604718957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00cd80 con 0x7f6b4c100620 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c19d340 con 0x7f6b4c100620 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b33fff640 1 -- 172.21.15.45:0/604718957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c005500 con 0x7f6b4c100620 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b33fff640 1 -- 172.21.15.45:0/604718957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c010690 con 0x7f6b4c100620 2024-04-25T00:31:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.815+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a4bd0 con 0x7f6b4c100620 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.817+0000 7f6b33fff640 1 -- 172.21.15.45:0/604718957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b3c0046c0 con 0x7f6b4c100620 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.817+0000 7f6b54414740 1 Processor -- start 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.817+0000 7f6b54414740 1 -- start start 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2034030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce2036960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fce1f8e670 con 0x55fce2029580 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fce1f36970 con 0x55fce2029ea0 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b54414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fce1f8a630 con 0x55fce2034570 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b33fff640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 0x7f6b340a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57780/0 (socket says 172.21.15.45:57780) 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/1373888234 learned_addr learned my addr 172.21.15.45:0/1373888234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4affd640 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2034030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce2036960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 0x7f6b340a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/1373888234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 msgr2=0x55fce2036960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b33fff640 1 -- 172.21.15.45:0/604718957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f6b3c0cfd40 con 0x7f6b4c100620 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce2036960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/1373888234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 msgr2=0x55fce2034030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2034030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/1373888234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce1f5a400 con 0x55fce2029580 2024-04-25T00:31:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.818+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6b44013a70 tx=0x7f6b44013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bdb9f98103ca17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b317fa640 1 -- 172.21.15.45:0/1373888234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44013540 con 0x55fce2029580 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 0x7f6b340a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b44021000 tx=0x7f6b4401f000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b317fa640 1 -- 172.21.15.45:0/1373888234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44022070 con 0x55fce2029580 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b317fa640 1 -- 172.21.15.45:0/1373888234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4401cd90 con 0x55fce2029580 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 -- 172.21.15.45:0/1373888234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 msgr2=0x55fce2029960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6b44013a70 tx=0x7f6b44013f30 comp rx=0 tx=0).stop 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 -- 172.21.15.45:0/1373888234 shutdown_connections 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce2036960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2034030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 --2- 172.21.15.45:0/1373888234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029580 0x55fce2029960 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 -- 172.21.15.45:0/1373888234 >> 172.21.15.45:0/1373888234 conn(0x55fce20258a0 msgr2=0x55fce2025c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 -- 172.21.15.45:0/1373888234 shutdown_connections 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.819+0000 7f6b54414740 1 -- 172.21.15.45:0/1373888234 wait complete. 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 -- start start 2024-04-25T00:31:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 Processor -- start 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029580 0x55fce2033f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2031210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fce2037360 con 0x55fce2034570 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fce1f4a050 con 0x55fce2029ea0 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fce20374e0 con 0x55fce2029580 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b54414740 10 client.0 _ll_register_callbacks cb 0x55fce1ca1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57782/0 (socket says 172.21.15.45:57782) 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/382044367 learned_addr learned my addr 172.21.15.45:0/382044367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4affd640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029580 0x55fce2033f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2031210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/382044367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029580 msgr2=0x55fce2033f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029580 0x55fce2033f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 msgr2=0x55fce2031210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2031210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/382044367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce1f1bdc0 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.820+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2031210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6b4401df50 tx=0x7f6b4401cff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed65e01bc27283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44013440 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f6b440135e0 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b44022070 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b4403d050 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 10 client.18249 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fce1ca23f0 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44022070 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b44005040 con 0x55fce2034570 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 10 client.18249 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:15.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 20 client.18249 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20445a0 0x55fce2046990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 4 client.18249 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.821+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce1f9bbb0 con 0x55fce20445a0 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.822+0000 7f6b4affd640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20445a0 0x55fce2046990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.822+0000 7f6b2eff5640 10 client.18249 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.822+0000 7f6b4affd640 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20445a0 0x55fce2046990 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93d3234bc61a7b25 server_cookie=fbeba6e468890018 in_seq=0 out_seq=0 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b2eff5640 1 -- 172.21.15.45:0/382044367 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fce1f9bbb0 con 0x55fce20445a0 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b2eff5640 10 client.18249 handle_command_reply: tid=0 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 client.18249 shutdown 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 msgr2=0x55fce20317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6b4401df50 tx=0x7f6b4401cff0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 shutdown_connections 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20445a0 0x55fce2046990 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce20317b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029ea0 0x55fce2031210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 --2- 172.21.15.45:0/382044367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029580 0x55fce2033f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b2eff5640 0 client.18249 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:15.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 >> 172.21.15.45:0/382044367 conn(0x55fce20258a0 msgr2=0x55fce202b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 shutdown_connections 2024-04-25T00:31:15.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 1 -- 172.21.15.45:0/382044367 wait complete. 2024-04-25T00:31:15.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.823+0000 7f6b54414740 20 client.18249 trim_cache size 0 max 16384 2024-04-25T00:31:15.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.836+0000 7f6b54414740 1 Processor -- start 2024-04-25T00:31:15.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.836+0000 7f6b54414740 1 -- start start 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce2033a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029ea0 0x55fce202b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1aedc0 con 0x55fce2034570 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1b0900 con 0x55fce2029580 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b54414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1af490 con 0x55fce2029ea0 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57786/0 (socket says 172.21.15.45:57786) 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/3239740813 learned_addr learned my addr 172.21.15.45:0/3239740813 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4affd640 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce2033a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029ea0 0x55fce202b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/3239740813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029ea0 msgr2=0x55fce202b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029ea0 0x55fce202b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/3239740813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 msgr2=0x55fce2033a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce2033a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 -- 172.21.15.45:0/3239740813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce2029cd0 con 0x55fce2034570 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6b440054b0 tx=0x7f6b440092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624d142edb9d4e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b30ff9640 1 -- 172.21.15.45:0/3239740813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44046070 con 0x55fce2034570 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b30ff9640 1 -- 172.21.15.45:0/3239740813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44022070 con 0x55fce2034570 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.837+0000 7f6b30ff9640 1 -- 172.21.15.45:0/3239740813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b440056a0 con 0x55fce2034570 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- 172.21.15.45:0/3239740813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 msgr2=0x55fce204bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6b440054b0 tx=0x7f6b440092a0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- 172.21.15.45:0/3239740813 shutdown_connections 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2034570 0x55fce204bff0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2029ea0 0x55fce202b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 --2- 172.21.15.45:0/3239740813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce2033a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- 172.21.15.45:0/3239740813 >> 172.21.15.45:0/3239740813 conn(0x55fce20258a0 msgr2=0x55fce2031570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- 172.21.15.45:0/3239740813 shutdown_connections 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- 172.21.15.45:0/3239740813 wait complete. 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 -- start start 2024-04-25T00:31:15.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.838+0000 7f6b54414740 1 Processor -- start 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce202e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce202eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1b0230 con 0x55fce2029ea0 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1afb60 con 0x55fce2029580 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c1acdc0 con 0x55fce2034570 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 10 client.0 _ll_register_callbacks cb 0x7f6b3c09bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57796/0 (socket says 172.21.15.45:57796) 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 -- 172.21.15.45:0/2795233353 learned_addr learned my addr 172.21.15.45:0/2795233353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4affd640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce202e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce202eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 -- 172.21.15.45:0/2795233353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 msgr2=0x55fce202eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce202eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 -- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 msgr2=0x55fce202e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce202e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 -- 172.21.15.45:0/2795233353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce1d6c7d0 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4b7fe640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce202eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b4a7fc640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6b400096c0 tx=0x7f6b400318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad95ded25e5adf98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40035050 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f6b40004450 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.839+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b44022070 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b4004d050 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 10 client.18255 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fce20aaaa0 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44022070 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b40004c60 con 0x55fce2029ea0 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 10 client.18255 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 10 client.18255 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 20 client.18255 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20ab250 0x55fce20ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 4 client.18255 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fce1f95f00 con 0x55fce20ab250 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.840+0000 7f6b4affd640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20ab250 0x55fce20ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b4affd640 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20ab250 0x55fce20ad670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef40635e9b79dc5d server_cookie=6e85f09ffcdb27dd in_seq=0 out_seq=0 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b2eff5640 10 client.18255 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b2eff5640 1 -- 172.21.15.45:0/2795233353 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fce1f95f00 con 0x55fce20ab250 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b2eff5640 10 client.18255 handle_command_reply: tid=0 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 client.18255 shutdown 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 msgr2=0x55fce202e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f6b400096c0 tx=0x7f6b400318b0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 shutdown_connections 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fce20ab250 0x55fce20ad670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fce2034570 0x55fce202eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b2eff5640 0 client.18255 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:15.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fce2029ea0 0x55fce202e570 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 --2- 172.21.15.45:0/2795233353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fce2029580 0x55fce202e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 >> 172.21.15.45:0/2795233353 conn(0x55fce20258a0 msgr2=0x55fce202b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 shutdown_connections 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 1 -- 172.21.15.45:0/2795233353 wait complete. 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.841+0000 7f6b54414740 20 client.18255 trim_cache size 0 max 16384 2024-04-25T00:31:15.848 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 msgr2=0x7f6b340a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 0x7f6b340a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b44021000 tx=0x7f6b4401f000 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 msgr2=0x7f6b4c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00e9f0 tx=0x7f6b3c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 shutdown_connections 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f6b340a2d00 0x7f6b340a51c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b4c106630 0x7f6b4c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4c100fe0 0x7f6b4c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 --2- 172.21.15.45:0/604718957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b4c100620 0x7f6b4c198060 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 >> 172.21.15.45:0/604718957 conn(0x7f6b4c0776a0 msgr2=0x7f6b4c103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 shutdown_connections 2024-04-25T00:31:15.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.842+0000 7f6b51709640 1 -- 172.21.15.45:0/604718957 wait complete. 2024-04-25T00:31:15.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 -- 172.21.15.45:0/1059672948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12401066b0 msgr2=0x7f1240106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 --2- 172.21.15.45:0/1059672948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12401066b0 0x7f1240106a90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1230009930 tx=0x7f123002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 -- 172.21.15.45:0/1059672948 shutdown_connections 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 --2- 172.21.15.45:0/1059672948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1240100fd0 0x7f124010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 --2- 172.21.15.45:0/1059672948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f1240100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 --2- 172.21.15.45:0/1059672948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12401066b0 0x7f1240106a90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 -- 172.21.15.45:0/1059672948 >> 172.21.15.45:0/1059672948 conn(0x7f1240077740 msgr2=0x7f1240077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 -- 172.21.15.45:0/1059672948 shutdown_connections 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.957+0000 7f1245f7d640 1 -- 172.21.15.45:0/1059672948 wait complete. 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 Processor -- start 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 -- start start 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f12401022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 0x7f1240102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12401109f0 con 0x7f1240100fd0 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1240110870 con 0x7f1240100610 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f1245f7d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1240110b70 con 0x7f12401066b0 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f123effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f123effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57812/0 (socket says 172.21.15.45:57812) 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f123effd640 1 -- 172.21.15.45:0/4210830731 learned_addr learned my addr 172.21.15.45:0/4210830731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f123f7fe640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f12401022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.958+0000 7f123ffff640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 0x7f1240102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 -- 172.21.15.45:0/4210830731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 msgr2=0x7f1240102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 0x7f1240102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 -- 172.21.15.45:0/4210830731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 msgr2=0x7f12401022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f12401022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 -- 172.21.15.45:0/4210830731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401a63f0 con 0x7f1240100fd0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123ffff640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 0x7f1240102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123f7fe640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f12401022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123effd640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f123400d950 tx=0x7f123400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123cff9640 1 -- 172.21.15.45:0/4210830731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234017070 con 0x7f1240100fd0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12401a66e0 con 0x7f1240100fd0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123cff9640 1 -- 172.21.15.45:0/4210830731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1234009c30 con 0x7f1240100fd0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f123cff9640 1 -- 172.21.15.45:0/4210830731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123400f6b0 con 0x7f1240100fd0 2024-04-25T00:31:15.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.959+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a6c30 con 0x7f1240100fd0 2024-04-25T00:31:15.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.961+0000 7f123cff9640 1 -- 172.21.15.45:0/4210830731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12340109d0 con 0x7f1240100fd0 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 Processor -- start 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 -- start start 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619ceaab670 con 0x5619ceb46580 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5619cea53970 con 0x5619ceb46ea0 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f1248c85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619ceaa7630 con 0x5619ceb51570 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123cff9640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 0x7f12280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57826/0 (socket says 172.21.15.45:57826) 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 -- 172.21.15.45:0/3400208505 learned_addr learned my addr 172.21.15.45:0/3400208505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 0x7f12280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123cff9640 1 -- 172.21.15.45:0/4210830731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f12340cf500 con 0x7f1240100fd0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123effd640 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 -- 172.21.15.45:0/3400208505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 msgr2=0x5619ceb53960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb53960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 -- 172.21.15.45:0/3400208505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 msgr2=0x5619ceb51030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb51030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.962+0000 7f123ffff640 1 -- 172.21.15.45:0/3400208505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cea77400 con 0x5619ceb46580 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f123ffff640 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1238013e50 tx=0x7f12380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87625d427852545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f123ffff640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 0x7f12280a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1238010be0 tx=0x7f1238015000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f1225ffb640 1 -- 172.21.15.45:0/3400208505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800ad50 con 0x5619ceb46580 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f1225ffb640 1 -- 172.21.15.45:0/3400208505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1238007c80 con 0x5619ceb46580 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f1225ffb640 1 -- 172.21.15.45:0/3400208505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238005930 con 0x5619ceb46580 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f1248c85740 1 -- 172.21.15.45:0/3400208505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 msgr2=0x5619ceb46960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.963+0000 7f1248c85740 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1238013e50 tx=0x7f12380107a0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- 172.21.15.45:0/3400208505 shutdown_connections 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb51030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- 172.21.15.45:0/3400208505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb46960 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- 172.21.15.45:0/3400208505 >> 172.21.15.45:0/3400208505 conn(0x5619ceb428a0 msgr2=0x5619ceb42c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- 172.21.15.45:0/3400208505 shutdown_connections 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- 172.21.15.45:0/3400208505 wait complete. 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- start start 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 Processor -- start 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619ceb54300 con 0x5619ceb46580 2024-04-25T00:31:15.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5619cea67050 con 0x5619ceb46ea0 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.964+0000 7f1248c85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619ceb54480 con 0x5619ceb51570 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f1248c85740 10 client.0 _ll_register_callbacks cb 0x5619ce7bebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57832/0 (socket says 172.21.15.45:57832) 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 -- 172.21.15.45:0/2995232940 learned_addr learned my addr 172.21.15.45:0/2995232940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123effd640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123ffff640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 -- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 msgr2=0x5619ceb4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 -- 172.21.15.45:0/2995232940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 msgr2=0x5619ceb4e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb4e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 -- 172.21.15.45:0/2995232940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cea38dc0 con 0x5619ceb46580 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f123f7fe640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1240101750 tx=0x7f1230031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad141c6d52842df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f121effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230035040 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1238007c80 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f1230004240 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f123004e050 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f1248c85740 10 client.18267 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.965+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619ce7bf3f0 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1238007c80 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1230004b80 con 0x5619ceb46580 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 10 client.18267 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:15.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 20 client.18267 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619ceb61670 0x5619ceb63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 4 client.18267 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ceab8bb0 con 0x5619ceb61670 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.966+0000 7f123effd640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619ceb61670 0x5619ceb63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f123effd640 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619ceb61670 0x5619ceb63a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c38195abd32ff08 server_cookie=c1522733ebd2062c in_seq=0 out_seq=0 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f121effd640 10 client.18267 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f121effd640 1 -- 172.21.15.45:0/2995232940 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619ceab8bb0 con 0x5619ceb61670 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f121effd640 10 client.18267 handle_command_reply: tid=0 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 client.18267 shutdown 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 msgr2=0x5619ceb50fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1240101750 tx=0x7f1230031920 comp rx=0 tx=0).stop 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 shutdown_connections 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619ceb61670 0x5619ceb63a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb4e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f121effd640 0 client.18267 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 --2- 172.21.15.45:0/2995232940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb50fb0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.967+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 >> 172.21.15.45:0/2995232940 conn(0x5619ceb428a0 msgr2=0x5619ceb486a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.968+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 shutdown_connections 2024-04-25T00:31:15.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.968+0000 7f1248c85740 1 -- 172.21.15.45:0/2995232940 wait complete. 2024-04-25T00:31:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.968+0000 7f1248c85740 20 client.18267 trim_cache size 0 max 16384 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 Processor -- start 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 -- start start 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb50ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb51020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12401b0e70 con 0x5619ceb46580 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12401b29b0 con 0x5619ceb46ea0 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f1248c85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12401b1540 con 0x5619ceb51570 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57838/0 (socket says 172.21.15.45:57838) 2024-04-25T00:31:15.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 -- 172.21.15.45:0/3030229703 learned_addr learned my addr 172.21.15.45:0/3030229703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123ffff640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb51020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123effd640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb50ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 -- 172.21.15.45:0/3030229703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 msgr2=0x5619ceb51020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb51020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 -- 172.21.15.45:0/3030229703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 msgr2=0x5619ceb50ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb50ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123f7fe640 1 -- 172.21.15.45:0/3030229703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ce8897d0 con 0x5619ceb46580 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123ffff640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb51020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.981+0000 7f123effd640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb50ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f1230001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f826e7fe23fa1bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f12257fa640 1 -- 172.21.15.45:0/3030229703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230057070 con 0x5619ceb46580 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f12257fa640 1 -- 172.21.15.45:0/3030229703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1230005500 con 0x5619ceb46580 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f12257fa640 1 -- 172.21.15.45:0/3030229703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230035070 con 0x5619ceb46580 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 -- 172.21.15.45:0/3030229703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 msgr2=0x5619ceb5d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f1230001e70 comp rx=0 tx=0).stop 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 -- 172.21.15.45:0/3030229703 shutdown_connections 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb51020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46ea0 0x5619ceb50ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 --2- 172.21.15.45:0/3030229703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46580 0x5619ceb5d1c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 -- 172.21.15.45:0/3030229703 >> 172.21.15.45:0/3030229703 conn(0x5619ceb428a0 msgr2=0x5619ceb48600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 -- 172.21.15.45:0/3030229703 shutdown_connections 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.982+0000 7f1248c85740 1 -- 172.21.15.45:0/3030229703 wait complete. 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 -- start start 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 Processor -- start 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 0x5619ceb55150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb55bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12401b22e0 con 0x5619ceb46ea0 2024-04-25T00:31:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12401b1c10 con 0x5619ceb46580 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12401aee70 con 0x5619ceb51570 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f1248c85740 10 client.0 _ll_register_callbacks cb 0x7f12300315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57846/0 (socket says 172.21.15.45:57846) 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 -- 172.21.15.45:0/3717655875 learned_addr learned my addr 172.21.15.45:0/3717655875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 0x5619ceb55150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 -- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 msgr2=0x5619ceb55bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123ffff640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb55bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb55bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 -- 172.21.15.45:0/3717655875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 msgr2=0x5619ceb55150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 0x5619ceb55150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123effd640 1 -- 172.21.15.45:0/3717655875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cebc6ba0 con 0x5619ceb46ea0 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 0x5619ceb55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.983+0000 7f123ffff640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb55bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f123effd640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1234097b10 tx=0x7f1234005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055a5420c55f144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12340b3070 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7f1234097d20 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1230005500 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1234020040 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 10 client.18273 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619cebc7e30 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1230005500 con 0x5619ceb46ea0 2024-04-25T00:31:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f12340bb040 con 0x5619ceb46ea0 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 10 client.18273 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 10 client.18273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 20 client.18273 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619cebc8560 0x5619cebca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 4 client.18273 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.984+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619ceab2f00 con 0x5619cebc8560 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619cebc8560 0x5619cebca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:15.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f123f7fe640 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619cebc8560 0x5619cebca980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=133aa0208ee34045 server_cookie=1c73bad1720d93e5 in_seq=0 out_seq=0 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f121effd640 10 client.18273 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f121effd640 1 -- 172.21.15.45:0/3717655875 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619ceab2f00 con 0x5619cebc8560 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f121effd640 10 client.18273 handle_command_reply: tid=0 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 client.18273 shutdown 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 msgr2=0x5619ceb55690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1234097b10 tx=0x7f1234005f90 comp rx=0 tx=0).stop 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 shutdown_connections 2024-04-25T00:31:15.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x5619cebc8560 0x5619cebca980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619ceb51570 0x5619ceb55bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619ceb46ea0 0x5619ceb55690 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 --2- 172.21.15.45:0/3717655875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619ceb46580 0x5619ceb55150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f121effd640 0 client.18273 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 >> 172.21.15.45:0/3717655875 conn(0x5619ceb428a0 msgr2=0x5619ceb48eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 shutdown_connections 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.985+0000 7f1248c85740 1 -- 172.21.15.45:0/3717655875 wait complete. 2024-04-25T00:31:15.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1248c85740 20 client.18273 trim_cache size 0 max 16384 2024-04-25T00:31:15.996 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 msgr2=0x7f12280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 0x7f12280a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1238010be0 tx=0x7f1238015000 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 msgr2=0x7f12401027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f123400d950 tx=0x7f123400de10 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 shutdown_connections 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f12280a2d00 0x7f12280a51c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12401066b0 0x7f1240102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1240100fd0 0x7f12401027f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 --2- 172.21.15.45:0/4210830731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1240100610 0x7f12401022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 >> 172.21.15.45:0/4210830731 conn(0x7f1240077740 msgr2=0x7f1240068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 shutdown_connections 2024-04-25T00:31:15.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:15.986+0000 7f1245f7d640 1 -- 172.21.15.45:0/4210830731 wait complete. 2024-04-25T00:31:15.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 -- 172.21.15.45:0/3417542030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 msgr2=0x7fdf94107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 --2- 172.21.15.45:0/3417542030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf94107b00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009930 tx=0x7fdf8402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 -- 172.21.15.45:0/3417542030 shutdown_connections 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 --2- 172.21.15.45:0/3417542030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941020d0 0x7fdf9410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 --2- 172.21.15.45:0/3417542030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf94101710 0x7fdf94101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 --2- 172.21.15.45:0/3417542030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf94107b00 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 -- 172.21.15.45:0/3417542030 >> 172.21.15.45:0/3417542030 conn(0x7fdf940fd180 msgr2=0x7fdf940ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 -- 172.21.15.45:0/3417542030 shutdown_connections 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.103+0000 7fdf9b015640 1 -- 172.21.15.45:0/3417542030 wait complete. 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 Processor -- start 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 -- start start 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf94101710 0x7fdf94198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf941020d0 0x7fdf94198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94111b80 con 0x7fdf94107720 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94111a00 con 0x7fdf94101710 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf9b015640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94111d00 con 0x7fdf941020d0 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf995b2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf995b2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57874/0 (socket says 172.21.15.45:57874) 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf995b2640 1 -- 172.21.15.45:0/390857643 learned_addr learned my addr 172.21.15.45:0/390857643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf98db1640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf94101710 0x7fdf94198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.104+0000 7fdf93fff640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf941020d0 0x7fdf94198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 -- 172.21.15.45:0/390857643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf941020d0 msgr2=0x7fdf94198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf941020d0 0x7fdf94198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 -- 172.21.15.45:0/390857643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf94101710 msgr2=0x7fdf94198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf94101710 0x7fdf94198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 -- 172.21.15.45:0/390857643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9419d010 con 0x7fdf94107720 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf995b2640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00cce0 tx=0x7fdf8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf91ffb640 1 -- 172.21.15.45:0/390857643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00f450 con 0x7fdf94107720 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf91ffb640 1 -- 172.21.15.45:0/390857643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c00fad0 con 0x7fdf94107720 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9419d300 con 0x7fdf94107720 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf91ffb640 1 -- 172.21.15.45:0/390857643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c018c10 con 0x7fdf94107720 2024-04-25T00:31:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.105+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941a4b90 con 0x7fdf94107720 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.107+0000 7fdf91ffb640 1 -- 172.21.15.45:0/390857643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdf8c00f5f0 con 0x7fdf94107720 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 Processor -- start 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 -- start start 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f14e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f15b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e60f0b3670 con 0x55e60f14eea0 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e60f05b970 con 0x55e60f159570 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf9de04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e60f0af630 con 0x55e60f14e580 2024-04-25T00:31:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57882/0 (socket says 172.21.15.45:57882) 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 -- 172.21.15.45:0/3358018789 learned_addr learned my addr 172.21.15.45:0/3358018789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf91ffb640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 0x7fdf7c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf93fff640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf93fff640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf995b2640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 0x7fdf7c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 -- 172.21.15.45:0/3358018789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 msgr2=0x55e60f14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 -- 172.21.15.45:0/3358018789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 msgr2=0x55e60f15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 -- 172.21.15.45:0/3358018789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60f07f400 con 0x55e60f14eea0 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf93fff640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf91ffb640 1 -- 172.21.15.45:0/390857643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7fdf8c0cff40 con 0x7fdf94107720 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.108+0000 7fdf98db1640 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009930 tx=0x7fdf84005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861af7ac908b3277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf7affd640 1 -- 172.21.15.45:0/3358018789 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84040070 con 0x55e60f14eea0 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf7affd640 1 -- 172.21.15.45:0/3358018789 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf84004490 con 0x55e60f14eea0 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf7affd640 1 -- 172.21.15.45:0/3358018789 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84035040 con 0x55e60f14eea0 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 -- 172.21.15.45:0/3358018789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 msgr2=0x55e60f159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf995b2640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 0x7fdf7c0a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c09bcf0 tx=0x7fdf8c06dfb0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009930 tx=0x7fdf84005c00 comp rx=0 tx=0).stop 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 -- 172.21.15.45:0/3358018789 shutdown_connections 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f159030 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 --2- 172.21.15.45:0/3358018789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 -- 172.21.15.45:0/3358018789 >> 172.21.15.45:0/3358018789 conn(0x55e60f14a8a0 msgr2=0x55e60f14ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 -- 172.21.15.45:0/3358018789 shutdown_connections 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.109+0000 7fdf9de04740 1 -- 172.21.15.45:0/3358018789 wait complete. 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 -- start start 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 Processor -- start 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e60f15c300 con 0x55e60f14eea0 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e60f06f050 con 0x55e60f159570 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e60f15c480 con 0x55e60f14e580 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf9de04740 10 client.0 _ll_register_callbacks cb 0x55e60edc6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57886/0 (socket says 172.21.15.45:57886) 2024-04-25T00:31:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 -- 172.21.15.45:0/4100910247 learned_addr learned my addr 172.21.15.45:0/4100910247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf93fff640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf93fff640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 -- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 msgr2=0x55e60f158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 -- 172.21.15.45:0/4100910247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 msgr2=0x55e60f1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.110+0000 7fdf98db1640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf98db1640 1 -- 172.21.15.45:0/4100910247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60f040dc0 con 0x55e60f14eea0 2024-04-25T00:31:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf98db1640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402a940 tx=0x7fdf8402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf105bf939dfd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84040030 con 0x55e60f14eea0 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7fdf84005430 con 0x55e60f14eea0 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf84004490 con 0x55e60f14eea0 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf84056050 con 0x55e60f14eea0 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 10 client.18285 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e60edc73f0 con 0x55e60f14eea0 2024-04-25T00:31:16.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf84004490 con 0x55e60f14eea0 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf840037f0 con 0x55e60f14eea0 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 10 client.18285 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 10 client.18285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 20 client.18285 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.111+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f169370 0x55e60f16b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.112+0000 7fdf9de04740 4 client.18285 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.112+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60f0c0bb0 con 0x55e60f169370 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.112+0000 7fdf93fff640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f169370 0x55e60f16b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.112+0000 7fdf73fff640 10 client.18285 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.112+0000 7fdf93fff640 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f169370 0x55e60f16b760 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ac807d839d78236 server_cookie=77dd95cf09b7fe08 in_seq=0 out_seq=0 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf73fff640 1 -- 172.21.15.45:0/4100910247 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e60f0c0bb0 con 0x55e60f169370 2024-04-25T00:31:16.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf73fff640 10 client.18285 handle_command_reply: tid=0 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 client.18285 shutdown 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 msgr2=0x55e60f1561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402a940 tx=0x7fdf8402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 shutdown_connections 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f169370 0x55e60f16b760 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14eea0 0x55e60f1561a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf73fff640 0 client.18285 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 --2- 172.21.15.45:0/4100910247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14e580 0x55e60f158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 >> 172.21.15.45:0/4100910247 conn(0x55e60f14a8a0 msgr2=0x55e60f149b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 shutdown_connections 2024-04-25T00:31:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 1 -- 172.21.15.45:0/4100910247 wait complete. 2024-04-25T00:31:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.113+0000 7fdf9de04740 20 client.18285 trim_cache size 0 max 16384 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.126+0000 7fdf9de04740 1 Processor -- start 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 -- start start 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941aecc0 con 0x55e60f14e580 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b0800 con 0x55e60f159570 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf9de04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941af390 con 0x55e60f14eea0 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57902/0 (socket says 172.21.15.45:57902) 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 -- 172.21.15.45:0/1200517782 learned_addr learned my addr 172.21.15.45:0/1200517782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf98db1640 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 -- 172.21.15.45:0/1200517782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 msgr2=0x55e60f1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 -- 172.21.15.45:0/1200517782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 msgr2=0x55e60f153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 -- 172.21.15.45:0/1200517782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60ee917d0 con 0x55e60f14e580 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf93fff640 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005fd0 tx=0x7fdf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b070c2dcb61d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf7a7fc640 1 -- 172.21.15.45:0/1200517782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88011be0 con 0x55e60f14e580 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf7a7fc640 1 -- 172.21.15.45:0/1200517782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88011d80 con 0x55e60f14e580 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.127+0000 7fdf7a7fc640 1 -- 172.21.15.45:0/1200517782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8801f070 con 0x55e60f14e580 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- 172.21.15.45:0/1200517782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 msgr2=0x55e60f16eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005fd0 tx=0x7fdf880046c0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- 172.21.15.45:0/1200517782 shutdown_connections 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f159570 0x55e60f153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 --2- 172.21.15.45:0/1200517782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f14e580 0x55e60f16eb70 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- 172.21.15.45:0/1200517782 >> 172.21.15.45:0/1200517782 conn(0x55e60f14a8a0 msgr2=0x55e60f164f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- 172.21.15.45:0/1200517782 shutdown_connections 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- 172.21.15.45:0/1200517782 wait complete. 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 -- start start 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.128+0000 7fdf9de04740 1 Processor -- start 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f14e580 0x55e60f1cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941b0130 con 0x55e60f159570 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941afa60 con 0x55e60f14e580 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf941accc0 con 0x55e60f14eea0 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf9de04740 10 client.0 _ll_register_callbacks cb 0x7fdf84004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57908/0 (socket says 172.21.15.45:57908) 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 -- 172.21.15.45:0/246510379 learned_addr learned my addr 172.21.15.45:0/246510379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf98db1640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f14e580 0x55e60f1cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 -- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 msgr2=0x55e60f1cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 -- 172.21.15.45:0/246510379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f14e580 msgr2=0x55e60f1cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f14e580 0x55e60f1cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 -- 172.21.15.45:0/246510379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60f1536c0 con 0x55e60f159570 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf93fff640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fdf88003670 tx=0x7fdf88013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493285ad8303be55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf73fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.129+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802f060 con 0x55e60f159570 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf88011d80 con 0x55e60f159570 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7fdf8801fae0 con 0x55e60f159570 2024-04-25T00:31:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf88003fe0 con 0x55e60f159570 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 10 client.18291 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e60f1d34b0 con 0x55e60f159570 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88011d80 con 0x55e60f159570 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf8801fce0 con 0x55e60f159570 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 10 client.18291 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 10 client.18291 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 20 client.18291 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f1d3b50 0x55e60f1d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 4 client.18291 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e60f0baf00 con 0x55e60f1d3b50 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.130+0000 7fdf98db1640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f1d3b50 0x55e60f1d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf98db1640 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f1d3b50 0x55e60f1d5f70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de1137f2809eb7df server_cookie=fed17194d5adeb96 in_seq=0 out_seq=0 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf73fff640 10 client.18291 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf73fff640 1 -- 172.21.15.45:0/246510379 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e60f0baf00 con 0x55e60f1d3b50 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf73fff640 10 client.18291 handle_command_reply: tid=0 2024-04-25T00:31:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 client.18291 shutdown 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 msgr2=0x55e60f152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fdf88003670 tx=0x7fdf88013040 comp rx=0 tx=0).stop 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 shutdown_connections 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55e60f1d3b50 0x55e60f1d5f70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e60f159570 0x55e60f152fd0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e60f14eea0 0x55e60f1cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf73fff640 0 client.18291 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 --2- 172.21.15.45:0/246510379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e60f14e580 0x55e60f1cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 >> 172.21.15.45:0/246510379 conn(0x55e60f14a8a0 msgr2=0x55e60f14fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 shutdown_connections 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 1 -- 172.21.15.45:0/246510379 wait complete. 2024-04-25T00:31:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.131+0000 7fdf9de04740 20 client.18291 trim_cache size 0 max 16384 2024-04-25T00:31:16.151 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 msgr2=0x7fdf7c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 0x7fdf7c0a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c09bcf0 tx=0x7fdf8c06dfb0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 msgr2=0x7fdf9419c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00cce0 tx=0x7fdf8c00a620 comp rx=0 tx=0).stop 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 shutdown_connections 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdf7c0a2d00 0x7fdf7c0a51c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf94107720 0x7fdf9419c8e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf941020d0 0x7fdf94198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 --2- 172.21.15.45:0/390857643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf94101710 0x7fdf94198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 >> 172.21.15.45:0/390857643 conn(0x7fdf940fd180 msgr2=0x7fdf940fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 shutdown_connections 2024-04-25T00:31:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.132+0000 7fdf9b015640 1 -- 172.21.15.45:0/390857643 wait complete. 2024-04-25T00:31:16.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:16 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3567921560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:16.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 -- 172.21.15.45:0/3463231768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc107730 msgr2=0x7efebc107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 --2- 172.21.15.45:0/3463231768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc107730 0x7efebc107b10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009930 tx=0x7efeb002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 -- 172.21.15.45:0/3463231768 shutdown_connections 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 --2- 172.21.15.45:0/3463231768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 --2- 172.21.15.45:0/3463231768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc069dc0 0x7efebc06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 --2- 172.21.15.45:0/3463231768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc107730 0x7efebc107b10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 -- 172.21.15.45:0/3463231768 >> 172.21.15.45:0/3463231768 conn(0x7efebc0776a0 msgr2=0x7efebc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 -- 172.21.15.45:0/3463231768 shutdown_connections 2024-04-25T00:31:16.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.257+0000 7efec1b5a640 1 -- 172.21.15.45:0/3463231768 wait complete. 2024-04-25T00:31:16.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 Processor -- start 2024-04-25T00:31:16.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 -- start start 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc107730 0x7efebc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111b40 con 0x7efebc069dc0 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1119c0 con 0x7efebc107730 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efec1b5a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111cc0 con 0x7efebc06a780 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57918/0 (socket says 172.21.15.45:57918) 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 -- 172.21.15.45:0/1302400114 learned_addr learned my addr 172.21.15.45:0/1302400114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebaffd640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebbfff640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc107730 0x7efebc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 -- 172.21.15.45:0/1302400114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 msgr2=0x7efebc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 -- 172.21.15.45:0/1302400114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc107730 msgr2=0x7efebc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc107730 0x7efebc19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebb7fe640 1 -- 172.21.15.45:0/1302400114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc19d190 con 0x7efebc069dc0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.258+0000 7efebaffd640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efebb7fe640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009230 tx=0x7efeb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efeb8ff9640 1 -- 172.21.15.45:0/1302400114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb003f030 con 0x7efebc069dc0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efeb8ff9640 1 -- 172.21.15.45:0/1302400114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb0003ad0 con 0x7efebc069dc0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc19d420 con 0x7efebc069dc0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efeb8ff9640 1 -- 172.21.15.45:0/1302400114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0035070 con 0x7efebc069dc0 2024-04-25T00:31:16.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.259+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a4d10 con 0x7efebc069dc0 2024-04-25T00:31:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.261+0000 7efeb8ff9640 1 -- 172.21.15.45:0/1302400114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efeb0042030 con 0x7efebc069dc0 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.261+0000 7efec4861740 1 Processor -- start 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.261+0000 7efec4861740 1 -- start start 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.261+0000 7efeb8ff9640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 0x7efea40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.261+0000 7efec4861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efec4861740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 0x558d40543030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efec4861740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d40545960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efec4861740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d4049d670 con 0x558d40543570 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efec4861740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558d40445970 con 0x558d40538ea0 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efec4861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558d40499630 con 0x558d40538580 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebbfff640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 0x7efea40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efeb8ff9640 1 -- 172.21.15.45:0/1302400114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7efeb00eff50 con 0x7efebc069dc0 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35472/0 (socket says 172.21.15.45:35472) 2024-04-25T00:31:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebb7fe640 1 -- 172.21.15.45:0/9481907 learned_addr learned my addr 172.21.15.45:0/9481907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 0x558d40543030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebbfff640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d40545960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 -- 172.21.15.45:0/9481907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 msgr2=0x558d40538960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 -- 172.21.15.45:0/9481907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 msgr2=0x558d40545960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d40545960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 -- 172.21.15.45:0/9481907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d40469400 con 0x558d40538ea0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebb7fe640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebbfff640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d40545960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebbfff640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 0x7efea40a51c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009270 tx=0x7efeb4009cf0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.262+0000 7efebaffd640 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 0x558d40543030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efebc077ca0 tx=0x7efeac00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498653f8fdce02fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efea1ffb640 1 -- 172.21.15.45:0/9481907 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac015070 con 0x558d40538ea0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efea1ffb640 1 -- 172.21.15.45:0/9481907 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac01a940 con 0x558d40538ea0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efea1ffb640 1 -- 172.21.15.45:0/9481907 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0118c0 con 0x558d40538ea0 2024-04-25T00:31:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 -- 172.21.15.45:0/9481907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 msgr2=0x558d40543030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 0x558d40543030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efebc077ca0 tx=0x7efeac00f730 comp rx=0 tx=0).stop 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 -- 172.21.15.45:0/9481907 shutdown_connections 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d40545960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538ea0 0x558d40543030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 --2- 172.21.15.45:0/9481907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40538960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 -- 172.21.15.45:0/9481907 >> 172.21.15.45:0/9481907 conn(0x558d405348a0 msgr2=0x558d40534c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 -- 172.21.15.45:0/9481907 shutdown_connections 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.263+0000 7efec4861740 1 -- 172.21.15.45:0/9481907 wait complete. 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 -- start start 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 Processor -- start 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40542ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40540720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d40546360 con 0x558d40538ea0 2024-04-25T00:31:16.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558d40459050 con 0x558d40543570 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558d405464e0 con 0x558d40538580 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efec4861740 10 client.0 _ll_register_callbacks cb 0x558d401b0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57928/0 (socket says 172.21.15.45:57928) 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 -- 172.21.15.45:0/3946441705 learned_addr learned my addr 172.21.15.45:0/3946441705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebb7fe640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40542ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebbfff640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40540720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 -- 172.21.15.45:0/3946441705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 msgr2=0x558d40542ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40542ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 -- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 msgr2=0x558d40540720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40540720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.264+0000 7efebaffd640 1 -- 172.21.15.45:0/3946441705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d4042adc0 con 0x558d40538ea0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efebaffd640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7efeac0115c0 tx=0x7efeac00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1070ac6de8ce7450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac015070 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7efeac01a460 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac01a940 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efeac02e050 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 10 client.18303 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d401b13f0 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac01a940 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efeac005440 con 0x558d40538ea0 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 10 client.18303 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 20 client.18303 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405535c0 0x558d405559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 4 client.18303 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.265+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d404aabb0 con 0x558d405535c0 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.266+0000 7efebb7fe640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405535c0 0x558d405559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.266+0000 7efebb7fe640 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405535c0 0x558d405559b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0e14319c6983f6 server_cookie=907b0bbfba96c399 in_seq=0 out_seq=0 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.266+0000 7efe9affd640 10 client.18303 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efe9affd640 1 -- 172.21.15.45:0/3946441705 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d404aabb0 con 0x558d405535c0 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efe9affd640 10 client.18303 handle_command_reply: tid=0 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 client.18303 shutdown 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 msgr2=0x558d405401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7efeac0115c0 tx=0x7efeac00bf70 comp rx=0 tx=0).stop 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 shutdown_connections 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405535c0 0x558d405559b0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538ea0 0x558d405401c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 --2- 172.21.15.45:0/3946441705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538580 0x558d40542ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 >> 172.21.15.45:0/3946441705 conn(0x558d405348a0 msgr2=0x558d4053a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efe9affd640 0 client.18303 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 shutdown_connections 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 1 -- 172.21.15.45:0/3946441705 wait complete. 2024-04-25T00:31:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.267+0000 7efec4861740 20 client.18303 trim_cache size 0 max 16384 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 Processor -- start 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 -- start start 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538580 0x558d4054f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d4055b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1aed70 con 0x558d40543570 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b08b0 con 0x558d40538580 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.280+0000 7efec4861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1af440 con 0x558d40538ea0 2024-04-25T00:31:16.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57934/0 (socket says 172.21.15.45:57934) 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 -- 172.21.15.45:0/970591230 learned_addr learned my addr 172.21.15.45:0/970591230 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebaffd640 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d4055b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebb7fe640 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538580 0x558d4054f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 -- 172.21.15.45:0/970591230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 msgr2=0x558d4055b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d4055b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 -- 172.21.15.45:0/970591230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538580 msgr2=0x558d4054f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538580 0x558d4054f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 -- 172.21.15.45:0/970591230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d4027b7d0 con 0x558d40543570 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efebbfff640 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccdbf58bc644a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efea17fa640 1 -- 172.21.15.45:0/970591230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4004040 con 0x558d40543570 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efea17fa640 1 -- 172.21.15.45:0/970591230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb40046c0 con 0x558d40543570 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.281+0000 7efea17fa640 1 -- 172.21.15.45:0/970591230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4021b20 con 0x558d40543570 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- 172.21.15.45:0/970591230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 msgr2=0x558d4055b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7efeb4005fd0 tx=0x7efeb401e000 comp rx=0 tx=0).stop 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- 172.21.15.45:0/970591230 shutdown_connections 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40543570 0x558d4055b5a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d4055b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 --2- 172.21.15.45:0/970591230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40538580 0x558d4054f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- 172.21.15.45:0/970591230 >> 172.21.15.45:0/970591230 conn(0x558d405348a0 msgr2=0x558d40533b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- 172.21.15.45:0/970591230 shutdown_connections 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- 172.21.15.45:0/970591230 wait complete. 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 -- start start 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.282+0000 7efec4861740 1 Processor -- start 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d40548e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40549390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b01e0 con 0x558d40538580 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1afb10 con 0x558d40543570 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1acd70 con 0x558d40538ea0 2024-04-25T00:31:16.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 10 client.0 _ll_register_callbacks cb 0x7efeb4014630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57940/0 (socket says 172.21.15.45:57940) 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 -- 172.21.15.45:0/425298112 learned_addr learned my addr 172.21.15.45:0/425298112 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebaffd640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d40548e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebbfff640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40549390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 -- 172.21.15.45:0/425298112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 msgr2=0x558d40548e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d40548e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 -- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 msgr2=0x558d40549390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40549390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 -- 172.21.15.45:0/425298112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d40549b90 con 0x558d40538580 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efebb7fe640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009a50 tx=0x7efeb0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adb3a319227dca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efe9affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0049040 con 0x558d40538580 2024-04-25T00:31:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9563+0+0 (secure 0 0 0) 0x7efeb00c00b0 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.283+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb40046c0 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efeb0087120 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 10 client.18309 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d405b98b0 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb40046c0 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efeb00ed510 con 0x558d40538580 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 10 client.18309 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 20 client.18309 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405ba000 0x558d405bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 4 client.18309 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d404a4f00 con 0x558d405ba000 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.284+0000 7efebaffd640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405ba000 0x558d405bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efebaffd640 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405ba000 0x558d405bc3f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d433da385fab67fd server_cookie=bbf3211a3e3aef24 in_seq=0 out_seq=0 2024-04-25T00:31:16.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efe9affd640 10 client.18309 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efe9affd640 1 -- 172.21.15.45:0/425298112 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d404a4f00 con 0x558d405ba000 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efe9affd640 10 client.18309 handle_command_reply: tid=0 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 client.18309 shutdown 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 msgr2=0x558d4054bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009a50 tx=0x7efeb0009260 comp rx=0 tx=0).stop 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 shutdown_connections 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d40543570 0x558d40549390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d40538ea0 0x558d40548e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558d405ba000 0x558d405bc3f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 --2- 172.21.15.45:0/425298112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d40538580 0x558d4054bac0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efe9affd640 0 client.18309 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 >> 172.21.15.45:0/425298112 conn(0x558d405348a0 msgr2=0x558d4054e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 shutdown_connections 2024-04-25T00:31:16.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 1 -- 172.21.15.45:0/425298112 wait complete. 2024-04-25T00:31:16.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.285+0000 7efec4861740 20 client.18309 trim_cache size 0 max 16384 2024-04-25T00:31:16.292 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 msgr2=0x7efea40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 0x7efea40a51c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009270 tx=0x7efeb4009cf0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 msgr2=0x7efebc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009230 tx=0x7efeb0004270 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 shutdown_connections 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7efea40a2d00 0x7efea40a51c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efebc107730 0x7efebc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc06a780 0x7efebc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 --2- 172.21.15.45:0/1302400114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efebc069dc0 0x7efebc1981f0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 >> 172.21.15.45:0/1302400114 conn(0x7efebc0776a0 msgr2=0x7efebc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 shutdown_connections 2024-04-25T00:31:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:16.286+0000 7efec1b5a640 1 -- 172.21.15.45:0/1302400114 wait complete. 2024-04-25T00:31:16.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:16 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3567921560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:16.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:16 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3567921560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:17.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:31:17.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:17.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:31:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:17 smithi071 ceph-mon[32667]: pgmap v287: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:31:17.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:17 smithi071 ceph-mon[32667]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:31:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:17 smithi045 ceph-mon[29194]: pgmap v287: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:31:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:17 smithi045 ceph-mon[29194]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 -- 172.21.15.45:0/2771329448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1032e0 msgr2=0x7f976c10b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 --2- 172.21.15.45:0/2771329448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1032e0 0x7f976c10b7c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9760009900 tx=0x7f976002a510 comp rx=0 tx=0).stop 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 -- 172.21.15.45:0/2771329448 shutdown_connections 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 --2- 172.21.15.45:0/2771329448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c10bd00 0x7f976c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 --2- 172.21.15.45:0/2771329448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1032e0 0x7f976c10b7c0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 --2- 172.21.15.45:0/2771329448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c1029c0 0x7f976c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 -- 172.21.15.45:0/2771329448 >> 172.21.15.45:0/2771329448 conn(0x7f976c077740 msgr2=0x7f976c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 -- 172.21.15.45:0/2771329448 shutdown_connections 2024-04-25T00:31:17.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.417+0000 7f9771ba5640 1 -- 172.21.15.45:0/2771329448 wait complete. 2024-04-25T00:31:17.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 Processor -- start 2024-04-25T00:31:17.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 -- start start 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 0x7f976c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c10bd00 0x7f976c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f976c111b00 con 0x7f976c1029c0 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f976c111980 con 0x7f976c1032e0 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f9771ba5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f976c111c80 con 0x7f976c10bd00 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57950/0 (socket says 172.21.15.45:57950) 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 -- 172.21.15.45:0/949622207 learned_addr learned my addr 172.21.15.45:0/949622207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976affd640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 0x7f976c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976bfff640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c10bd00 0x7f976c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 -- 172.21.15.45:0/949622207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c10bd00 msgr2=0x7f976c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c10bd00 0x7f976c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 -- 172.21.15.45:0/949622207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 msgr2=0x7f976c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 0x7f976c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976b7fe640 1 -- 172.21.15.45:0/949622207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c19d010 con 0x7f976c1029c0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.418+0000 7f976affd640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 0x7f976c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f976b7fe640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f9768ff9640 1 -- 172.21.15.45:0/949622207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c017070 con 0x7f976c1029c0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c19d300 con 0x7f976c1029c0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f9768ff9640 1 -- 172.21.15.45:0/949622207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c005500 con 0x7f976c1029c0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f9768ff9640 1 -- 172.21.15.45:0/949622207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c004e00 con 0x7f976c1029c0 2024-04-25T00:31:17.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.419+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1a4b90 con 0x7f976c1029c0 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 Processor -- start 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f9768ff9640 1 -- 172.21.15.45:0/949622207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f975c004fa0 con 0x7f976c1029c0 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 -- start start 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e36cd452c0 con 0x55e36cdec010 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e36cced910 con 0x55e36cde2980 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.422+0000 7f977498f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e36cd41280 con 0x55e36cde1290 2024-04-25T00:31:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57958/0 (socket says 172.21.15.45:57958) 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976affd640 1 -- 172.21.15.45:0/487079293 learned_addr learned my addr 172.21.15.45:0/487079293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f9768ff9640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 0x7f97540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 -- 172.21.15.45:0/487079293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 msgr2=0x55e36cde1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 -- 172.21.15.45:0/487079293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 msgr2=0x55e36cdee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 -- 172.21.15.45:0/487079293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36ccfefb0 con 0x55e36cde2980 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976affd640 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f9768ff9640 1 -- 172.21.15.45:0/949622207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f975c0cf3a0 con 0x7f976c1029c0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 0x7f97540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f976bfff640 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde0cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9764009f90 tx=0x7f9764010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf5c3fa4e56a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.423+0000 7f9751ffb640 1 -- 172.21.15.45:0/487079293 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764024070 con 0x55e36cde2980 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f976bfff640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 0x7f97540a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9764021000 tx=0x7f976401e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f9751ffb640 1 -- 172.21.15.45:0/487079293 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976400ed70 con 0x55e36cde2980 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f9751ffb640 1 -- 172.21.15.45:0/487079293 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976401edf0 con 0x55e36cde2980 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 -- 172.21.15.45:0/487079293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 msgr2=0x55e36cde0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde0cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9764009f90 tx=0x7f9764010860 comp rx=0 tx=0).stop 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 -- 172.21.15.45:0/487079293 shutdown_connections 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cdee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 --2- 172.21.15.45:0/487079293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde0cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 -- 172.21.15.45:0/487079293 >> 172.21.15.45:0/487079293 conn(0x55e36cda9950 msgr2=0x55e36cddd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 -- 172.21.15.45:0/487079293 shutdown_connections 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.424+0000 7f977498f740 1 -- 172.21.15.45:0/487079293 wait complete. 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 -- start start 2024-04-25T00:31:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 Processor -- start 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e36cdeeea0 con 0x55e36cdec010 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e36cd00e90 con 0x55e36cde2980 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e36cdef020 con 0x55e36cde1290 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f977498f740 10 client.0 _ll_register_callbacks cb 0x55e36cd3c970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f976bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f976bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57972/0 (socket says 172.21.15.45:57972) 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f976bfff640 1 -- 172.21.15.45:0/786384183 learned_addr learned my addr 172.21.15.45:0/786384183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.425+0000 7f976affd640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976b7fe640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 -- 172.21.15.45:0/786384183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 msgr2=0x55e36cde8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 -- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 msgr2=0x55e36cde94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 -- 172.21.15.45:0/786384183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36cb217e0 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f976bfff640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f976401e930 tx=0x7f9764004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afce9dc28c76e33f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f974affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764024040 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f976400eaa0 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976400ed70 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9764039050 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f977498f740 10 client.18321 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e36cb81f70 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.426+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976400ed70 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97640040c0 con 0x55e36cdec010 2024-04-25T00:31:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 10 client.18321 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 20 client.18321 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36cdfc050 0x55e36cdfe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 4 client.18321 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36cd195a0 con 0x55e36cdfc050 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.427+0000 7f976b7fe640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36cdfc050 0x55e36cdfe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f976b7fe640 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36cdfc050 0x55e36cdfe440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c278ebda1a6d7 server_cookie=931c49187f970d06 in_seq=0 out_seq=0 2024-04-25T00:31:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f974affd640 10 client.18321 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f974affd640 1 -- 172.21.15.45:0/786384183 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e36cd195a0 con 0x55e36cdfc050 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f974affd640 10 client.18321 handle_command_reply: tid=0 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f977498f740 1 client.18321 shutdown 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 msgr2=0x55e36cde9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f976401e930 tx=0x7f9764004ab0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.428+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 shutdown_connections 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36cdfc050 0x55e36cdfe440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36cde9a20 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cde94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 --2- 172.21.15.45:0/786384183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cde8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f974affd640 0 client.18321 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 >> 172.21.15.45:0/786384183 conn(0x55e36cda9950 msgr2=0x55e36cde3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 shutdown_connections 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 1 -- 172.21.15.45:0/786384183 wait complete. 2024-04-25T00:31:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.429+0000 7f977498f740 20 client.18321 trim_cache size 0 max 16384 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.446+0000 7f977498f740 1 Processor -- start 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.446+0000 7f977498f740 1 -- start start 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cdf7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdeaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1aec60 con 0x55e36cdec010 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b07a0 con 0x55e36cde2980 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f977498f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1af330 con 0x55e36cde1290 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57978/0 (socket says 172.21.15.45:57978) 2024-04-25T00:31:17.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 -- 172.21.15.45:0/3121544283 learned_addr learned my addr 172.21.15.45:0/3121544283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976affd640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdeaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976b7fe640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cdf7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 -- 172.21.15.45:0/3121544283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 msgr2=0x55e36cdf7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cdf7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 -- 172.21.15.45:0/3121544283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 msgr2=0x55e36cdeaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdeaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 -- 172.21.15.45:0/3121544283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36ca19170 con 0x55e36cdec010 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976affd640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdeaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f976bfff640 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9764009a60 tx=0x7f9764010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899ec01e045a6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f97517fa640 1 -- 172.21.15.45:0/3121544283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764042070 con 0x55e36cdec010 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f97517fa640 1 -- 172.21.15.45:0/3121544283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976402c030 con 0x55e36cdec010 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.447+0000 7f97517fa640 1 -- 172.21.15.45:0/3121544283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764024040 con 0x55e36cdec010 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- 172.21.15.45:0/3121544283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 msgr2=0x55e36ce03c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9764009a60 tx=0x7f9764010d60 comp rx=0 tx=0).stop 2024-04-25T00:31:17.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- 172.21.15.45:0/3121544283 shutdown_connections 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cdec010 0x55e36ce03c50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdeaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 --2- 172.21.15.45:0/3121544283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cde1290 0x55e36cdf7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- 172.21.15.45:0/3121544283 >> 172.21.15.45:0/3121544283 conn(0x55e36cda9950 msgr2=0x55e36cdea7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- 172.21.15.45:0/3121544283 shutdown_connections 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- 172.21.15.45:0/3121544283 wait complete. 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.448+0000 7f977498f740 1 -- start start 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 Processor -- start 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdf1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 0x55e36cdf1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b00d0 con 0x55e36cde1290 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1afa00 con 0x55e36cde2980 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1acc60 con 0x55e36cdec010 2024-04-25T00:31:17.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f977498f740 10 client.0 _ll_register_callbacks cb 0x7f9764022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57982/0 (socket says 172.21.15.45:57982) 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 -- 172.21.15.45:0/887000358 learned_addr learned my addr 172.21.15.45:0/887000358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976bfff640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 0x55e36cdf1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976affd640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdf1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 -- 172.21.15.45:0/887000358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 msgr2=0x55e36cdf1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 0x55e36cdf1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 -- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 msgr2=0x55e36cdf1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdf1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 -- 172.21.15.45:0/887000358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36cdf26d0 con 0x55e36cde1290 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976bfff640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 0x55e36cdf1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976affd640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdf1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f976b7fe640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f975c09b1a0 tx=0x7f975c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946b3a9245fa49a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.449+0000 7f974affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:17.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c020040 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f975c0997a0 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976402c030 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f975c097740 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 10 client.18327 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e36ce624a0 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976402c030 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f975c084030 con 0x55e36cde1290 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 10 client.18327 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 20 client.18327 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36ce62bc0 0x55e36ce64fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 4 client.18327 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e36cd52800 con 0x55e36ce62bc0 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.450+0000 7f976affd640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36ce62bc0 0x55e36ce64fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.451+0000 7f974affd640 10 client.18327 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.451+0000 7f976affd640 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36ce62bc0 0x55e36ce64fe0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad2ceace0b6186d server_cookie=664518a66b9804c7 in_seq=0 out_seq=0 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f974affd640 1 -- 172.21.15.45:0/887000358 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e36cd52800 con 0x55e36ce62bc0 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f974affd640 10 client.18327 handle_command_reply: tid=0 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 client.18327 shutdown 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 msgr2=0x55e36cdf4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f975c09b1a0 tx=0x7f975c009c30 comp rx=0 tx=0).stop 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 shutdown_connections 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e36cdec010 0x55e36cdf1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e36cde2980 0x55e36cdf1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55e36ce62bc0 0x55e36ce64fe0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 --2- 172.21.15.45:0/887000358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e36cde1290 0x55e36cdf4620 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 >> 172.21.15.45:0/887000358 conn(0x55e36cda9950 msgr2=0x55e36cddc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f974affd640 0 client.18327 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 shutdown_connections 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 1 -- 172.21.15.45:0/887000358 wait complete. 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.452+0000 7f977498f740 20 client.18327 trim_cache size 0 max 16384 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "6fadd591-8cf1-462b-878c-86b4b751da28", 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:31:17.459 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 msgr2=0x7f97540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 0x7f97540a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9764021000 tx=0x7f976401e000 comp rx=0 tx=0).stop 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 msgr2=0x7f976c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 shutdown_connections 2024-04-25T00:31:17.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f97540a2d00 0x7f97540a51c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f976c10bd00 0x7f976c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f976c1032e0 0x7f976c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 --2- 172.21.15.45:0/949622207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f976c1029c0 0x7f976c198020 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 >> 172.21.15.45:0/949622207 conn(0x7f976c077740 msgr2=0x7f976c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 shutdown_connections 2024-04-25T00:31:17.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.453+0000 7f9771ba5640 1 -- 172.21.15.45:0/949622207 wait complete. 2024-04-25T00:31:17.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:31:17.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:31:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:17 smithi122 ceph-mon[32959]: pgmap v287: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T00:31:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:17 smithi122 ceph-mon[32959]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:31:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 -- 172.21.15.45:0/3691923263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644100fd0 msgr2=0x7f764410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 --2- 172.21.15.45:0/3691923263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644100fd0 0x7f764410d4e0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f763c009cd0 tx=0x7f763c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 -- 172.21.15.45:0/3691923263 shutdown_connections 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 --2- 172.21.15.45:0/3691923263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644100fd0 0x7f764410d4e0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 --2- 172.21.15.45:0/3691923263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100610 0x7f7644100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 --2- 172.21.15.45:0/3691923263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76441066b0 0x7f7644106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 -- 172.21.15.45:0/3691923263 >> 172.21.15.45:0/3691923263 conn(0x7f7644077740 msgr2=0x7f7644077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 -- 172.21.15.45:0/3691923263 shutdown_connections 2024-04-25T00:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 -- 172.21.15.45:0/3691923263 wait complete. 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.571+0000 7f764889a640 1 Processor -- start 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 -- start start 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644100610 0x7f764410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100fd0 0x7f764410e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7644111b30 con 0x7f76441066b0 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76441119b0 con 0x7f7644100fd0 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764889a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7644111cb0 con 0x7f7644100610 2024-04-25T00:31:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58008/0 (socket says 172.21.15.45:58008) 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 -- 172.21.15.45:0/841034985 learned_addr learned my addr 172.21.15.45:0/841034985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7639d9b640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100fd0 0x7f764410e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f764259c640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644100610 0x7f764410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 -- 172.21.15.45:0/841034985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644100610 msgr2=0x7f764410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644100610 0x7f764410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 -- 172.21.15.45:0/841034985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100fd0 msgr2=0x7f764410e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100fd0 0x7f764410e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.572+0000 7f7642d9d640 1 -- 172.21.15.45:0/841034985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441a6410 con 0x7f76441066b0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f7642d9d640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f763c005cc0 tx=0x7f763c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f763bfff640 1 -- 172.21.15.45:0/841034985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c004040 con 0x7f76441066b0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441a66a0 con 0x7f76441066b0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f763bfff640 1 -- 172.21.15.45:0/841034985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c0046c0 con 0x7f76441066b0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f763bfff640 1 -- 172.21.15.45:0/841034985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c046480 con 0x7f76441066b0 2024-04-25T00:31:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.573+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a6c40 con 0x7f76441066b0 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.575+0000 7f763bfff640 1 -- 172.21.15.45:0/841034985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f763c0305a0 con 0x7f76441066b0 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.575+0000 7f764b5a2740 1 Processor -- start 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 -- start start 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50630 0x556c9ae50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556c9adb56b0 con 0x556c9ae50f50 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c9ad5d880 con 0x556c9ae5b620 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764b5a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c9adb1670 con 0x556c9ae50630 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f763bfff640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 0x7f76300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58018/0 (socket says 172.21.15.45:58018) 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 -- 172.21.15.45:0/3440039139 learned_addr learned my addr 172.21.15.45:0/3440039139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f7642d9d640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 0x7f76300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f7639d9b640 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f7639d9b640 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50630 0x556c9ae50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f763bfff640 1 -- 172.21.15.45:0/841034985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f763c02e030 con 0x7f76441066b0 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 -- 172.21.15.45:0/3440039139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50630 msgr2=0x556c9ae50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50630 0x556c9ae50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 -- 172.21.15.45:0/3440039139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 msgr2=0x556c9ae5da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae5da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.576+0000 7f764259c640 1 -- 172.21.15.45:0/3440039139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c9ab937d0 con 0x556c9ae50f50 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f764259c640 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f763400ab70 tx=0x7f763400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0d237494f2e650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f7642d9d640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 0x7f76300a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ee3d0 tx=0x7f763c0dd040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f7638d99640 1 -- 172.21.15.45:0/3440039139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004220 con 0x556c9ae50f50 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f7638d99640 1 -- 172.21.15.45:0/3440039139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76340043c0 con 0x556c9ae50f50 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f7638d99640 1 -- 172.21.15.45:0/3440039139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340187e0 con 0x556c9ae50f50 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f764b5a2740 1 -- 172.21.15.45:0/3440039139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 msgr2=0x556c9ae5b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f763400ab70 tx=0x7f763400f790 comp rx=0 tx=0).stop 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.577+0000 7f764b5a2740 1 -- 172.21.15.45:0/3440039139 shutdown_connections 2024-04-25T00:31:17.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae5da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50f50 0x556c9ae5b0e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3440039139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50630 0x556c9ae50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- 172.21.15.45:0/3440039139 >> 172.21.15.45:0/3440039139 conn(0x556c9ae4c950 msgr2=0x556c9ae4cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- 172.21.15.45:0/3440039139 shutdown_connections 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- 172.21.15.45:0/3440039139 wait complete. 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- start start 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 Processor -- start 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50f50 0x556c9ae58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae5b620 0x556c9ae58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556c9ae5e410 con 0x556c9ae50630 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c9ad70f50 con 0x556c9ae50f50 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c9ae5e590 con 0x556c9ae5b620 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764b5a2740 10 client.0 _ll_register_callbacks cb 0x556c9adac000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58032/0 (socket says 172.21.15.45:58032) 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f7639d9b640 1 -- 172.21.15.45:0/974931303 learned_addr learned my addr 172.21.15.45:0/974931303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.578+0000 7f764259c640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50f50 0x556c9ae58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae5b620 0x556c9ae58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 -- 172.21.15.45:0/974931303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae5b620 msgr2=0x556c9ae58fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae5b620 0x556c9ae58fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 -- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50f50 msgr2=0x556c9ae58a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50f50 0x556c9ae58a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 -- 172.21.15.45:0/974931303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9aac93f0 con 0x556c9ae50630 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7639d9b640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c010 tx=0x7f762c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17677f26ba38d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c018070 con 0x556c9ae50630 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76340043c0 con 0x556c9ae50630 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f762c010d60 con 0x556c9ae50630 2024-04-25T00:31:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f762c028050 con 0x556c9ae50630 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f764b5a2740 10 client.18339 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c9aa890f0 con 0x556c9ae50630 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76340043c0 con 0x556c9ae50630 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.579+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f762c01f070 con 0x556c9ae50630 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 10 client.18339 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 20 client.18339 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9ae6b840 0x556c9ae6dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 4 client.18339 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c9adc2bf0 con 0x556c9ae6b840 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764259c640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9ae6b840 0x556c9ae6dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f764259c640 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9ae6b840 0x556c9ae6dc90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6dfcbd802379df server_cookie=eb39069cace1c227 in_seq=0 out_seq=0 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.580+0000 7f7629ffb640 10 client.18339 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f7629ffb640 1 -- 172.21.15.45:0/974931303 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c9adc2bf0 con 0x556c9ae6b840 2024-04-25T00:31:17.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f7629ffb640 10 client.18339 handle_command_reply: tid=0 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 client.18339 shutdown 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 msgr2=0x556c9ae58540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c010 tx=0x7f762c004730 comp rx=0 tx=0).stop 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 shutdown_connections 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae5b620 0x556c9ae58fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50f50 0x556c9ae58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9ae6b840 0x556c9ae6dc90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 --2- 172.21.15.45:0/974931303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae58540 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f7629ffb640 0 client.18339 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 >> 172.21.15.45:0/974931303 conn(0x556c9ae4c950 msgr2=0x556c9ae52750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 shutdown_connections 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 1 -- 172.21.15.45:0/974931303 wait complete. 2024-04-25T00:31:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.581+0000 7f764b5a2740 20 client.18339 trim_cache size 0 max 16384 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 Processor -- start 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 -- start start 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae4cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b0dd0 con 0x556c9ae50630 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b2910 con 0x556c9ae5b620 2024-04-25T00:31:17.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f764b5a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b14a0 con 0x556c9ae50f50 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58046/0 (socket says 172.21.15.45:58046) 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.599+0000 7f7639d9b640 1 -- 172.21.15.45:0/3217938483 learned_addr learned my addr 172.21.15.45:0/3217938483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f764259c640 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 -- 172.21.15.45:0/3217938483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 msgr2=0x556c9ae52ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae52ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 -- 172.21.15.45:0/3217938483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 msgr2=0x556c9ae4cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae4cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 -- 172.21.15.45:0/3217938483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c9ae46f00 con 0x556c9ae50630 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f7639d9b640 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f762c00bfb0 tx=0x7f762c0124c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab65c23e1dd29cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f762bfff640 1 -- 172.21.15.45:0/3217938483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c018070 con 0x556c9ae50630 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f762bfff640 1 -- 172.21.15.45:0/3217938483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762c01c040 con 0x556c9ae50630 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.600+0000 7f762bfff640 1 -- 172.21.15.45:0/3217938483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c003fa0 con 0x556c9ae50630 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- 172.21.15.45:0/3217938483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 msgr2=0x556c9ae672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f762c00bfb0 tx=0x7f762c0124c0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- 172.21.15.45:0/3217938483 shutdown_connections 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae5b620 0x556c9ae4cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- 172.21.15.45:0/3217938483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae50630 0x556c9ae672d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- 172.21.15.45:0/3217938483 >> 172.21.15.45:0/3217938483 conn(0x556c9ae4c950 msgr2=0x556c9ae58500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- 172.21.15.45:0/3217938483 shutdown_connections 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- 172.21.15.45:0/3217938483 wait complete. 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- start start 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 Processor -- start 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50630 0x556c9ae5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae5ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b2240 con 0x556c9ae5b620 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b1b70 con 0x556c9ae50630 2024-04-25T00:31:17.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76441aedd0 con 0x556c9ae50f50 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f764b5a2740 10 client.0 _ll_register_callbacks cb 0x7f762c00b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f7639d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58050/0 (socket says 172.21.15.45:58050) 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.601+0000 7f7639d9b640 1 -- 172.21.15.45:0/4261027372 learned_addr learned my addr 172.21.15.45:0/4261027372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50630 0x556c9ae5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f764259c640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae5ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 -- 172.21.15.45:0/4261027372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 msgr2=0x556c9ae5ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae5ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 -- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50630 msgr2=0x556c9ae5ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50630 0x556c9ae5ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 -- 172.21.15.45:0/4261027372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9aeca350 con 0x556c9ae5b620 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7639d9b640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b4e0 tx=0x7f762c003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ff6428cc18825 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7629ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:17.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c018070 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f762c01c040 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f762c010d70 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f762c005230 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f764b5a2740 10 client.18345 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c9aed1710 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.602+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762c01c040 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f762c01f050 con 0x556c9ae5b620 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 10 client.18345 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 20 client.18345 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9aed1cd0 0x556c9aed40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 4 client.18345 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c9adbcf40 con 0x556c9aed1cd0 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764259c640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9aed1cd0 0x556c9aed40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f7629ffb640 10 client.18345 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764259c640 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9aed1cd0 0x556c9aed40c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4006e8ca0b48f87 server_cookie=e355c5709ad75f58 in_seq=0 out_seq=0 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f7629ffb640 1 -- 172.21.15.45:0/4261027372 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c9adbcf40 con 0x556c9aed1cd0 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f7629ffb640 10 client.18345 handle_command_reply: tid=0 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.603+0000 7f764b5a2740 1 client.18345 shutdown 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 msgr2=0x556c9aec9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f762c00b4e0 tx=0x7f762c003870 comp rx=0 tx=0).stop 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 shutdown_connections 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x556c9aed1cd0 0x556c9aed40c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c9ae5b620 0x556c9aec9d90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c9ae50f50 0x556c9ae5ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f7629ffb640 0 client.18345 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 --2- 172.21.15.45:0/4261027372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c9ae50630 0x556c9ae5ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 >> 172.21.15.45:0/4261027372 conn(0x556c9ae4c950 msgr2=0x556c9ae58d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 shutdown_connections 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 1 -- 172.21.15.45:0/4261027372 wait complete. 2024-04-25T00:31:17.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764b5a2740 20 client.18345 trim_cache size 0 max 16384 2024-04-25T00:31:17.611 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:31:17.611 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:31:17.611 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 msgr2=0x7f76300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 0x7f76300a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ee3d0 tx=0x7f763c0dd040 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 msgr2=0x7f764410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f763c005cc0 tx=0x7f763c003970 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 shutdown_connections 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f76300a2d00 0x7f76300a51c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76441066b0 0x7f764410e9f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7644100fd0 0x7f764410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 --2- 172.21.15.45:0/841034985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644100610 0x7f764410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 >> 172.21.15.45:0/841034985 conn(0x7f7644077740 msgr2=0x7f76440ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 shutdown_connections 2024-04-25T00:31:17.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.604+0000 7f764889a640 1 -- 172.21.15.45:0/841034985 wait complete. 2024-04-25T00:31:17.615 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:31:17.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- 172.21.15.45:0/3015195348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c8107750 msgr2=0x7f01c8107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/3015195348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c8107750 0x7f01c8107b30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009900 tx=0x7f01b802a480 comp rx=0 tx=0).stop 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- 172.21.15.45:0/3015195348 shutdown_connections 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/3015195348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 0x7f01c810e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/3015195348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c81017c0 0x7f01c8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/3015195348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c8107750 0x7f01c8107b30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- 172.21.15.45:0/3015195348 >> 172.21.15.45:0/3015195348 conn(0x7f01c80788d0 msgr2=0x7f01c8078ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- 172.21.15.45:0/3015195348 shutdown_connections 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- 172.21.15.45:0/3015195348 wait complete. 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 Processor -- start 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.717+0000 7f01cd5e7640 1 -- start start 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 0x7f01c8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c8107750 0x7f01c819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8111b10 con 0x7f01c81017c0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8111990 con 0x7f01c8107750 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01cd5e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8111c90 con 0x7f01c8102160 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58062/0 (socket says 172.21.15.45:58062) 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 -- 172.21.15.45:0/2351165591 learned_addr learned my addr 172.21.15.45:0/2351165591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c77fe640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c8107750 0x7f01c819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c67fc640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 0x7f01c8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 -- 172.21.15.45:0/2351165591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 msgr2=0x7f01c8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 0x7f01c8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 -- 172.21.15.45:0/2351165591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c8107750 msgr2=0x7f01c819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c8107750 0x7f01c819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 -- 172.21.15.45:0/2351165591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c819cfe0 con 0x7f01c81017c0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01c6ffd640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006010 tx=0x7f01b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.718+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b803f030 con 0x7f01c81017c0 2024-04-25T00:31:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.719+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c819d270 con 0x7f01c81017c0 2024-04-25T00:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.719+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8003ce0 con 0x7f01c81017c0 2024-04-25T00:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.719+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8035070 con 0x7f01c81017c0 2024-04-25T00:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.719+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c81a4b60 con 0x7f01c81017c0 2024-04-25T00:31:17.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.719+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c80689b0 con 0x7f01c81017c0 2024-04-25T00:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.721+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f01b8042030 con 0x7f01c81017c0 2024-04-25T00:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.721+0000 7f01affff640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 0x7f01b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.722+0000 7f01c67fc640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 0x7f01b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.722+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f01b80effc0 con 0x7f01c81017c0 2024-04-25T00:31:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.722+0000 7f01c67fc640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 0x7f01b00a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01c8078f40 tx=0x7f01bc00b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.725+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01b80bc290 con 0x7f01c81017c0 2024-04-25T00:31:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.952+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01c8109bf0 con 0x7f01c81017c0 2024-04-25T00:31:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.953+0000 7f01affff640 1 -- 172.21.15.45:0/2351165591 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f01b80c0590 con 0x7f01c81017c0 2024-04-25T00:31:17.953 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:31:17.953 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:31:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.954+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 msgr2=0x7f01b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.954+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 0x7f01b00a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01c8078f40 tx=0x7f01bc00b390 comp rx=0 tx=0).stop 2024-04-25T00:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.954+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 msgr2=0x7f01c8197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.954+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006010 tx=0x7f01b80042f0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.954+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 shutdown_connections 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f01b00a2d00 0x7f01b00a51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01c8107750 0x7f01c819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01c8102160 0x7f01c8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 --2- 172.21.15.45:0/2351165591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01c81017c0 0x7f01c8197ff0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 >> 172.21.15.45:0/2351165591 conn(0x7f01c80788d0 msgr2=0x7f01c81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 shutdown_connections 2024-04-25T00:31:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:17.955+0000 7f01cd5e7640 1 -- 172.21.15.45:0/2351165591 wait complete. 2024-04-25T00:31:17.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:31:18.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- 172.21.15.45:0/1116512976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c81032f0 msgr2=0x7f57c810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- 172.21.15.45:0/1116512976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c81032f0 0x7f57c810b7d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009ca0 tx=0x7f57b802a9e0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- 172.21.15.45:0/1116512976 shutdown_connections 2024-04-25T00:31:18.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- 172.21.15.45:0/1116512976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c810bd10 0x7f57c810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- 172.21.15.45:0/1116512976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c81032f0 0x7f57c810b7d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- 172.21.15.45:0/1116512976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81029d0 0x7f57c8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- 172.21.15.45:0/1116512976 >> 172.21.15.45:0/1116512976 conn(0x7f57c80769c0 msgr2=0x7f57c8076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- 172.21.15.45:0/1116512976 shutdown_connections 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- 172.21.15.45:0/1116512976 wait complete. 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 Processor -- start 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- start start 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c81029d0 0x7f57c8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 0x7f57c81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8111b10 con 0x7f57c810bd10 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8111990 con 0x7f57c81029d0 2024-04-25T00:31:18.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.070+0000 7f57ceefc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8111c90 con 0x7f57c81032f0 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58080/0 (socket says 172.21.15.45:58080) 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 -- 172.21.15.45:0/975397301 learned_addr learned my addr 172.21.15.45:0/975397301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57ccc98640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c81029d0 0x7f57c8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57c7fff640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 0x7f57c81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 -- 172.21.15.45:0/975397301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 msgr2=0x7f57c81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 0x7f57c81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 -- 172.21.15.45:0/975397301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c81029d0 msgr2=0x7f57c8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c81029d0 0x7f57c8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 -- 172.21.15.45:0/975397301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c819d060 con 0x7f57c810bd10 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57c7fff640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 0x7f57c81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57cd499640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d950 tx=0x7f57c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57c5ffb640 1 -- 172.21.15.45:0/975397301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0017030 con 0x7f57c810bd10 2024-04-25T00:31:18.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c819d350 con 0x7f57c810bd10 2024-04-25T00:31:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57c5ffb640 1 -- 172.21.15.45:0/975397301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c0004020 con 0x7f57c810bd10 2024-04-25T00:31:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.071+0000 7f57c5ffb640 1 -- 172.21.15.45:0/975397301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003c00 con 0x7f57c810bd10 2024-04-25T00:31:18.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.072+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81a4be0 con 0x7f57c810bd10 2024-04-25T00:31:18.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.073+0000 7f57c5ffb640 1 -- 172.21.15.45:0/975397301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f57c001a070 con 0x7f57c810bd10 2024-04-25T00:31:18.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 Processor -- start 2024-04-25T00:31:18.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 -- start start 2024-04-25T00:31:18.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57c5ffb640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 0x7f57b40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8ea0 0x5636517c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56365171d670 con 0x5636517b8ea0 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5636516c5970 con 0x5636517b8580 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.074+0000 7f57d1c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563651719630 con 0x5636517c3570 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57c7fff640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 0x7f57b40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57c5ffb640 1 -- 172.21.15.45:0/975397301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f57c00cf080 con 0x7f57c810bd10 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8ea0 0x5636517c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35682/0 (socket says 172.21.15.45:35682) 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 -- 172.21.15.45:0/2351390333 learned_addr learned my addr 172.21.15.45:0/2351390333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57ccc98640 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 -- 172.21.15.45:0/2351390333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 msgr2=0x5636517b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 -- 172.21.15.45:0/2351390333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8ea0 msgr2=0x5636517c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8ea0 0x5636517c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 -- 172.21.15.45:0/2351390333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636516e9400 con 0x5636517c3570 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57c7fff640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 0x7f57b40a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f57c80fff30 tx=0x7f57b8009880 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57cd499640 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f57c0097b90 tx=0x7f57c009abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538d052bd06a5f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57beffd640 1 -- 172.21.15.45:0/2351390333 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c006f5f0 con 0x5636517c3570 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57beffd640 1 -- 172.21.15.45:0/2351390333 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c009b1c0 con 0x5636517c3570 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.075+0000 7f57beffd640 1 -- 172.21.15.45:0/2351390333 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00041d0 con 0x5636517c3570 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- 172.21.15.45:0/2351390333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 msgr2=0x5636517c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f57c0097b90 tx=0x7f57c009abe0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- 172.21.15.45:0/2351390333 shutdown_connections 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517c3570 0x5636517c5960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8ea0 0x5636517c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 --2- 172.21.15.45:0/2351390333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- 172.21.15.45:0/2351390333 >> 172.21.15.45:0/2351390333 conn(0x5636517b48a0 msgr2=0x5636517b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- 172.21.15.45:0/2351390333 shutdown_connections 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- 172.21.15.45:0/2351390333 wait complete. 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 -- start start 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.076+0000 7f57d1c09740 1 Processor -- start 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5636517c6360 con 0x5636517c3570 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5636516d9050 con 0x5636517b8580 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636517c64e0 con 0x5636517b8ea0 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57d1c09740 10 client.0 _ll_register_callbacks cb 0x563651430be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58104/0 (socket says 172.21.15.45:58104) 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 -- 172.21.15.45:0/570833043 learned_addr learned my addr 172.21.15.45:0/570833043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57c7fff640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57ccc98640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 -- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 msgr2=0x5636517c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 -- 172.21.15.45:0/570833043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 msgr2=0x5636517c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 -- 172.21.15.45:0/570833043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636516aadc0 con 0x5636517c3570 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57ccc98640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57cd499640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009610 tx=0x7f57c00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f48a787b8e15b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.077+0000 7f57a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00a3e30 con 0x5636517c3570 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f57c00041d0 con 0x5636517c3570 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c009b1c0 con 0x5636517c3570 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57c009f570 con 0x5636517c3570 2024-04-25T00:31:18.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 10 client.18363 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636514313f0 con 0x5636517c3570 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c009b1c0 con 0x5636517c3570 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f57c00cc670 con 0x5636517c3570 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 10 client.18363 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 10 client.18363 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 20 client.18363 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636517d34d0 0x5636517d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 4 client.18363 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365172abb0 con 0x5636517d34d0 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.078+0000 7f57ccc98640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636517d34d0 0x5636517d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.079+0000 7f57ccc98640 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636517d34d0 0x5636517d58c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3fc5c5bf235cec92 server_cookie=56097669365595d9 in_seq=0 out_seq=0 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.079+0000 7f57a7fff640 10 client.18363 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57a7fff640 1 -- 172.21.15.45:0/570833043 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56365172abb0 con 0x5636517d34d0 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57a7fff640 10 client.18363 handle_command_reply: tid=0 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 client.18363 shutdown 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 msgr2=0x5636517c0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009610 tx=0x7f57c00045b0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 shutdown_connections 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636517d34d0 0x5636517d58c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517c0850 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57a7fff640 0 client.18363 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:18.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 --2- 172.21.15.45:0/570833043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 >> 172.21.15.45:0/570833043 conn(0x5636517b48a0 msgr2=0x5636517b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 shutdown_connections 2024-04-25T00:31:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 1 -- 172.21.15.45:0/570833043 wait complete. 2024-04-25T00:31:18.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.080+0000 7f57d1c09740 20 client.18363 trim_cache size 0 max 16384 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.094+0000 7f57d1c09740 1 Processor -- start 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 -- start start 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 0x5636517d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81aec90 con 0x5636517b8580 2024-04-25T00:31:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81b07d0 con 0x5636517c3570 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57d1c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81af360 con 0x5636517b8ea0 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58110/0 (socket says 172.21.15.45:58110) 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 -- 172.21.15.45:0/3859268096 learned_addr learned my addr 172.21.15.45:0/3859268096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57c7fff640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57cd499640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 0x5636517d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 -- 172.21.15.45:0/3859268096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 msgr2=0x5636517d88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 -- 172.21.15.45:0/3859268096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 msgr2=0x5636517d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 0x5636517d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 -- 172.21.15.45:0/3859268096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636514fb7d0 con 0x5636517b8580 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57cd499640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 0x5636517d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57ccc98640 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ab7725418b68e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.095+0000 7f57be7fc640 1 -- 172.21.15.45:0/3859268096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0025070 con 0x5636517b8580 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57be7fc640 1 -- 172.21.15.45:0/3859268096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0003db0 con 0x5636517b8580 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57be7fc640 1 -- 172.21.15.45:0/3859268096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0010b00 con 0x5636517b8580 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 -- 172.21.15.45:0/3859268096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 msgr2=0x5636517c2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b00046c0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 -- 172.21.15.45:0/3859268096 shutdown_connections 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517c3570 0x5636517d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 --2- 172.21.15.45:0/3859268096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517b8580 0x5636517c2a80 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 -- 172.21.15.45:0/3859268096 >> 172.21.15.45:0/3859268096 conn(0x5636517b48a0 msgr2=0x5636517bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 -- 172.21.15.45:0/3859268096 shutdown_connections 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.096+0000 7f57d1c09740 1 -- 172.21.15.45:0/3859268096 wait complete. 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 -- start start 2024-04-25T00:31:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 Processor -- start 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517d8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81b0100 con 0x5636517c3570 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81afa30 con 0x5636517b8580 2024-04-25T00:31:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81acc90 con 0x5636517b8ea0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57d1c09740 10 client.0 _ll_register_callbacks cb 0x7f57b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58120/0 (socket says 172.21.15.45:58120) 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 -- 172.21.15.45:0/1200527556 learned_addr learned my addr 172.21.15.45:0/1200527556 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57c7fff640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57ccc98640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517d8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 -- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 msgr2=0x5636517d5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 -- 172.21.15.45:0/1200527556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 msgr2=0x5636517d8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.097+0000 7f57cd499640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517d8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57cd499640 1 -- 172.21.15.45:0/1200527556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636517d6bb0 con 0x5636517c3570 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57ccc98640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517d8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57cd499640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f57c009f730 tx=0x7f57c0004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503863460b1add1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c009ad00 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f57c00ccb40 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b0003db0 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57c00cc880 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 10 client.18369 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636517d6bb0 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0003db0 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f57c009a610 con 0x5636517c3570 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 10 client.18369 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 10 client.18369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 20 client.18369 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636518398b0 0x56365183bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 4 client.18369 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.098+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563651724f00 con 0x5636518398b0 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57ccc98640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636518398b0 0x56365183bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57ccc98640 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636518398b0 0x56365183bcc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa57adfcbb64950f server_cookie=8170a9a211613f25 in_seq=0 out_seq=0 2024-04-25T00:31:18.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57a7fff640 10 client.18369 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57a7fff640 1 -- 172.21.15.45:0/1200527556 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563651724f00 con 0x5636518398b0 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57a7fff640 10 client.18369 handle_command_reply: tid=0 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 client.18369 shutdown 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 msgr2=0x5636517d64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f57c009f730 tx=0x7f57c0004400 comp rx=0 tx=0).stop 2024-04-25T00:31:18.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 shutdown_connections 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5636518398b0 0x56365183bcc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.099+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636517c3570 0x5636517d64c0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636517b8ea0 0x5636517d5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57a7fff640 0 client.18369 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 1 --2- 172.21.15.45:0/1200527556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636517b8580 0x5636517d8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 >> 172.21.15.45:0/1200527556 conn(0x5636517b48a0 msgr2=0x5636517be790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 shutdown_connections 2024-04-25T00:31:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 1 -- 172.21.15.45:0/1200527556 wait complete. 2024-04-25T00:31:18.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57d1c09740 20 client.18369 trim_cache size 0 max 16384 2024-04-25T00:31:18.108 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 msgr2=0x7f57b40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 0x7f57b40a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f57c80fff30 tx=0x7f57b8009880 comp rx=0 tx=0).stop 2024-04-25T00:31:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 msgr2=0x7f57c819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d950 tx=0x7f57c000de10 comp rx=0 tx=0).stop 2024-04-25T00:31:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 shutdown_connections 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f57b40a2d00 0x7f57b40a51c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c810bd10 0x7f57c819c930 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57c81032f0 0x7f57c81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 --2- 172.21.15.45:0/975397301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c81029d0 0x7f57c8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 >> 172.21.15.45:0/975397301 conn(0x7f57c80769c0 msgr2=0x7f57c80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 shutdown_connections 2024-04-25T00:31:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.100+0000 7f57ceefc640 1 -- 172.21.15.45:0/975397301 wait complete. 2024-04-25T00:31:18.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 -- 172.21.15.45:0/3533149892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 msgr2=0x7f78f010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 --2- 172.21.15.45:0/3533149892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f010e5f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009cd0 tx=0x7f78e002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 -- 172.21.15.45:0/3533149892 shutdown_connections 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 --2- 172.21.15.45:0/3533149892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f010e5f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 --2- 172.21.15.45:0/3533149892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 0x7f78f006a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 --2- 172.21.15.45:0/3533149892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f0107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 -- 172.21.15.45:0/3533149892 >> 172.21.15.45:0/3533149892 conn(0x7f78f00776a0 msgr2=0x7f78f0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 -- 172.21.15.45:0/3533149892 shutdown_connections 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.217+0000 7f78f79d5640 1 -- 172.21.15.45:0/3533149892 wait complete. 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 Processor -- start 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 -- start start 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 0x7f78f0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0111b40 con 0x7f78f006a780 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01119c0 con 0x7f78f0107730 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f79d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0111cc0 con 0x7f78f0069dc0 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58146/0 (socket says 172.21.15.45:58146) 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 -- 172.21.15.45:0/421654785 learned_addr learned my addr 172.21.15.45:0/421654785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f5771640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 0x7f78f0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f5f72640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 -- 172.21.15.45:0/421654785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 msgr2=0x7f78f0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 0x7f78f0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 -- 172.21.15.45:0/421654785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 msgr2=0x7f78f019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f4f70640 1 -- 172.21.15.45:0/421654785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f019ceb0 con 0x7f78f006a780 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.218+0000 7f78f5f72640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78f4f70640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c920 tx=0x7f78e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78ee7fc640 1 -- 172.21.15.45:0/421654785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4007c70 con 0x7f78f006a780 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019d1a0 con 0x7f78f006a780 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78ee7fc640 1 -- 172.21.15.45:0/421654785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e4007e10 con 0x7f78f006a780 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78ee7fc640 1 -- 172.21.15.45:0/421654785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4018890 con 0x7f78f006a780 2024-04-25T00:31:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.219+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f01a4a30 con 0x7f78f006a780 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.221+0000 7f78ee7fc640 1 -- 172.21.15.45:0/421654785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f78e4018a70 con 0x7f78f006a780 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.221+0000 7f78fa7b7740 1 Processor -- start 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.221+0000 7f78fa7b7740 1 -- start start 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d585960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d592960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c10d4ea670 con 0x55c10d585ea0 2024-04-25T00:31:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c10d492970 con 0x55c10d590570 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c10d4e6630 con 0x55c10d585580 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78ee7fc640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 0x7f78d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58148/0 (socket says 172.21.15.45:58148) 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 -- 172.21.15.45:0/519230960 learned_addr learned my addr 172.21.15.45:0/519230960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f4f70640 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d592960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5f72640 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d585960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5f72640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 0x7f78d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78ee7fc640 1 -- 172.21.15.45:0/421654785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f78e40cfa10 con 0x7f78f006a780 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 -- 172.21.15.45:0/519230960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 msgr2=0x55c10d585960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d585960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 -- 172.21.15.45:0/519230960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 msgr2=0x55c10d592960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d592960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 -- 172.21.15.45:0/519230960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10d4b6400 con 0x55c10d585ea0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5771640 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f78e800bc60 tx=0x7f78e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd689ce8e88bf98a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.222+0000 7f78f5f72640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 0x7f78d80a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78e0004af0 tx=0x7f78e00095c0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78d77fe640 1 -- 172.21.15.45:0/519230960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8015070 con 0x55c10d585ea0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78d77fe640 1 -- 172.21.15.45:0/519230960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e8011920 con 0x55c10d585ea0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78d77fe640 1 -- 172.21.15.45:0/519230960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80108d0 con 0x55c10d585ea0 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 -- 172.21.15.45:0/519230960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 msgr2=0x55c10d590030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f78e800bc60 tx=0x7f78e800f730 comp rx=0 tx=0).stop 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 -- 172.21.15.45:0/519230960 shutdown_connections 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d592960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d590030 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/519230960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d585960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 -- 172.21.15.45:0/519230960 >> 172.21.15.45:0/519230960 conn(0x55c10d5818a0 msgr2=0x55c10d581c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 -- 172.21.15.45:0/519230960 shutdown_connections 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.223+0000 7f78fa7b7740 1 -- 172.21.15.45:0/519230960 wait complete. 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 -- start start 2024-04-25T00:31:18.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 Processor -- start 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 0x55c10d59a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 0x55c10d594700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c10d593360 con 0x55c10d585ea0 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c10d4a6050 con 0x55c10d585580 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c10d5934e0 con 0x55c10d590570 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78fa7b7740 10 client.0 _ll_register_callbacks cb 0x55c10d1fdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58150/0 (socket says 172.21.15.45:58150) 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78f4f70640 1 -- 172.21.15.45:0/2295297293 learned_addr learned my addr 172.21.15.45:0/2295297293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.224+0000 7f78f5771640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 0x55c10d59a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f5f72640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 0x55c10d594700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 -- 172.21.15.45:0/2295297293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 msgr2=0x55c10d594700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 0x55c10d594700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 -- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 msgr2=0x55c10d59a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 0x55c10d59a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 -- 172.21.15.45:0/2295297293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10d477dc0 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f5771640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 0x55c10d59a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f5f72640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 0x55c10d594700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f78e409bb00 tx=0x7f78e406aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a6442dd70fcc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400de20 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f78e40981e0 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e8011920 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78e409bb70 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 10 client.18381 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c10d1fe3f0 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e8011920 con 0x55c10d585ea0 2024-04-25T00:31:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78e409afc0 con 0x55c10d585ea0 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 10 client.18381 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 10 client.18381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 20 client.18381 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d5a4960 0x55c10d5a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 4 client.18381 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.225+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c10d4f7bb0 con 0x55c10d5a4960 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.226+0000 7f78f5771640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d5a4960 0x55c10d5a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.226+0000 7f78d4ff9640 10 client.18381 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.226+0000 7f78f5771640 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d5a4960 0x55c10d5a6db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=464b2ec14aa9957d server_cookie=34e2b4f4cc22675e in_seq=0 out_seq=0 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78d4ff9640 1 -- 172.21.15.45:0/2295297293 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c10d4f7bb0 con 0x55c10d5a4960 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78d4ff9640 10 client.18381 handle_command_reply: tid=0 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 client.18381 shutdown 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 msgr2=0x55c10d59a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f78e409bb00 tx=0x7f78e406aaa0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 shutdown_connections 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d5a4960 0x55c10d5a6db0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d590570 0x55c10d594700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d59a8d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78d4ff9640 0 client.18381 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2295297293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585580 0x55c10d59a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 >> 172.21.15.45:0/2295297293 conn(0x55c10d5818a0 msgr2=0x55c10d580b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 shutdown_connections 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2295297293 wait complete. 2024-04-25T00:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.227+0000 7f78fa7b7740 20 client.18381 trim_cache size 0 max 16384 2024-04-25T00:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 Processor -- start 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 -- start start 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d58cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d5acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ac9a0 con 0x55c10d585ea0 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ae4e0 con 0x55c10d590570 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ad070 con 0x55c10d585580 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58152/0 (socket says 172.21.15.45:58152) 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 -- 172.21.15.45:0/2789568462 learned_addr learned my addr 172.21.15.45:0/2789568462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f5f72640 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d5acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 -- 172.21.15.45:0/2789568462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 msgr2=0x55c10d58cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d58cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 -- 172.21.15.45:0/2789568462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 msgr2=0x55c10d5acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d5acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 -- 172.21.15.45:0/2789568462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10d2c87d0 con 0x55c10d585ea0 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78f4f70640 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f78e4097fd0 tx=0x7f78e40cfd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3394265f3876ae50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.241+0000 7f78d6ffd640 1 -- 172.21.15.45:0/2789568462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e409ff20 con 0x55c10d585ea0 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78d6ffd640 1 -- 172.21.15.45:0/2789568462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e40b8070 con 0x55c10d585ea0 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78d6ffd640 1 -- 172.21.15.45:0/2789568462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e409add0 con 0x55c10d585ea0 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2789568462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 msgr2=0x55c10d5ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f78e4097fd0 tx=0x7f78e40cfd90 comp rx=0 tx=0).stop 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2789568462 shutdown_connections 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d590570 0x55c10d5acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d585ea0 0x55c10d5ac680 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/2789568462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d58cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2789568462 >> 172.21.15.45:0/2789568462 conn(0x55c10d5818a0 msgr2=0x55c10d59a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2789568462 shutdown_connections 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- 172.21.15.45:0/2789568462 wait complete. 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 -- start start 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.242+0000 7f78fa7b7740 1 Processor -- start 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585ea0 0x55c10d59dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 0x55c10d59e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ade10 con 0x55c10d590570 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ad740 con 0x55c10d585ea0 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01aa9a0 con 0x55c10d585580 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78fa7b7740 10 client.0 _ll_register_callbacks cb 0x7f78e400cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5f72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 0x55c10d59e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.45:35770/0 (socket says 172.21.15.45:35770) 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 -- 172.21.15.45:0/532140742 learned_addr learned my addr 172.21.15.45:0/532140742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f4f70640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585ea0 0x55c10d59dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 -- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585ea0 msgr2=0x55c10d59dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585ea0 0x55c10d59dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 -- 172.21.15.45:0/532140742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 msgr2=0x55c10d59e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 0x55c10d59e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5771640 1 -- 172.21.15.45:0/532140742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10d609f00 con 0x55c10d585580 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.243+0000 7f78f5f72640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 0x55c10d59e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78f5771640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78e80104e0 tx=0x7f78e800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c84cc3b52e73342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8015050 con 0x55c10d585580 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e40b8070 con 0x55c10d585580 2024-04-25T00:31:18.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f78e8004440 con 0x55c10d585580 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78e8005040 con 0x55c10d585580 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 10 client.27481 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c10d60b0a0 con 0x55c10d585580 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e40b8070 con 0x55c10d585580 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78e80035f0 con 0x55c10d585580 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 10 client.27481 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 10 client.27481 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 20 client.27481 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d60b6b0 0x55c10d60dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 4 client.27481 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.244+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c10d4f1f00 con 0x55c10d60b6b0 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78f4f70640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d60b6b0 0x55c10d60dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78f4f70640 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d60b6b0 0x55c10d60dad0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3fa53dc9f49dfa2 server_cookie=699097c4817720d4 in_seq=0 out_seq=0 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78d4ff9640 10 client.27481 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78d4ff9640 1 -- 172.21.15.45:0/532140742 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c10d4f1f00 con 0x55c10d60b6b0 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78d4ff9640 10 client.27481 handle_command_reply: tid=0 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 client.27481 shutdown 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 msgr2=0x55c10d5a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78e80104e0 tx=0x7f78e800bf10 comp rx=0 tx=0).stop 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 shutdown_connections 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55c10d60b6b0 0x55c10d60dad0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c10d590570 0x55c10d59e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c10d585ea0 0x55c10d59dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78d4ff9640 0 client.27481 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 --2- 172.21.15.45:0/532140742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c10d585580 0x55c10d5a0c40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 >> 172.21.15.45:0/532140742 conn(0x55c10d5818a0 msgr2=0x55c10d5802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 shutdown_connections 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 1 -- 172.21.15.45:0/532140742 wait complete. 2024-04-25T00:31:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.245+0000 7f78fa7b7740 20 client.27481 trim_cache size 0 max 16384 2024-04-25T00:31:18.254 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 msgr2=0x7f78d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 0x7f78d80a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78e0004af0 tx=0x7f78e00095c0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 msgr2=0x7f78f0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c920 tx=0x7f78e400cde0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 shutdown_connections 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f78d80a2d00 0x7f78d80a51c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78f0107730 0x7f78f019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78f006a780 0x7f78f0198400 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 --2- 172.21.15.45:0/421654785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f0069dc0 0x7f78f0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 >> 172.21.15.45:0/421654785 conn(0x7f78f00776a0 msgr2=0x7f78f00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 shutdown_connections 2024-04-25T00:31:18.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.246+0000 7f78f79d5640 1 -- 172.21.15.45:0/421654785 wait complete. 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:31:18.256 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:18 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2351165591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:18.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:31:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 -- 172.21.15.45:0/789099637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c100c90 msgr2=0x7f146c1063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 --2- 172.21.15.45:0/789099637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c100c90 0x7f146c1063c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f1460009900 tx=0x7f146002a410 comp rx=0 tx=0).stop 2024-04-25T00:31:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 -- 172.21.15.45:0/789099637 shutdown_connections 2024-04-25T00:31:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 --2- 172.21.15.45:0/789099637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c106aa0 0x7f146c10e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 --2- 172.21.15.45:0/789099637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c100c90 0x7f146c1063c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 --2- 172.21.15.45:0/789099637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c1002e0 0x7f146c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 -- 172.21.15.45:0/789099637 >> 172.21.15.45:0/789099637 conn(0x7f146c0758d0 msgr2=0x7f146c075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 -- 172.21.15.45:0/789099637 shutdown_connections 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.361+0000 7f1473658640 1 -- 172.21.15.45:0/789099637 wait complete. 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 Processor -- start 2024-04-25T00:31:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 -- start start 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c1002e0 0x7f146c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c19ca20 0x7f146c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111be0 con 0x7f146c106aa0 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111a60 con 0x7f146c1002e0 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1473658640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111d60 con 0x7f146c19ca20 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58194/0 (socket says 172.21.15.45:58194) 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1470bf3640 1 -- 172.21.15.45:0/2512838279 learned_addr learned my addr 172.21.15.45:0/2512838279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f14713f4640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c1002e0 0x7f146c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.362+0000 7f1471bf5640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c19ca20 0x7f146c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 -- 172.21.15.45:0/2512838279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c19ca20 msgr2=0x7f146c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c19ca20 0x7f146c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 -- 172.21.15.45:0/2512838279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c1002e0 msgr2=0x7f146c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c1002e0 0x7f146c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 -- 172.21.15.45:0/2512838279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c19d510 con 0x7f146c106aa0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1470bf3640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f146002a930 tx=0x7f1460003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f145e7fc640 1 -- 172.21.15.45:0/2512838279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146003f070 con 0x7f146c106aa0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f145e7fc640 1 -- 172.21.15.45:0/2512838279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1460031c90 con 0x7f146c106aa0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c19d7a0 con 0x7f146c106aa0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f145e7fc640 1 -- 172.21.15.45:0/2512838279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460035030 con 0x7f146c106aa0 2024-04-25T00:31:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.363+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1a4f60 con 0x7f146c106aa0 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:31:18.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:18 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2351165591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f145e7fc640 1 -- 172.21.15.45:0/2512838279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1460042070 con 0x7f146c106aa0 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 Processor -- start 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 -- start start 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56ea0 0x558d23f61030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d23ebb670 con 0x558d23f56580 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558d23e63970 con 0x558d23f56ea0 2024-04-25T00:31:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1476365740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558d23eb7630 con 0x558d23f61570 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.366+0000 7f1471bf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58198/0 (socket says 172.21.15.45:58198) 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 -- 172.21.15.45:0/3697874095 learned_addr learned my addr 172.21.15.45:0/3697874095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f145e7fc640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 0x7f14580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f14713f4640 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56ea0 0x558d23f61030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1470bf3640 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 0x7f14580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 -- 172.21.15.45:0/3697874095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 msgr2=0x558d23f63960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f145e7fc640 1 -- 172.21.15.45:0/2512838279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f14600eff30 con 0x7f146c106aa0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f63960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 -- 172.21.15.45:0/3697874095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56ea0 msgr2=0x558d23f61030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56ea0 0x558d23f61030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 -- 172.21.15.45:0/3697874095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d23e87400 con 0x558d23f56580 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1464014a60 tx=0x7f1464014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6524b30a843bd087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f14577fe640 1 -- 172.21.15.45:0/3697874095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146401ec50 con 0x558d23f56580 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f14577fe640 1 -- 172.21.15.45:0/3697874095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464023070 con 0x558d23f56580 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f14577fe640 1 -- 172.21.15.45:0/3697874095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146401de00 con 0x558d23f56580 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.367+0000 7f1471bf5640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 0x7f14580a51c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1464022000 tx=0x7f146401e000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- 172.21.15.45:0/3697874095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 msgr2=0x558d23f56960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1464014a60 tx=0x7f1464014f20 comp rx=0 tx=0).stop 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- 172.21.15.45:0/3697874095 shutdown_connections 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f63960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56ea0 0x558d23f61030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 --2- 172.21.15.45:0/3697874095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56580 0x558d23f56960 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- 172.21.15.45:0/3697874095 >> 172.21.15.45:0/3697874095 conn(0x558d23f528a0 msgr2=0x558d23f52c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- 172.21.15.45:0/3697874095 shutdown_connections 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- 172.21.15.45:0/3697874095 wait complete. 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 -- start start 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.368+0000 7f1476365740 1 Processor -- start 2024-04-25T00:31:18.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f60f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f5e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d23f64360 con 0x558d23f56ea0 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558d23e77050 con 0x558d23f56580 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558d23f644e0 con 0x558d23f61570 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1476365740 10 client.0 _ll_register_callbacks cb 0x558d23bcebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58210/0 (socket says 172.21.15.45:58210) 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 -- 172.21.15.45:0/1503033802 learned_addr learned my addr 172.21.15.45:0/1503033802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f14713f4640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f60f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1471bf5640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f5e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 -- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 msgr2=0x558d23f5e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f5e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 -- 172.21.15.45:0/1503033802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 msgr2=0x558d23f60f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f60f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 -- 172.21.15.45:0/1503033802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d23e48dc0 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1471bf5640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f5e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f14713f4640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f60f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1470bf3640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f1460009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81127f2a466d9014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.369+0000 7f1454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460049040 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f14600ed540 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1464023070 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14600bbaa0 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 10 client.18399 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d23bcf3f0 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464023070 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1460080060 con 0x558d23f56ea0 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 10 client.18399 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 10 client.18399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 20 client.18399 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23f715d0 0x558d23f73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 4 client.18399 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.370+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d23ec8bb0 con 0x558d23f715d0 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.371+0000 7f14713f4640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23f715d0 0x558d23f73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.371+0000 7f1454ff9640 10 client.18399 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.371+0000 7f14713f4640 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23f715d0 0x558d23f73a30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dae22cd30269e98 server_cookie=413c13fe199a0aef in_seq=0 out_seq=0 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1454ff9640 1 -- 172.21.15.45:0/1503033802 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d23ec8bb0 con 0x558d23f715d0 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1454ff9640 10 client.18399 handle_command_reply: tid=0 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 client.18399 shutdown 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 msgr2=0x558d23f5e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f1460009260 comp rx=0 tx=0).stop 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 shutdown_connections 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23f715d0 0x558d23f73a30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f5e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f5e1f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1454ff9640 0 client.18399 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 --2- 172.21.15.45:0/1503033802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f60f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 >> 172.21.15.45:0/1503033802 conn(0x558d23f528a0 msgr2=0x558d23f51b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 shutdown_connections 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 1 -- 172.21.15.45:0/1503033802 wait complete. 2024-04-25T00:31:18.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.372+0000 7f1476365740 20 client.18399 trim_cache size 0 max 16384 2024-04-25T00:31:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.385+0000 7f1476365740 1 Processor -- start 2024-04-25T00:31:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.385+0000 7f1476365740 1 -- start start 2024-04-25T00:31:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f79840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1aeb80 con 0x558d23f56ea0 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1b06c0 con 0x558d23f56580 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1476365740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1af250 con 0x558d23f61570 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58224/0 (socket says 172.21.15.45:58224) 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 -- 172.21.15.45:0/971111791 learned_addr learned my addr 172.21.15.45:0/971111791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f14713f4640 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1471bf5640 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f79840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 -- 172.21.15.45:0/971111791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 msgr2=0x558d23f79840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f79840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 -- 172.21.15.45:0/971111791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 msgr2=0x558d23f6d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f6d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 -- 172.21.15.45:0/971111791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d23c997d0 con 0x558d23f56ea0 2024-04-25T00:31:18.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1470bf3640 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f14600ed8c0 tx=0x7f146003b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d78cf4937095ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1456ffd640 1 -- 172.21.15.45:0/971111791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460049070 con 0x558d23f56ea0 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1456ffd640 1 -- 172.21.15.45:0/971111791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14600c0440 con 0x558d23f56ea0 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.386+0000 7f1456ffd640 1 -- 172.21.15.45:0/971111791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600c4870 con 0x558d23f56ea0 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- 172.21.15.45:0/971111791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 msgr2=0x558d23f79300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f14600ed8c0 tx=0x7f146003b990 comp rx=0 tx=0).stop 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- 172.21.15.45:0/971111791 shutdown_connections 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f79840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f79300 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 --2- 172.21.15.45:0/971111791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f6d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- 172.21.15.45:0/971111791 >> 172.21.15.45:0/971111791 conn(0x558d23f528a0 msgr2=0x558d23f51b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- 172.21.15.45:0/971111791 shutdown_connections 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- 172.21.15.45:0/971111791 wait complete. 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 -- start start 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.387+0000 7f1476365740 1 Processor -- start 2024-04-25T00:31:18.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f67310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1afff0 con 0x558d23f56ea0 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1af920 con 0x558d23f56580 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1acb80 con 0x558d23f61570 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 10 client.0 _ll_register_callbacks cb 0x7f1460001e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58240/0 (socket says 172.21.15.45:58240) 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 -- 172.21.15.45:0/94316168 learned_addr learned my addr 172.21.15.45:0/94316168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1471bf5640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f14713f4640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f67310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 -- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 msgr2=0x558d23f67d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f67d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 -- 172.21.15.45:0/94316168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 msgr2=0x558d23f67310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f67310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 -- 172.21.15.45:0/94316168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d23fd6ce0 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1470bf3640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f14600ba3f0 tx=0x7f14600c0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64d5d84c93be63a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146009e020 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f14600c4870 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.388+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14600c0440 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1460049070 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 10 client.18405 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d23fd6ce0 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14600c0440 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1460080020 con 0x558d23f56ea0 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 10 client.18405 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 10 client.18405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 20 client.18405 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23fd82a0 0x558d23fda6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 4 client.18405 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d23ec2f00 con 0x558d23fd82a0 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.389+0000 7f14713f4640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23fd82a0 0x558d23fda6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f14713f4640 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23fd82a0 0x558d23fda6c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=111001019f246216 server_cookie=26a79efaf2cf1081 in_seq=0 out_seq=0 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1454ff9640 10 client.18405 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1454ff9640 1 -- 172.21.15.45:0/94316168 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d23ec2f00 con 0x558d23fd82a0 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1454ff9640 10 client.18405 handle_command_reply: tid=0 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 client.18405 shutdown 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 msgr2=0x558d23f67850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f14600ba3f0 tx=0x7f14600c0300 comp rx=0 tx=0).stop 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 shutdown_connections 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x558d23fd82a0 0x558d23fda6c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558d23f61570 0x558d23f67d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d23f56ea0 0x558d23f67850 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1454ff9640 0 client.18405 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 --2- 172.21.15.45:0/94316168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d23f56580 0x558d23f67310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 >> 172.21.15.45:0/94316168 conn(0x558d23f528a0 msgr2=0x558d23f6c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 shutdown_connections 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 1 -- 172.21.15.45:0/94316168 wait complete. 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.390+0000 7f1476365740 20 client.18405 trim_cache size 0 max 16384 2024-04-25T00:31:18.397 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 msgr2=0x7f14580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 0x7f14580a51c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1464022000 tx=0x7f146401e000 comp rx=0 tx=0).stop 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 msgr2=0x7f146c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f146002a930 tx=0x7f1460003b90 comp rx=0 tx=0).stop 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 shutdown_connections 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f14580a2d00 0x7f14580a51c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f146c19ca20 0x7f146c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146c106aa0 0x7f146c1986a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 --2- 172.21.15.45:0/2512838279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f146c1002e0 0x7f146c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 >> 172.21.15.45:0/2512838279 conn(0x7f146c0758d0 msgr2=0x7f146c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 shutdown_connections 2024-04-25T00:31:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.391+0000 7f1473658640 1 -- 172.21.15.45:0/2512838279 wait complete. 2024-04-25T00:31:18.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.506+0000 7f2286b29640 1 -- 172.21.15.45:0/2303826614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2280100c90 msgr2=0x7f22801063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.506+0000 7f2286b29640 1 --2- 172.21.15.45:0/2303826614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2280100c90 0x7f22801063c0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2274009900 tx=0x7f227402a410 comp rx=0 tx=0).stop 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- 172.21.15.45:0/2303826614 shutdown_connections 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- 172.21.15.45:0/2303826614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2280106aa0 0x7f228010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- 172.21.15.45:0/2303826614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2280100c90 0x7f22801063c0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- 172.21.15.45:0/2303826614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22801002e0 0x7f22801006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- 172.21.15.45:0/2303826614 >> 172.21.15.45:0/2303826614 conn(0x7f22800758d0 msgr2=0x7f2280075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- 172.21.15.45:0/2303826614 shutdown_connections 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- 172.21.15.45:0/2303826614 wait complete. 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 Processor -- start 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- start start 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22801002e0 0x7f2280198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2280106aa0 0x7f2280198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111b90 con 0x7f228019c9d0 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111a10 con 0x7f22801002e0 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.507+0000 7f2286b29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111d10 con 0x7f2280106aa0 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58260/0 (socket says 172.21.15.45:58260) 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 -- 172.21.15.45:0/1344117276 learned_addr learned my addr 172.21.15.45:0/1344117276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22848c5640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22801002e0 0x7f2280198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f227ffff640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2280106aa0 0x7f2280198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 -- 172.21.15.45:0/1344117276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2280106aa0 msgr2=0x7f2280198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2280106aa0 0x7f2280198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 -- 172.21.15.45:0/1344117276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22801002e0 msgr2=0x7f2280198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22801002e0 0x7f2280198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 -- 172.21.15.45:0/1344117276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228019d4c0 con 0x7f228019c9d0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f22850c6640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f2278009fd0 tx=0x7f227800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f227dffb640 1 -- 172.21.15.45:0/1344117276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278017070 con 0x7f228019c9d0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228019d7b0 con 0x7f228019c9d0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f227dffb640 1 -- 172.21.15.45:0/1344117276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2278003e50 con 0x7f228019c9d0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f227dffb640 1 -- 172.21.15.45:0/1344117276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22780106c0 con 0x7f228019c9d0 2024-04-25T00:31:18.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.508+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a4f20 con 0x7f228019c9d0 2024-04-25T00:31:18.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.511+0000 7f227dffb640 1 -- 172.21.15.45:0/1344117276 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2278003970 con 0x7f228019c9d0 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 Processor -- start 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 -- start start 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f227dffb640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4edc60 0x56437d4fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 0x56437d4fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56437d452670 con 0x56437d4ed2b0 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56437d3fa970 con 0x56437d4fc980 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f228983a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56437d44e630 con 0x56437d4edc60 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f227dffb640 1 -- 172.21.15.45:0/1344117276 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f22780cf4a0 con 0x7f228019c9d0 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f227ffff640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58274/0 (socket says 172.21.15.45:58274) 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 -- 172.21.15.45:0/2416266756 learned_addr learned my addr 172.21.15.45:0/2416266756 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f227ffff640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4edc60 0x56437d4fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22850c6640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 0x56437d4fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 -- 172.21.15.45:0/2416266756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4edc60 msgr2=0x56437d4fc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4edc60 0x56437d4fc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 -- 172.21.15.45:0/2416266756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 msgr2=0x56437d4fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 0x56437d4fed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 -- 172.21.15.45:0/2416266756 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56437d41e400 con 0x56437d4ed2b0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22850c6640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 0x56437d4fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.512+0000 7f22848c5640 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f227000b760 tx=0x7f227000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554b194315a3f474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f227ffff640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 0x7f22680a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f2274032f60 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f2266ffd640 1 -- 172.21.15.45:0/2416266756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270003dd0 con 0x56437d4ed2b0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f2266ffd640 1 -- 172.21.15.45:0/2416266756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270015070 con 0x56437d4ed2b0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f2266ffd640 1 -- 172.21.15.45:0/2416266756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270004e00 con 0x56437d4ed2b0 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 -- 172.21.15.45:0/2416266756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 msgr2=0x56437d4ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f227000b760 tx=0x7f227000bc20 comp rx=0 tx=0).stop 2024-04-25T00:31:18.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 -- 172.21.15.45:0/2416266756 shutdown_connections 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4fc980 0x56437d4fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4edc60 0x56437d4fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 --2- 172.21.15.45:0/2416266756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4ed690 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 -- 172.21.15.45:0/2416266756 >> 172.21.15.45:0/2416266756 conn(0x56437d4e98a0 msgr2=0x56437d4e9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 -- 172.21.15.45:0/2416266756 shutdown_connections 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.513+0000 7f228983a740 1 -- 172.21.15.45:0/2416266756 wait complete. 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 -- start start 2024-04-25T00:31:18.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 Processor -- start 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4fc980 0x56437d4f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56437d4ff740 con 0x56437d4edc60 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56437d40e050 con 0x56437d4ed2b0 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56437d4ff8c0 con 0x56437d4fc980 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f228983a740 10 client.0 _ll_register_callbacks cb 0x56437d165be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f227ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f227ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58290/0 (socket says 172.21.15.45:58290) 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.514+0000 7f227ffff640 1 -- 172.21.15.45:0/2979183633 learned_addr learned my addr 172.21.15.45:0/2979183633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f22848c5640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f22850c6640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4fc980 0x56437d4f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 -- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4fc980 msgr2=0x56437d4f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4fc980 0x56437d4f9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 -- 172.21.15.45:0/2979183633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 msgr2=0x56437d4fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 -- 172.21.15.45:0/2979183633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437d3dfdc0 con 0x56437d4edc60 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f227ffff640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2274033000 tx=0x7f227402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96635f2dd270731a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f225ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227404d070 con 0x56437d4edc60 2024-04-25T00:31:18.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2270015070 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f2274005240 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2274059050 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f228983a740 10 client.18417 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56437d1663f0 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.515+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270015070 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22740035f0 con 0x56437d4edc60 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 10 client.18417 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 10 client.18417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 20 client.18417 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d50c870 0x56437d50ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 4 client.18417 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56437d45fbb0 con 0x56437d50c870 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f22848c5640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d50c870 0x56437d50ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f22848c5640 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d50c870 0x56437d50ed10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c7b7ebb4d8da81a server_cookie=5dd956b2d8320f9b in_seq=0 out_seq=0 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.516+0000 7f225ffff640 10 client.18417 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f225ffff640 1 -- 172.21.15.45:0/2979183633 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56437d45fbb0 con 0x56437d50c870 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f225ffff640 10 client.18417 handle_command_reply: tid=0 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 client.18417 shutdown 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 msgr2=0x56437d4f9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2274033000 tx=0x7f227402a410 comp rx=0 tx=0).stop 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 shutdown_connections 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d50c870 0x56437d50ed10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4fc980 0x56437d4f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d4f9570 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f225ffff640 0 client.18417 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 --2- 172.21.15.45:0/2979183633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 >> 172.21.15.45:0/2979183633 conn(0x56437d4e98a0 msgr2=0x56437d4ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 shutdown_connections 2024-04-25T00:31:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.517+0000 7f228983a740 1 -- 172.21.15.45:0/2979183633 wait complete. 2024-04-25T00:31:18.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.518+0000 7f228983a740 20 client.18417 trim_cache size 0 max 16384 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 Processor -- start 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 -- start start 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d508640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ac9c0 con 0x56437d4edc60 2024-04-25T00:31:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ae500 con 0x56437d4ed2b0 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f228983a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ad090 con 0x56437d4e98a0 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f22850c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f22850c6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58302/0 (socket says 172.21.15.45:58302) 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f22850c6640 1 -- 172.21.15.45:0/2818916135 learned_addr learned my addr 172.21.15.45:0/2818916135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f227ffff640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.531+0000 7f22850c6640 1 -- 172.21.15.45:0/2818916135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 msgr2=0x56437d508640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22848c5640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d508640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22850c6640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d508640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22850c6640 1 -- 172.21.15.45:0/2818916135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 msgr2=0x56437d4f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22850c6640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22850c6640 1 -- 172.21.15.45:0/2818916135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56437d2307d0 con 0x56437d4edc60 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f227ffff640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22848c5640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d508640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22850c6640 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2278009370 tx=0x7f2278086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7689e5e05480163b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22667fc640 1 -- 172.21.15.45:0/2818916135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278020040 con 0x56437d4edc60 2024-04-25T00:31:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22667fc640 1 -- 172.21.15.45:0/2818916135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227809b6a0 con 0x56437d4edc60 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f22667fc640 1 -- 172.21.15.45:0/2818916135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278097d10 con 0x56437d4edc60 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 -- 172.21.15.45:0/2818916135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 msgr2=0x56437d5145e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2278009370 tx=0x7f2278086040 comp rx=0 tx=0).stop 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 -- 172.21.15.45:0/2818916135 shutdown_connections 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4edc60 0x56437d5145e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4ed2b0 0x56437d4f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 --2- 172.21.15.45:0/2818916135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d508640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.532+0000 7f228983a740 1 -- 172.21.15.45:0/2818916135 >> 172.21.15.45:0/2818916135 conn(0x56437d4f9c10 msgr2=0x56437d4f43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- 172.21.15.45:0/2818916135 shutdown_connections 2024-04-25T00:31:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- 172.21.15.45:0/2818916135 wait complete. 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- start start 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 Processor -- start 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d4f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4edc60 0x56437d4f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ade30 con 0x56437d4ed2b0 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ad760 con 0x56437d4edc60 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22801aa9c0 con 0x56437d4e98a0 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f228983a740 10 client.0 _ll_register_callbacks cb 0x7f2274009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f227ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f227ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58314/0 (socket says 172.21.15.45:58314) 2024-04-25T00:31:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f227ffff640 1 -- 172.21.15.45:0/296815586 learned_addr learned my addr 172.21.15.45:0/296815586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f22848c5640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d4f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.533+0000 7f22850c6640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4edc60 0x56437d4f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 -- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 msgr2=0x56437d4f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d4f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 -- 172.21.15.45:0/296815586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4edc60 msgr2=0x56437d4f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4edc60 0x56437d4f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 -- 172.21.15.45:0/296815586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56437d5720a0 con 0x56437d4ed2b0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f227ffff640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f22740039c0 tx=0x7f2274003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c95030923bad18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227404d070 con 0x56437d4ed2b0 2024-04-25T00:31:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f2274003e30 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227809b6a0 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2274035050 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f228983a740 10 client.18423 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56437d5732f0 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227809b6a0 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.534+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22740049c0 con 0x56437d4ed2b0 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 10 client.18423 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 10 client.18423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 20 client.18423 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d4fc980 0x56437d575ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 4 client.18423 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56437d459f00 con 0x56437d4fc980 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f22848c5640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d4fc980 0x56437d575ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f22848c5640 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d4fc980 0x56437d575ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2b0e3734b88f06f server_cookie=f5c8c7e2ca56bede in_seq=0 out_seq=0 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f225ffff640 10 client.18423 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f225ffff640 1 -- 172.21.15.45:0/296815586 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56437d459f00 con 0x56437d4fc980 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f225ffff640 10 client.18423 handle_command_reply: tid=0 2024-04-25T00:31:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 client.18423 shutdown 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 msgr2=0x56437d4f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f22740039c0 tx=0x7f2274003820 comp rx=0 tx=0).stop 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 shutdown_connections 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x56437d4fc980 0x56437d575ad0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56437d4edc60 0x56437d4f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56437d4ed2b0 0x56437d4f69a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f225ffff640 0 client.18423 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 --2- 172.21.15.45:0/296815586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56437d4e98a0 0x56437d4f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.535+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 >> 172.21.15.45:0/296815586 conn(0x56437d4f9c10 msgr2=0x56437d4f4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 shutdown_connections 2024-04-25T00:31:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f228983a740 1 -- 172.21.15.45:0/296815586 wait complete. 2024-04-25T00:31:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f228983a740 20 client.18423 trim_cache size 0 max 16384 2024-04-25T00:31:18.550 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 msgr2=0x7f22680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 0x7f22680a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f2274032f60 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 msgr2=0x7f228019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f2278009fd0 tx=0x7f227800b260 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 shutdown_connections 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f22680a2d00 0x7f22680a51c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228019c9d0 0x7f228019ce80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2280106aa0 0x7f2280198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 --2- 172.21.15.45:0/1344117276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22801002e0 0x7f2280198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 >> 172.21.15.45:0/1344117276 conn(0x7f22800758d0 msgr2=0x7f2280107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 shutdown_connections 2024-04-25T00:31:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.536+0000 7f2286b29640 1 -- 172.21.15.45:0/1344117276 wait complete. 2024-04-25T00:31:18.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:31:18.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:18 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2351165591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:18.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.657+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1184296860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 msgr2=0x7fb864102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.657+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1184296860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864102da0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009ca0 tx=0x7fb85c02a9e0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1184296860 shutdown_connections 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1184296860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb86410bd00 0x7fb86410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1184296860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8641032e0 0x7fb86410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1184296860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864102da0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1184296860 >> 172.21.15.45:0/1184296860 conn(0x7fb864077740 msgr2=0x7fb864077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1184296860 shutdown_connections 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1184296860 wait complete. 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 Processor -- start 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 -- start start 2024-04-25T00:31:18.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8641032e0 0x7fb864198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.658+0000 7fb86b4bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 0x7fb86419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb86b4bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111b00 con 0x7fb8641029c0 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb86b4bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111980 con 0x7fb8641032e0 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb86b4bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111c80 con 0x7fb86410bd00 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58324/0 (socket says 172.21.15.45:58324) 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 -- 172.21.15.45:0/1400152832 learned_addr learned my addr 172.21.15.45:0/1400152832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb868a58640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8641032e0 0x7fb864198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 0x7fb86419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 -- 172.21.15.45:0/1400152832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 msgr2=0x7fb86419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 0x7fb86419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 -- 172.21.15.45:0/1400152832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8641032e0 msgr2=0x7fb864198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8641032e0 0x7fb864198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 -- 172.21.15.45:0/1400152832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86419d010 con 0x7fb8641029c0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 0x7fb86419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb869259640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005cc0 tx=0x7fb85c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb85a7fc640 1 -- 172.21.15.45:0/1400152832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c047070 con 0x7fb8641029c0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86419d2a0 con 0x7fb8641029c0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb85a7fc640 1 -- 172.21.15.45:0/1400152832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c003e00 con 0x7fb8641029c0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb85a7fc640 1 -- 172.21.15.45:0/1400152832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c033050 con 0x7fb8641029c0 2024-04-25T00:31:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.659+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a4b90 con 0x7fb8641029c0 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb85a7fc640 1 -- 172.21.15.45:0/1400152832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb85c04a070 con 0x7fb8641029c0 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 Processor -- start 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 -- start start 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50055030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50057960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f4ffaf670 con 0x558f5004a580 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558f4ff57970 con 0x558f50055570 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558f4ffab630 con 0x558f5004aea0 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb85a7fc640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 0x7fb8500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb869a5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb869a5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58340/0 (socket says 172.21.15.45:58340) 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.662+0000 7fb869a5a640 1 -- 172.21.15.45:0/2835280960 learned_addr learned my addr 172.21.15.45:0/2835280960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 0x7fb8500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb85a7fc640 1 -- 172.21.15.45:0/1400152832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb85c0f0440 con 0x7fb8641029c0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869259640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50055030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb868a58640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50057960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 -- 172.21.15.45:0/2835280960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 msgr2=0x558f50055030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50055030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 -- 172.21.15.45:0/2835280960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 msgr2=0x558f50057960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50057960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 -- 172.21.15.45:0/2835280960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4ff7b400 con 0x558f5004a580 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb868a58640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50057960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb860012a30 tx=0x7fb860012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88cb0d0b72a537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 0x7fb8500a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb8600107a0 tx=0x7fb86001d000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb84f7fe640 1 -- 172.21.15.45:0/2835280960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860023070 con 0x558f5004a580 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb84f7fe640 1 -- 172.21.15.45:0/2835280960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb860009990 con 0x558f5004a580 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.663+0000 7fb84f7fe640 1 -- 172.21.15.45:0/2835280960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860004c00 con 0x558f5004a580 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- 172.21.15.45:0/2835280960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 msgr2=0x558f5004a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb860012a30 tx=0x7fb860012e30 comp rx=0 tx=0).stop 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- 172.21.15.45:0/2835280960 shutdown_connections 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50057960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50055030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/2835280960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f5004a960 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- 172.21.15.45:0/2835280960 >> 172.21.15.45:0/2835280960 conn(0x558f500468a0 msgr2=0x558f50046c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- 172.21.15.45:0/2835280960 shutdown_connections 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- 172.21.15.45:0/2835280960 wait complete. 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 -- start start 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.664+0000 7fb86e1d5740 1 Processor -- start 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50052220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50052830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f50058300 con 0x558f5004a580 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558f4ff6b050 con 0x558f50055570 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558f50058480 con 0x558f5004aea0 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb86e1d5740 10 client.0 _ll_register_callbacks cb 0x558f4fcc2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58344/0 (socket says 172.21.15.45:58344) 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 -- 172.21.15.45:0/58899355 learned_addr learned my addr 172.21.15.45:0/58899355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb868a58640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50052220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869a5a640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 -- 172.21.15.45:0/58899355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 msgr2=0x558f50052220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50052220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 -- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 msgr2=0x558f50052830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50052830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 -- 172.21.15.45:0/58899355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f4ff3cdc0 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb869259640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ed880 tx=0x7fb85c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b79966ae91efc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb84cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.665+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0bcca0 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb85c02f850 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb860009990 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb85c0bc5a0 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 10 client.18435 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f4fcc33f0 con 0x558f5004a580 2024-04-25T00:31:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb860009990 con 0x558f5004a580 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb85c004b40 con 0x558f5004a580 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 10 client.18435 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 20 client.18435 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500655a0 0x558f500679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 4 client.18435 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f4ffbcbb0 con 0x558f500655a0 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.666+0000 7fb868a58640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500655a0 0x558f500679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.667+0000 7fb868a58640 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500655a0 0x558f500679d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83793ba18a913396 server_cookie=278034bb860bdc84 in_seq=0 out_seq=0 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.667+0000 7fb84cff9640 10 client.18435 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb84cff9640 1 -- 172.21.15.45:0/58899355 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f4ffbcbb0 con 0x558f500655a0 2024-04-25T00:31:18.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb84cff9640 10 client.18435 handle_command_reply: tid=0 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 client.18435 shutdown 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 msgr2=0x558f50054f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ed880 tx=0x7fb85c0095c0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 shutdown_connections 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50052830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f50052220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500655a0 0x558f500679d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/58899355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f50054f30 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb84cff9640 0 client.18435 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 >> 172.21.15.45:0/58899355 conn(0x558f500468a0 msgr2=0x558f50045b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 shutdown_connections 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 1 -- 172.21.15.45:0/58899355 wait complete. 2024-04-25T00:31:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.668+0000 7fb86e1d5740 20 client.18435 trim_cache size 0 max 16384 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 Processor -- start 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 -- start start 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f5005dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50054a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aebe0 con 0x558f5004a580 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641b0720 con 0x558f50055570 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.681+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641af2b0 con 0x558f5004aea0 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58346/0 (socket says 172.21.15.45:58346) 2024-04-25T00:31:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 -- 172.21.15.45:0/765039602 learned_addr learned my addr 172.21.15.45:0/765039602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb868a58640 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f5005dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 -- 172.21.15.45:0/765039602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 msgr2=0x558f5005dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f5005dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 -- 172.21.15.45:0/765039602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 msgr2=0x558f50054a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50054a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 -- 172.21.15.45:0/765039602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4fd8d7d0 con 0x558f5004a580 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb869259640 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004af0 tx=0x7fb85c0bc120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e92860191122b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb84effd640 1 -- 172.21.15.45:0/765039602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0c0950 con 0x558f5004a580 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb84effd640 1 -- 172.21.15.45:0/765039602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c0c5210 con 0x558f5004a580 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.682+0000 7fb84effd640 1 -- 172.21.15.45:0/765039602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c02f850 con 0x558f5004a580 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- 172.21.15.45:0/765039602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 msgr2=0x558f500611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004af0 tx=0x7fb85c0bc120 comp rx=0 tx=0).stop 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- 172.21.15.45:0/765039602 shutdown_connections 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f50055570 0x558f50054a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f5005dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/765039602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5004a580 0x558f500611f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- 172.21.15.45:0/765039602 >> 172.21.15.45:0/765039602 conn(0x558f500468a0 msgr2=0x558f5004c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- 172.21.15.45:0/765039602 shutdown_connections 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- 172.21.15.45:0/765039602 wait complete. 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- start start 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 Processor -- start 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f5004a580 0x558f500c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f500c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641b0050 con 0x558f50055570 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641af980 con 0x558f5004a580 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.683+0000 7fb86e1d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641acbe0 con 0x558f5004aea0 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb86e1d5740 10 client.0 _ll_register_callbacks cb 0x7fb85c0b88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58362/0 (socket says 172.21.15.45:58362) 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 -- 172.21.15.45:0/1346665852 learned_addr learned my addr 172.21.15.45:0/1346665852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869259640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f5004a580 0x558f500c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb868a58640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f500c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 -- 172.21.15.45:0/1346665852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 msgr2=0x558f500c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f500c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 -- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f5004a580 msgr2=0x558f500c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f5004a580 0x558f500c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 -- 172.21.15.45:0/1346665852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5006d350 con 0x558f50055570 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb869a5a640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb860009820 tx=0x7fb860004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4be6f6d521456ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb84cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860023040 con 0x558f50055570 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb86000d920 con 0x558f50055570 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c0c5210 con 0x558f50055570 2024-04-25T00:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb86003a050 con 0x558f50055570 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb86e1d5740 10 client.18441 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.684+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f500cfd80 con 0x558f50055570 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c0c5210 con 0x558f50055570 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb86001fbb0 con 0x558f50055570 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 10 client.18441 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 20 client.18441 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500d02a0 0x558f500d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 4 client.18441 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f4ffb6f00 con 0x558f500d02a0 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb869259640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500d02a0 0x558f500d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb84cff9640 10 client.18441 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.685+0000 7fb869259640 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500d02a0 0x558f500d26c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca390c81a60f016 server_cookie=9fb3aff39cfd27e4 in_seq=0 out_seq=0 2024-04-25T00:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb84cff9640 1 -- 172.21.15.45:0/1346665852 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f4ffb6f00 con 0x558f500d02a0 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb84cff9640 10 client.18441 handle_command_reply: tid=0 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 client.18441 shutdown 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 msgr2=0x558f5006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb860009820 tx=0x7fb860004890 comp rx=0 tx=0).stop 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 shutdown_connections 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x558f500d02a0 0x558f500d26c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f50055570 0x558f5006cc60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb84cff9640 0 client.18441 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558f5004aea0 0x558f500c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 --2- 172.21.15.45:0/1346665852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f5004a580 0x558f500c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 >> 172.21.15.45:0/1346665852 conn(0x558f500468a0 msgr2=0x558f5004bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 shutdown_connections 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 1 -- 172.21.15.45:0/1346665852 wait complete. 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.686+0000 7fb86e1d5740 20 client.18441 trim_cache size 0 max 16384 2024-04-25T00:31:18.694 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 msgr2=0x7fb8500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 0x7fb8500a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb8600107a0 tx=0x7fb86001d000 comp rx=0 tx=0).stop 2024-04-25T00:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 msgr2=0x7fb864198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005cc0 tx=0x7fb85c003c10 comp rx=0 tx=0).stop 2024-04-25T00:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 shutdown_connections 2024-04-25T00:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb8500a2d00 0x7fb8500a51c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86410bd00 0x7fb86419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8641032e0 0x7fb864198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 --2- 172.21.15.45:0/1400152832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8641029c0 0x7fb864198020 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 >> 172.21.15.45:0/1400152832 conn(0x7fb864077740 msgr2=0x7fb8640fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 shutdown_connections 2024-04-25T00:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:18.687+0000 7fb86b4bd640 1 -- 172.21.15.45:0/1400152832 wait complete. 2024-04-25T00:31:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:19 smithi071 ceph-mon[32667]: pgmap v289: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-25T00:31:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:19 smithi045 ceph-mon[29194]: pgmap v289: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-25T00:31:19.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:19 smithi122 ceph-mon[32959]: pgmap v289: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 216 B/s wr, 10 op/s 2024-04-25T00:31:19.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:31:19.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:19.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:31:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- 172.21.15.45:0/29437671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b0100610 msgr2=0x7fe6b0100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/29437671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b0100610 0x7fe6b0100a90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009930 tx=0x7fe6a002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- 172.21.15.45:0/29437671 shutdown_connections 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/29437671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 0x7fe6b010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/29437671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b0100610 0x7fe6b0100a90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/29437671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b01066b0 0x7fe6b0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- 172.21.15.45:0/29437671 >> 172.21.15.45:0/29437671 conn(0x7fe6b0077740 msgr2=0x7fe6b0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- 172.21.15.45:0/29437671 shutdown_connections 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- 172.21.15.45:0/29437671 wait complete. 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 Processor -- start 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.819+0000 7fe6b79a9640 1 -- start start 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b0100610 0x7fe6b0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 0x7fe6b01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01109f0 con 0x7fe6b01066b0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0110870 con 0x7fe6b0100fd0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0110b70 con 0x7fe6b0100610 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58380/0 (socket says 172.21.15.45:58380) 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 -- 172.21.15.45:0/793314275 learned_addr learned my addr 172.21.15.45:0/793314275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5745640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b0100610 0x7fe6b0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 0x7fe6b01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 -- 172.21.15.45:0/793314275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b0100610 msgr2=0x7fe6b0197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b0100610 0x7fe6b0197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 -- 172.21.15.45:0/793314275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 msgr2=0x7fe6b01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 0x7fe6b01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 -- 172.21.15.45:0/793314275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b019cf90 con 0x7fe6b01066b0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b5f46640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eaa0 tx=0x7fe6ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6a67fc640 1 -- 172.21.15.45:0/793314275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00cdb0 con 0x7fe6b01066b0 2024-04-25T00:31:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6a67fc640 1 -- 172.21.15.45:0/793314275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac005020 con 0x7fe6b01066b0 2024-04-25T00:31:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b019d280 con 0x7fe6b01066b0 2024-04-25T00:31:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.820+0000 7fe6a67fc640 1 -- 172.21.15.45:0/793314275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0116b0 con 0x7fe6b01066b0 2024-04-25T00:31:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.821+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b01a4b10 con 0x7fe6b01066b0 2024-04-25T00:31:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6a67fc640 1 -- 172.21.15.45:0/793314275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6ac005560 con 0x7fe6b01066b0 2024-04-25T00:31:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 Processor -- start 2024-04-25T00:31:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 -- start start 2024-04-25T00:31:19.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db38710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56544da9c2c0 con 0x56544db43010 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56544da44910 con 0x56544db38290 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6ba798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56544da98280 con 0x56544db39980 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58386/0 (socket says 172.21.15.45:58386) 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 -- 172.21.15.45:0/3605915781 learned_addr learned my addr 172.21.15.45:0/3605915781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6a67fc640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 0x7fe69c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b5745640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db38710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 -- 172.21.15.45:0/3605915781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 msgr2=0x56544db37cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b5f46640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 0x7fe69c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db37cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 -- 172.21.15.45:0/3605915781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 msgr2=0x56544db38710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db38710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.824+0000 7fe6b4f44640 1 -- 172.21.15.45:0/3605915781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544da55fb0 con 0x56544db43010 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe6b5745640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db38710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe6a67fc640 1 -- 172.21.15.45:0/793314275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fe6ac0cf9b0 con 0x7fe6b01066b0 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe6a002a940 tx=0x7fe6a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db5232847746b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3605915781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0035040 con 0x56544db43010 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe6b5f46640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 0x7fe69c0a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009270 tx=0x7fe6ac091050 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3605915781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a0031a20 con 0x56544db43010 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.825+0000 7fe69b7fe640 1 -- 172.21.15.45:0/3605915781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004a00 con 0x56544db43010 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- 172.21.15.45:0/3605915781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 msgr2=0x56544db45400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe6a002a940 tx=0x7fe6a0005fd0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- 172.21.15.45:0/3605915781 shutdown_connections 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db45400 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db38710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 --2- 172.21.15.45:0/3605915781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db37cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- 172.21.15.45:0/3605915781 >> 172.21.15.45:0/3605915781 conn(0x56544db00950 msgr2=0x56544db347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- 172.21.15.45:0/3605915781 shutdown_connections 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- 172.21.15.45:0/3605915781 wait complete. 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 -- start start 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.826+0000 7fe6ba798740 1 Processor -- start 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 0x56544db403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56544db45e40 con 0x56544db43010 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56544da57e90 con 0x56544db39980 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56544db45fc0 con 0x56544db38290 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 10 client.0 _ll_register_callbacks cb 0x56544da93970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58398/0 (socket says 172.21.15.45:58398) 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b4f44640 1 -- 172.21.15.45:0/4149336486 learned_addr learned my addr 172.21.15.45:0/4149336486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 0x56544db403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 -- 172.21.15.45:0/4149336486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 msgr2=0x56544db3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 -- 172.21.15.45:0/4149336486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 msgr2=0x56544db408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 -- 172.21.15.45:0/4149336486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544d8787e0 con 0x56544db39980 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6b5745640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 0x56544db403a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0101750 tx=0x7fe6a800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f8153d649ab58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8011440 con 0x56544db39980 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a0031a20 con 0x56544db39980 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.827+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fe6a80115e0 con 0x56544db39980 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6a8023050 con 0x56544db39980 2024-04-25T00:31:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 10 client.27569 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56544d8d8f70 con 0x56544db39980 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a0031a20 con 0x56544db39980 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe6a8010a60 con 0x56544db39980 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 10 client.27569 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 10 client.27569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 20 client.27569 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544db530a0 0x56544db554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 4 client.27569 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56544da705a0 con 0x56544db530a0 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.828+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544db530a0 0x56544db554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.829+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544db530a0 0x56544db554c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1d8349eeb783fe server_cookie=8d7e456504051ae0 in_seq=0 out_seq=0 2024-04-25T00:31:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.829+0000 7fe698ff9640 10 client.27569 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.829+0000 7fe698ff9640 1 -- 172.21.15.45:0/4149336486 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56544da705a0 con 0x56544db530a0 2024-04-25T00:31:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.829+0000 7fe698ff9640 10 client.27569 handle_command_reply: tid=0 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.829+0000 7fe6ba798740 1 client.27569 shutdown 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 msgr2=0x56544db403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 0x56544db403a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0101750 tx=0x7fe6a800be60 comp rx=0 tx=0).stop 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 shutdown_connections 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544db530a0 0x56544db554c0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db39980 0x56544db403a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 --2- 172.21.15.45:0/4149336486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe698ff9640 0 client.27569 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 >> 172.21.15.45:0/4149336486 conn(0x56544db00950 msgr2=0x56544db3a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 shutdown_connections 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 1 -- 172.21.15.45:0/4149336486 wait complete. 2024-04-25T00:31:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.830+0000 7fe6ba798740 20 client.27569 trim_cache size 0 max 16384 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 Processor -- start 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 -- start start 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db4ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db43010 0x56544db5b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01aeb60 con 0x56544db39980 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01b06a0 con 0x56544db43010 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6ba798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01af230 con 0x56544db38290 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58406/0 (socket says 172.21.15.45:58406) 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 -- 172.21.15.45:0/665616480 learned_addr learned my addr 172.21.15.45:0/665616480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db43010 0x56544db5b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 -- 172.21.15.45:0/665616480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 msgr2=0x56544db4ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db4ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 -- 172.21.15.45:0/665616480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db43010 msgr2=0x56544db5b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db43010 0x56544db5b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 -- 172.21.15.45:0/665616480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544d770170 con 0x56544db39980 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe6b5745640 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c010 tx=0x7fe6a80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d297bc70856f8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.848+0000 7fe69affd640 1 -- 172.21.15.45:0/665616480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a801b670 con 0x56544db39980 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe69affd640 1 -- 172.21.15.45:0/665616480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a8020070 con 0x56544db39980 2024-04-25T00:31:19.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe69affd640 1 -- 172.21.15.45:0/665616480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8010ba0 con 0x56544db39980 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- 172.21.15.45:0/665616480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 msgr2=0x56544db5ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c010 tx=0x7fe6a80058d0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- 172.21.15.45:0/665616480 shutdown_connections 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db43010 0x56544db5b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db39980 0x56544db5ad70 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 --2- 172.21.15.45:0/665616480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db38290 0x56544db4ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- 172.21.15.45:0/665616480 >> 172.21.15.45:0/665616480 conn(0x56544db00950 msgr2=0x56544db33950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- 172.21.15.45:0/665616480 shutdown_connections 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- 172.21.15.45:0/665616480 wait complete. 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.849+0000 7fe6ba798740 1 -- start start 2024-04-25T00:31:19.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 Processor -- start 2024-04-25T00:31:19.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db4b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01affd0 con 0x56544db43010 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01af900 con 0x56544db38290 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01acb60 con 0x56544db39980 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6ba798740 10 client.0 _ll_register_callbacks cb 0x7fe6a80043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58416/0 (socket says 172.21.15.45:58416) 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 -- 172.21.15.45:0/2733039463 learned_addr learned my addr 172.21.15.45:0/2733039463 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db4b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b5745640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 -- 172.21.15.45:0/2733039463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 msgr2=0x56544db4b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db4b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 -- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 msgr2=0x56544db4b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db4b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.850+0000 7fe6b4f44640 1 -- 172.21.15.45:0/2733039463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544db487c0 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6b4f44640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005550 tx=0x7fe6a0031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab75d7d4ddc1216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0035040 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fe6a0004030 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a8020070 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6a0004230 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 10 client.18459 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56544dbbe2a0 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a8020070 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe6a0004c30 con 0x56544db43010 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 10 client.18459 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 20 client.18459 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:19.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544dbbe5f0 0x56544dbc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 4 client.18459 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.851+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56544daa9800 con 0x56544dbbe5f0 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.852+0000 7fe6b5745640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544dbbe5f0 0x56544dbc09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.852+0000 7fe698ff9640 10 client.18459 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.852+0000 7fe6b5745640 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544dbbe5f0 0x56544dbc09e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d85fc5a18dda199 server_cookie=ff484934c85f2050 in_seq=0 out_seq=0 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe698ff9640 1 -- 172.21.15.45:0/2733039463 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56544daa9800 con 0x56544dbbe5f0 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe698ff9640 10 client.18459 handle_command_reply: tid=0 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 client.18459 shutdown 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 msgr2=0x56544db48250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005550 tx=0x7fe6a0031740 comp rx=0 tx=0).stop 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 shutdown_connections 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x56544dbbe5f0 0x56544dbc09e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56544db43010 0x56544db48250 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56544db39980 0x56544db4b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe698ff9640 0 client.18459 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.853+0000 7fe6ba798740 1 --2- 172.21.15.45:0/2733039463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56544db38290 0x56544db4b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 >> 172.21.15.45:0/2733039463 conn(0x56544db00950 msgr2=0x56544db41690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 shutdown_connections 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6ba798740 1 -- 172.21.15.45:0/2733039463 wait complete. 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6ba798740 20 client.18459 trim_cache size 0 max 16384 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "2269511e-8f5e-46d4-a681-693932aa5cb3", 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:31:19.860 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:19.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 msgr2=0x7fe69c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 0x7fe69c0a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009270 tx=0x7fe6ac091050 comp rx=0 tx=0).stop 2024-04-25T00:31:19.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 msgr2=0x7fe6b019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eaa0 tx=0x7fe6ac00ef60 comp rx=0 tx=0).stop 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 shutdown_connections 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe69c0a2d00 0x7fe69c0a51c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b01066b0 0x7fe6b019c860 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6b0100fd0 0x7fe6b01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 --2- 172.21.15.45:0/793314275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6b0100610 0x7fe6b0197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 >> 172.21.15.45:0/793314275 conn(0x7fe6b0077740 msgr2=0x7fe6b00ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.854+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 shutdown_connections 2024-04-25T00:31:19.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.855+0000 7fe6b79a9640 1 -- 172.21.15.45:0/793314275 wait complete. 2024-04-25T00:31:19.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:31:19.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 -- 172.21.15.45:0/866506665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe4101710 msgr2=0x7effe4101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 --2- 172.21.15.45:0/866506665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe4101710 0x7effe4101b90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7effd8009930 tx=0x7effd802a420 comp rx=0 tx=0).stop 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 -- 172.21.15.45:0/866506665 shutdown_connections 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 --2- 172.21.15.45:0/866506665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe41020d0 0x7effe410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 --2- 172.21.15.45:0/866506665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe4101710 0x7effe4101b90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 --2- 172.21.15.45:0/866506665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.969+0000 7effe8eb3640 1 -- 172.21.15.45:0/866506665 >> 172.21.15.45:0/866506665 conn(0x7effe40fd180 msgr2=0x7effe40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.970+0000 7effe8eb3640 1 -- 172.21.15.45:0/866506665 shutdown_connections 2024-04-25T00:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.970+0000 7effe8eb3640 1 -- 172.21.15.45:0/866506665 wait complete. 2024-04-25T00:31:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.970+0000 7effe8eb3640 1 Processor -- start 2024-04-25T00:31:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.970+0000 7effe8eb3640 1 -- start start 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe4101710 0x7effe4073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe40744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effe4111b30 con 0x7effe41020d0 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effe41119b0 con 0x7effe4101710 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effe4111cb0 con 0x7effe4107720 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58426/0 (socket says 172.21.15.45:58426) 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 -- 172.21.15.45:0/820148325 learned_addr learned my addr 172.21.15.45:0/820148325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe259c640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe4101710 0x7effe4073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe2d9d640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe40744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 -- 172.21.15.45:0/820148325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 msgr2=0x7effe40744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe40744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 -- 172.21.15.45:0/820148325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe4101710 msgr2=0x7effe4073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe4101710 0x7effe4073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 -- 172.21.15.45:0/820148325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe41a65d0 con 0x7effe41020d0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe2d9d640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe40744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe1d9b640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7effd8009230 tx=0x7effd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effd37fe640 1 -- 172.21.15.45:0/820148325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd803f030 con 0x7effe41020d0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effd37fe640 1 -- 172.21.15.45:0/820148325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd8003ad0 con 0x7effe41020d0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe41a6860 con 0x7effe41020d0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.971+0000 7effd37fe640 1 -- 172.21.15.45:0/820148325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8035070 con 0x7effe41020d0 2024-04-25T00:31:19.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.972+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41a6d50 con 0x7effe41020d0 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effd37fe640 1 -- 172.21.15.45:0/820148325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7effd8042030 con 0x7effe41020d0 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 Processor -- start 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 -- start start 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d23e2286b0 con 0x55d23e2c3630 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d23e1d0880 con 0x55d23e2ce620 2024-04-25T00:31:19.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.974+0000 7effebbc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d23e224670 con 0x55d23e2c3f50 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effd37fe640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 0x7effcc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58442/0 (socket says 172.21.15.45:58442) 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 -- 172.21.15.45:0/1980284681 learned_addr learned my addr 172.21.15.45:0/1980284681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe259c640 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 0x7effcc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effd37fe640 1 -- 172.21.15.45:0/820148325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7effd80efdb0 con 0x7effe41020d0 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe1d9b640 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 -- 172.21.15.45:0/1980284681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 msgr2=0x55d23e2ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 -- 172.21.15.45:0/1980284681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 msgr2=0x55d23e2d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 -- 172.21.15.45:0/1980284681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23e0067d0 con 0x55d23e2c3630 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7effdc0113e0 tx=0x7effdc0118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee105bc7afb5647f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effe2d9d640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 0x7effcc0a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effdc009370 tx=0x7effdc016000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effd0ff9640 1 -- 172.21.15.45:0/1980284681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc024070 con 0x55d23e2c3630 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effd0ff9640 1 -- 172.21.15.45:0/1980284681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effdc009d70 con 0x55d23e2c3630 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.975+0000 7effd0ff9640 1 -- 172.21.15.45:0/1980284681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc004260 con 0x55d23e2c3630 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 -- 172.21.15.45:0/1980284681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 msgr2=0x55d23e2c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7effdc0113e0 tx=0x7effdc0118a0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 -- 172.21.15.45:0/1980284681 shutdown_connections 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 --2- 172.21.15.45:0/1980284681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2c3a10 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 -- 172.21.15.45:0/1980284681 >> 172.21.15.45:0/1980284681 conn(0x55d23e2bf950 msgr2=0x55d23e2bfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 -- 172.21.15.45:0/1980284681 shutdown_connections 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.976+0000 7effebbc3740 1 -- 172.21.15.45:0/1980284681 wait complete. 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 -- start start 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 Processor -- start 2024-04-25T00:31:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2ce620 0x55d23e2cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d23e2d13b0 con 0x55d23e2c3630 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d23e1e3f50 con 0x55d23e2c3f50 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d23e2d1530 con 0x55d23e2ce620 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effebbc3740 10 client.0 _ll_register_callbacks cb 0x55d23e21f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58446/0 (socket says 172.21.15.45:58446) 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 -- 172.21.15.45:0/4109602852 learned_addr learned my addr 172.21.15.45:0/4109602852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe1d9b640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe2d9d640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2ce620 0x55d23e2cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 -- 172.21.15.45:0/4109602852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2ce620 msgr2=0x55d23e2cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2ce620 0x55d23e2cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 -- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 msgr2=0x55d23e2cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 -- 172.21.15.45:0/4109602852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23df3c3f0 con 0x55d23e2c3630 2024-04-25T00:31:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe1d9b640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.977+0000 7effe259c640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7effd400be70 tx=0x7effd4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438cb2c40627448b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd401f060 con 0x55d23e2c3630 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7effd40108a0 con 0x55d23e2c3630 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d23e235bf0 con 0x55d23e2c3630 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effd4010aa0 con 0x55d23e2c3630 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 10 client.18471 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d23defc0f0 con 0x55d23e2c3630 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d23e235bf0 con 0x55d23e2c3630 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effd4019030 con 0x55d23e2c3630 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 10 client.18471 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 20 client.18471 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e2de8d0 0x55d23e2e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 4 client.18471 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.978+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d23e22ff40 con 0x55d23e2de8d0 2024-04-25T00:31:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.979+0000 7effe1d9b640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e2de8d0 0x55d23e2e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.979+0000 7effe1d9b640 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e2de8d0 0x55d23e2e0d60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cead192453768a server_cookie=25420fc4932ac5d1 in_seq=0 out_seq=0 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.979+0000 7effc9ffb640 10 client.18471 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effc9ffb640 1 -- 172.21.15.45:0/4109602852 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d23e22ff40 con 0x55d23e2de8d0 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effc9ffb640 10 client.18471 handle_command_reply: tid=0 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 client.18471 shutdown 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 msgr2=0x55d23e2cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7effd400be70 tx=0x7effd4012820 comp rx=0 tx=0).stop 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 shutdown_connections 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2ce620 0x55d23e2cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e2de8d0 0x55d23e2e0d60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 --2- 172.21.15.45:0/4109602852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cb630 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effc9ffb640 0 client.18471 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 >> 172.21.15.45:0/4109602852 conn(0x55d23e2bf950 msgr2=0x55d23e2c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 shutdown_connections 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 1 -- 172.21.15.45:0/4109602852 wait complete. 2024-04-25T00:31:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.980+0000 7effebbc3740 20 client.18471 trim_cache size 0 max 16384 2024-04-25T00:31:20.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 Processor -- start 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 -- start start 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.997+0000 7effebbc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effe41b0e00 con 0x55d23e2c3630 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effebbc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effe41b2940 con 0x55d23e2ce620 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effebbc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effe41b14d0 con 0x55d23e2c3f50 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58450/0 (socket says 172.21.15.45:58450) 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 -- 172.21.15.45:0/25390967 learned_addr learned my addr 172.21.15.45:0/25390967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe2d9d640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 -- 172.21.15.45:0/25390967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 msgr2=0x55d23e2ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe1d9b640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 -- 172.21.15.45:0/25390967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 msgr2=0x55d23e2da270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2da270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 -- 172.21.15.45:0/25390967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23e2c63f0 con 0x55d23e2c3630 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe2d9d640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2da270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe1d9b640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effe259c640 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7effd400b8b0 tx=0x7effd400bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d96abd5677db548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effcbfff640 1 -- 172.21.15.45:0/25390967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd402e020 con 0x55d23e2c3630 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effcbfff640 1 -- 172.21.15.45:0/25390967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd40106a0 con 0x55d23e2c3630 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.998+0000 7effcbfff640 1 -- 172.21.15.45:0/25390967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4005640 con 0x55d23e2c3630 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- 172.21.15.45:0/25390967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 msgr2=0x55d23e2cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7effd400b8b0 tx=0x7effd400bb30 comp rx=0 tx=0).stop 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- 172.21.15.45:0/25390967 shutdown_connections 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2ce620 0x55d23e2da270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3f50 0x55d23e2ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- 172.21.15.45:0/25390967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2c3630 0x55d23e2cdd60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- 172.21.15.45:0/25390967 >> 172.21.15.45:0/25390967 conn(0x55d23e2bf950 msgr2=0x55d23e2beaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- 172.21.15.45:0/25390967 shutdown_connections 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- 172.21.15.45:0/25390967 wait complete. 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- start start 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 Processor -- start 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3630 0x55d23e2d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effe41b2270 con 0x55d23e2ce620 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effe41b1ba0 con 0x55d23e2c3f50 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effe41aee00 con 0x55d23e2c3630 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:19.999+0000 7effebbc3740 10 client.0 _ll_register_callbacks cb 0x7effd400b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58452/0 (socket says 172.21.15.45:58452) 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 -- 172.21.15.45:0/2100258911 learned_addr learned my addr 172.21.15.45:0/2100258911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe259c640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3630 0x55d23e2d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe1d9b640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 -- 172.21.15.45:0/2100258911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3630 msgr2=0x55d23e2d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3630 0x55d23e2d6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 -- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 msgr2=0x55d23e2c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 -- 172.21.15.45:0/2100258911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23e2c8c30 con 0x55d23e2ce620 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effe2d9d640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7effdc009fd0 tx=0x7effdc0142f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657343ebcf14d4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc024040 con 0x55d23e2ce620 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7effdc0046e0 con 0x55d23e2ce620 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd40106a0 con 0x55d23e2ce620 2024-04-25T00:31:20.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effdc039050 con 0x55d23e2ce620 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effebbc3740 10 client.18477 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.000+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d23e344dc0 con 0x55d23e2ce620 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd40106a0 con 0x55d23e2ce620 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effdc01d030 con 0x55d23e2ce620 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 10 client.18477 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 20 client.18477 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e345280 0x55d23e3476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 4 client.18477 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d23e1ad6f0 con 0x55d23e345280 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effe259c640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e345280 0x55d23e3476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effe259c640 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e345280 0x55d23e3476a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9be00f3cb72471 server_cookie=69ed92fd30e5a908 in_seq=0 out_seq=0 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.001+0000 7effc9ffb640 10 client.18477 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effc9ffb640 1 -- 172.21.15.45:0/2100258911 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d23e1ad6f0 con 0x55d23e345280 2024-04-25T00:31:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effc9ffb640 10 client.18477 handle_command_reply: tid=0 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 client.18477 shutdown 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 msgr2=0x55d23e2c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7effdc009fd0 tx=0x7effdc0142f0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 shutdown_connections 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55d23e345280 0x55d23e3476a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23e2ce620 0x55d23e2c8460 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23e2c3f50 0x55d23e2c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effc9ffb640 0 client.18477 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 --2- 172.21.15.45:0/2100258911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23e2c3630 0x55d23e2d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 >> 172.21.15.45:0/2100258911 conn(0x55d23e2bf950 msgr2=0x55d23e2cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 shutdown_connections 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 1 -- 172.21.15.45:0/2100258911 wait complete. 2024-04-25T00:31:20.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effebbc3740 20 client.18477 trim_cache size 0 max 16384 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "2269511e-8f5e-46d4-a681-693932aa5cb3": { 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "2269511e-8f5e-46d4-a681-693932aa5cb3", 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:31:20.011 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 msgr2=0x7effcc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 0x7effcc0a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effdc009370 tx=0x7effdc016000 comp rx=0 tx=0).stop 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 msgr2=0x7effe4073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7effd8009230 tx=0x7effd8004270 comp rx=0 tx=0).stop 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 shutdown_connections 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7effcc0a2d00 0x7effcc0a51c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe4107720 0x7effe40744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effe41020d0 0x7effe4073f80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 --2- 172.21.15.45:0/820148325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effe4101710 0x7effe4073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 >> 172.21.15.45:0/820148325 conn(0x7effe40fd180 msgr2=0x7effe40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 shutdown_connections 2024-04-25T00:31:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:20.002+0000 7effe8eb3640 1 -- 172.21.15.45:0/820148325 wait complete. 2024-04-25T00:31:20.014 INFO:tasks.cephfs.filesystem:scrub status for tag:2269511e-8f5e-46d4-a681-693932aa5cb3 - {'path': '/', 'tag': '2269511e-8f5e-46d4-a681-693932aa5cb3', 'options': 'recursive,force'} 2024-04-25T00:31:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:21 smithi071 ceph-mon[32667]: pgmap v290: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:21 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:31:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:21 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:21 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:31:21.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:21 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:21 smithi045 ceph-mon[29194]: pgmap v290: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:21 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:31:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:21 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:21 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:31:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:21 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:21 smithi122 ceph-mon[32959]: pgmap v290: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:21 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:31:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:21 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:21 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:31:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:21 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:23.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:23 smithi071 ceph-mon[32667]: pgmap v291: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:23.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:23 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:31:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:23 smithi045 ceph-mon[29194]: pgmap v291: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:23.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:23 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:31:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:23 smithi122 ceph-mon[32959]: pgmap v291: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:31:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:23 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:31:25.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:25 smithi071 ceph-mon[32667]: pgmap v292: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T00:31:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:25 smithi045 ceph-mon[29194]: pgmap v292: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T00:31:25.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:25 smithi122 ceph-mon[32959]: pgmap v292: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-25T00:31:27.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:27 smithi071 ceph-mon[32667]: pgmap v293: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T00:31:27.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:27 smithi045 ceph-mon[29194]: pgmap v293: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T00:31:27.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:27 smithi122 ceph-mon[32959]: pgmap v293: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T00:31:29.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:29 smithi071 ceph-mon[32667]: pgmap v294: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T00:31:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:29 smithi045 ceph-mon[29194]: pgmap v294: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T00:31:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:29 smithi122 ceph-mon[32959]: pgmap v294: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-04-25T00:31:31.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:31 smithi071 ceph-mon[32667]: pgmap v295: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-25T00:31:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:31 smithi045 ceph-mon[29194]: pgmap v295: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-25T00:31:31.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:31 smithi122 ceph-mon[32959]: pgmap v295: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-25T00:31:33.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:33 smithi071 ceph-mon[32667]: pgmap v296: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:33 smithi045 ceph-mon[29194]: pgmap v296: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:33 smithi122 ceph-mon[32959]: pgmap v296: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:35.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:35 smithi045 ceph-mon[29194]: pgmap v297: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:31:35.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:35 smithi122 ceph-mon[32959]: pgmap v297: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:31:35.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:35 smithi071 ceph-mon[32667]: pgmap v297: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T00:31:37.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:37 smithi045 ceph-mon[29194]: pgmap v298: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:37.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:37 smithi122 ceph-mon[32959]: pgmap v298: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:37.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:37 smithi071 ceph-mon[32667]: pgmap v298: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:39 smithi045 ceph-mon[29194]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:39 smithi122 ceph-mon[32959]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:39.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:39 smithi071 ceph-mon[32667]: pgmap v299: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:41 smithi045 ceph-mon[29194]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:41.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:41 smithi122 ceph-mon[32959]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:41.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:41 smithi071 ceph-mon[32667]: pgmap v300: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:43 smithi045 ceph-mon[29194]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:43.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:43 smithi071 ceph-mon[32667]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:43 smithi122 ceph-mon[32959]: pgmap v301: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:45.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:45 smithi045 ceph-mon[29194]: pgmap v302: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:31:45.558 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:45 smithi122 ceph-mon[32959]: pgmap v302: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:31:45.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:45 smithi071 ceph-mon[32667]: pgmap v302: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:31:47.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:47 smithi045 ceph-mon[29194]: pgmap v303: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:31:47.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:47 smithi122 ceph-mon[32959]: pgmap v303: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:31:47.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:47 smithi071 ceph-mon[32667]: pgmap v303: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:31:49.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:49 smithi045 ceph-mon[29194]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:49 smithi122 ceph-mon[32959]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:49.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:49 smithi071 ceph-mon[32667]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:50.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:31:50.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:31:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 -- 172.21.15.45:0/3002801006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404100620 msgr2=0x7fb404100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/3002801006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404100620 0x7fb404100aa0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009930 tx=0x7fb3f402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 -- 172.21.15.45:0/3002801006 shutdown_connections 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/3002801006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100fe0 0x7fb40410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/3002801006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404100620 0x7fb404100aa0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/3002801006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404106630 0x7fb404106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 -- 172.21.15.45:0/3002801006 >> 172.21.15.45:0/3002801006 conn(0x7fb4040776a0 msgr2=0x7fb404077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 -- 172.21.15.45:0/3002801006 shutdown_connections 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.133+0000 7fb40bc9a640 1 -- 172.21.15.45:0/3002801006 wait complete. 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 Processor -- start 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 -- start start 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100620 0x7fb4041980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404100fe0 0x7fb404198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb404110a40 con 0x7fb404106630 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041108c0 con 0x7fb404100620 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40bc9a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb404110bc0 con 0x7fb404100fe0 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52288/0 (socket says 172.21.15.45:52288) 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 -- 172.21.15.45:0/1152156662 learned_addr learned my addr 172.21.15.45:0/1152156662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb409235640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404100fe0 0x7fb404198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb409a36640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100620 0x7fb4041980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 -- 172.21.15.45:0/1152156662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404100fe0 msgr2=0x7fb404198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404100fe0 0x7fb404198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 -- 172.21.15.45:0/1152156662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100620 msgr2=0x7fb4041980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100620 0x7fb4041980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.134+0000 7fb40a237640 1 -- 172.21.15.45:0/1152156662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40419d0c0 con 0x7fb404106630 2024-04-25T00:31:50.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb40a237640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00eb10 tx=0x7fb3fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb3faffd640 1 -- 172.21.15.45:0/1152156662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc004b20 con 0x7fb404106630 2024-04-25T00:31:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40419d3b0 con 0x7fb404106630 2024-04-25T00:31:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb3faffd640 1 -- 172.21.15.45:0/1152156662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc004cc0 con 0x7fb404106630 2024-04-25T00:31:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb3faffd640 1 -- 172.21.15.45:0/1152156662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc010760 con 0x7fb404106630 2024-04-25T00:31:50.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.135+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a4c40 con 0x7fb404106630 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.137+0000 7fb3faffd640 1 -- 172.21.15.45:0/1152156662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb3fc005020 con 0x7fb404106630 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.137+0000 7fb40e9a7740 1 Processor -- start 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 -- start start 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 0x5651d10ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 0x5651d10bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5651d10126b0 con 0x5651d10ade30 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5651d0fba880 con 0x5651d10bca30 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5651d100e670 con 0x5651d10ad510 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb409a36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 0x5651d10bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb409a36640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 0x5651d10bc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52300/0 (socket says 172.21.15.45:52300) 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb409a36640 1 -- 172.21.15.45:0/2422881476 learned_addr learned my addr 172.21.15.45:0/2422881476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb3faffd640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 0x7fb3f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb409235640 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 0x5651d10ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.138+0000 7fb409235640 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb3faffd640 1 -- 172.21.15.45:0/1152156662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb3fc0cfef0 con 0x7fb404106630 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb40a237640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 0x7fb3f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 -- 172.21.15.45:0/2422881476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 msgr2=0x5651d10bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 -- 172.21.15.45:0/2422881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 msgr2=0x5651d10bc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 0x5651d10bc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 -- 172.21.15.45:0/2422881476 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d0df07d0 con 0x5651d10ad510 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb409235640 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 0x5651d10ad8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009900 tx=0x7fb3f4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98581f60a18a1cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb40a237640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 0x7fb3f00a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cd330 tx=0x7fb3fc00efd0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb3effff640 1 -- 172.21.15.45:0/2422881476 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4035040 con 0x5651d10ad510 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb3effff640 1 -- 172.21.15.45:0/2422881476 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f4031840 con 0x5651d10ad510 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.139+0000 7fb3effff640 1 -- 172.21.15.45:0/2422881476 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f403c620 con 0x5651d10ad510 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2422881476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 msgr2=0x5651d10ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 0x5651d10ad8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009900 tx=0x7fb3f4005b50 comp rx=0 tx=0).stop 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2422881476 shutdown_connections 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ade30 0x5651d10bc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2422881476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ad510 0x5651d10ad8f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2422881476 >> 172.21.15.45:0/2422881476 conn(0x5651d10a9950 msgr2=0x5651d10a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2422881476 shutdown_connections 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2422881476 wait complete. 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 -- start start 2024-04-25T00:31:50.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.140+0000 7fb40e9a7740 1 Processor -- start 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5651d10bf810 con 0x5651d10ad510 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5651d0fcdf50 con 0x5651d10ade30 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5651d10bf990 con 0x5651d10bca30 2024-04-25T00:31:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb40e9a7740 10 client.0 _ll_register_callbacks cb 0x5651d1009000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52314/0 (socket says 172.21.15.45:52314) 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 -- 172.21.15.45:0/2897031800 learned_addr learned my addr 172.21.15.45:0/2897031800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409a36640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 -- 172.21.15.45:0/2897031800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 msgr2=0x5651d10ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 -- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 msgr2=0x5651d10b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.141+0000 7fb409235640 1 -- 172.21.15.45:0/2897031800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d0d263f0 con 0x5651d10ad510 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb409235640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4031b90 tx=0x7fb3f403de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdbfcd407fbab83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4051060 con 0x5651d10ad510 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb3f403d850 con 0x5651d10ad510 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f4031840 con 0x5651d10ad510 2024-04-25T00:31:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f4058050 con 0x5651d10ad510 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb40e9a7740 10 client.18489 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651d0ce60f0 con 0x5651d10ad510 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.142+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f4031840 con 0x5651d10ad510 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb3f403a040 con 0x5651d10ad510 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 10 client.18489 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 20 client.18489 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d10ccb40 0x5651d10cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 4 client.18489 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d101fbf0 con 0x5651d10ccb40 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb409a36640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d10ccb40 0x5651d10cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb409a36640 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d10ccb40 0x5651d10cef30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdbfc7dbfcbe510 server_cookie=18102525b7ad8952 in_seq=0 out_seq=0 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.143+0000 7fb3ed7fa640 10 client.18489 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.144+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/2897031800 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651d101fbf0 con 0x5651d10ccb40 2024-04-25T00:31:50.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.144+0000 7fb3ed7fa640 10 client.18489 handle_command_reply: tid=0 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.144+0000 7fb40e9a7740 1 client.18489 shutdown 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.144+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 msgr2=0x5651d10b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.144+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4031b90 tx=0x7fb3f403de20 comp rx=0 tx=0).stop 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 shutdown_connections 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d10ccb40 0x5651d10cef30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2897031800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10b98d0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb3ed7fa640 0 client.18489 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 >> 172.21.15.45:0/2897031800 conn(0x5651d10a9950 msgr2=0x5651d10af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 shutdown_connections 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2897031800 wait complete. 2024-04-25T00:31:50.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.145+0000 7fb40e9a7740 20 client.18489 trim_cache size 0 max 16384 2024-04-25T00:31:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 Processor -- start 2024-04-25T00:31:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 -- start start 2024-04-25T00:31:50.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ac9c0 con 0x5651d10ad510 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ae500 con 0x5651d10ade30 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ad090 con 0x5651d10bca30 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52318/0 (socket says 172.21.15.45:52318) 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 -- 172.21.15.45:0/2441788593 learned_addr learned my addr 172.21.15.45:0/2441788593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409a36640 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 -- 172.21.15.45:0/2441788593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 msgr2=0x5651d10b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 -- 172.21.15.45:0/2441788593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 msgr2=0x5651d10b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409235640 1 -- 172.21.15.45:0/2441788593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d10adc60 con 0x5651d10ad510 2024-04-25T00:31:50.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.163+0000 7fb409a36640 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb409235640 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005ab0 tx=0x7fb3f403de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36510140a70f5ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb3ef7fe640 1 -- 172.21.15.45:0/2441788593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4035070 con 0x5651d10ad510 2024-04-25T00:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb3ef7fe640 1 -- 172.21.15.45:0/2441788593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f4003ad0 con 0x5651d10ad510 2024-04-25T00:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb3ef7fe640 1 -- 172.21.15.45:0/2441788593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f403da70 con 0x5651d10ad510 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2441788593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 msgr2=0x5651d10bc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4005ab0 tx=0x7fb3f403de90 comp rx=0 tx=0).stop 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2441788593 shutdown_connections 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10bca30 0x5651d10b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10ade30 0x5651d10b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/2441788593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10bc000 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2441788593 >> 172.21.15.45:0/2441788593 conn(0x5651d10a9950 msgr2=0x5651d10c8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.164+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2441788593 shutdown_connections 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 -- 172.21.15.45:0/2441788593 wait complete. 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 -- start start 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 Processor -- start 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ade30 0x5651d10d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ade30 con 0x5651d10ad510 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ad760 con 0x5651d10bca30 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041aa9c0 con 0x5651d10ade30 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb40e9a7740 10 client.0 _ll_register_callbacks cb 0x7fb400009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb409235640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52326/0 (socket says 172.21.15.45:52326) 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.165+0000 7fb409235640 1 -- 172.21.15.45:0/551225991 learned_addr learned my addr 172.21.15.45:0/551225991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409a36640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ade30 0x5651d10d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 -- 172.21.15.45:0/551225991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ade30 msgr2=0x5651d10d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ade30 0x5651d10d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 -- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 msgr2=0x5651d10cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 -- 172.21.15.45:0/551225991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d10cfba0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb409235640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fb3f402a940 tx=0x7fb3f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cbd073d8986533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4035070 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fb3f40037f0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f4003ad0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f40318d0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb40e9a7740 10 client.18495 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651d11328a0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.166+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f4003ad0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb3f404aba0 con 0x5651d10ad510 2024-04-25T00:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 10 client.18495 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 20 client.18495 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d1132dc0 0x5651d11351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 4 client.18495 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651d1019f40 con 0x5651d1132dc0 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb409a36640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d1132dc0 0x5651d11351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb409a36640 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d1132dc0 0x5651d11351b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23727455d09e08df server_cookie=f597f12bbe5843e in_seq=0 out_seq=0 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb3ed7fa640 10 client.18495 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb3ed7fa640 1 -- 172.21.15.45:0/551225991 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651d1019f40 con 0x5651d1132dc0 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb3ed7fa640 10 client.18495 handle_command_reply: tid=0 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.167+0000 7fb40e9a7740 1 client.18495 shutdown 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 msgr2=0x5651d10d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fb3f402a940 tx=0x7fb3f4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 shutdown_connections 2024-04-25T00:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651d10bca30 0x5651d10cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651d10ade30 0x5651d10d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x5651d1132dc0 0x5651d11351b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 --2- 172.21.15.45:0/551225991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651d10ad510 0x5651d10d1f10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 >> 172.21.15.45:0/551225991 conn(0x5651d10a9950 msgr2=0x5651d10a8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb3ed7fa640 0 client.18495 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 shutdown_connections 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 1 -- 172.21.15.45:0/551225991 wait complete. 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40e9a7740 20 client.18495 trim_cache size 0 max 16384 2024-04-25T00:31:50.174 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:50.175 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:31:50.175 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:31:50.175 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 msgr2=0x7fb3f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 0x7fb3f00a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cd330 tx=0x7fb3fc00efd0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 msgr2=0x7fb40419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00eb10 tx=0x7fb3fc00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 shutdown_connections 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb3f00a2d00 0x7fb3f00a51c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb404106630 0x7fb40419c990 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb404100fe0 0x7fb404198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 --2- 172.21.15.45:0/1152156662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb404100620 0x7fb4041980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.168+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 >> 172.21.15.45:0/1152156662 conn(0x7fb4040776a0 msgr2=0x7fb40406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.169+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 shutdown_connections 2024-04-25T00:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.169+0000 7fb40bc9a640 1 -- 172.21.15.45:0/1152156662 wait complete. 2024-04-25T00:31:50.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:31:50.180 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:31:50.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 -- 172.21.15.45:0/525778808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec100690 msgr2=0x7fc5ec100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 --2- 172.21.15.45:0/525778808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec100690 0x7fc5ec100b10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009930 tx=0x7fc5d402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 -- 172.21.15.45:0/525778808 shutdown_connections 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 --2- 172.21.15.45:0/525778808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec101050 0x7fc5ec10d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 --2- 172.21.15.45:0/525778808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec100690 0x7fc5ec100b10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 --2- 172.21.15.45:0/525778808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec106650 0x7fc5ec106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 -- 172.21.15.45:0/525778808 >> 172.21.15.45:0/525778808 conn(0x7fc5ec0777a0 msgr2=0x7fc5ec077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 -- 172.21.15.45:0/525778808 shutdown_connections 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.282+0000 7fc5f177e640 1 -- 172.21.15.45:0/525778808 wait complete. 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 Processor -- start 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 -- start start 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec100690 0x7fc5ec198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec106650 0x7fc5ec1a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec110a60 con 0x7fc5ec101050 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1108e0 con 0x7fc5ec100690 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5f177e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec110be0 con 0x7fc5ec106650 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52342/0 (socket says 172.21.15.45:52342) 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 -- 172.21.15.45:0/1716465064 learned_addr learned my addr 172.21.15.45:0/1716465064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5eaffd640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec100690 0x7fc5ec198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5eb7fe640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec106650 0x7fc5ec1a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 -- 172.21.15.45:0/1716465064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec106650 msgr2=0x7fc5ec1a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec106650 0x7fc5ec1a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 -- 172.21.15.45:0/1716465064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec100690 msgr2=0x7fc5ec198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec100690 0x7fc5ec198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 -- 172.21.15.45:0/1716465064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ec1a44a0 con 0x7fc5ec101050 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.283+0000 7fc5e3fff640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009230 tx=0x7fc5d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.284+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d403f030 con 0x7fc5ec101050 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.284+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d4003ad0 con 0x7fc5ec101050 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.284+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ec1a4670 con 0x7fc5ec101050 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.284+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4035070 con 0x7fc5ec101050 2024-04-25T00:31:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.284+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ec1a4ae0 con 0x7fc5ec101050 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.285+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ec101d80 con 0x7fc5ec101050 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.286+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc5d4042030 con 0x7fc5ec101050 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.286+0000 7fc5e8ff9640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 0x7fc5d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.286+0000 7fc5eaffd640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 0x7fc5d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.287+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fc5d40efd00 con 0x7fc5ec101050 2024-04-25T00:31:50.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.287+0000 7fc5eaffd640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 0x7fc5d80a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005d00 tx=0x7fc5dc00b410 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.291+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5d40bc560 con 0x7fc5ec101050 2024-04-25T00:31:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.516+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5ec108bd0 con 0x7fc5ec101050 2024-04-25T00:31:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.517+0000 7fc5e8ff9640 1 -- 172.21.15.45:0/1716465064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7fc5d40bbe60 con 0x7fc5ec101050 2024-04-25T00:31:50.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:31:50.518 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:31:50.518 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 msgr2=0x7fc5d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 0x7fc5d80a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005d00 tx=0x7fc5dc00b410 comp rx=0 tx=0).stop 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 msgr2=0x7fc5ec199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009230 tx=0x7fc5d4004270 comp rx=0 tx=0).stop 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 shutdown_connections 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc5d80a2d00 0x7fc5d80a51c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5ec106650 0x7fc5ec1a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5ec101050 0x7fc5ec199420 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 --2- 172.21.15.45:0/1716465064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5ec100690 0x7fc5ec198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 >> 172.21.15.45:0/1716465064 conn(0x7fc5ec0777a0 msgr2=0x7fc5ec103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 shutdown_connections 2024-04-25T00:31:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.519+0000 7fc5f177e640 1 -- 172.21.15.45:0/1716465064 wait complete. 2024-04-25T00:31:50.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 -- 172.21.15.45:0/363538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c100fe0 msgr2=0x7f008c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 --2- 172.21.15.45:0/363538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c100fe0 0x7f008c10d4f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0084009cd0 tx=0x7f008402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 -- 172.21.15.45:0/363538257 shutdown_connections 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 --2- 172.21.15.45:0/363538257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c100fe0 0x7f008c10d4f0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 --2- 172.21.15.45:0/363538257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 --2- 172.21.15.45:0/363538257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c106630 0x7f008c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 -- 172.21.15.45:0/363538257 >> 172.21.15.45:0/363538257 conn(0x7f008c0776a0 msgr2=0x7f008c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 -- 172.21.15.45:0/363538257 shutdown_connections 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.635+0000 7f0092076640 1 -- 172.21.15.45:0/363538257 wait complete. 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 Processor -- start 2024-04-25T00:31:50.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 -- start start 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c100fe0 0x7f008c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f008c110a40 con 0x7f008c106630 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1108c0 con 0x7f008c100620 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f0092076640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f008c110bc0 con 0x7f008c100fe0 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52354/0 (socket says 172.21.15.45:52354) 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 -- 172.21.15.45:0/2039145471 learned_addr learned my addr 172.21.15.45:0/2039145471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008affd640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c100fe0 0x7f008c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 -- 172.21.15.45:0/2039145471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c100fe0 msgr2=0x7f008c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c100fe0 0x7f008c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 -- 172.21.15.45:0/2039145471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 msgr2=0x7f008c196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.636+0000 7f008bfff640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f008bfff640 1 -- 172.21.15.45:0/2039145471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c19bde0 con 0x7f008c106630 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f008bfff640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0084005cc0 tx=0x7f0084003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f0088ff9640 1 -- 172.21.15.45:0/2039145471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084004040 con 0x7f008c106630 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f0088ff9640 1 -- 172.21.15.45:0/2039145471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00840046c0 con 0x7f008c106630 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c19c070 con 0x7f008c106630 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f0088ff9640 1 -- 172.21.15.45:0/2039145471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084030e70 con 0x7f008c106630 2024-04-25T00:31:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.637+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1a3960 con 0x7f008c106630 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.639+0000 7f0088ff9640 1 -- 172.21.15.45:0/2039145471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f00840305a0 con 0x7f008c106630 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 Processor -- start 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 -- start start 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0088ff9640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 0x7f00740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0088ff9640 1 -- 172.21.15.45:0/2039145471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f008402e030 con 0x7f008c106630 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008affd640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 0x7f00740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606b37f8670 con 0x5606b38a0780 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5606b37a0970 con 0x5606b3893c80 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f0094d89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5606b37f4630 con 0x5606b38932d0 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52364/0 (socket says 172.21.15.45:52364) 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 -- 172.21.15.45:0/2958002099 learned_addr learned my addr 172.21.15.45:0/2958002099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008affd640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 -- 172.21.15.45:0/2958002099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 msgr2=0x5606b38936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 -- 172.21.15.45:0/2958002099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 msgr2=0x5606b38a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008bfff640 1 -- 172.21.15.45:0/2958002099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b37c4400 con 0x5606b38a0780 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.640+0000 7f008affd640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f008affd640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 0x7f00740a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f007c0058c0 tx=0x7f007c0138f0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f008bfff640 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f00840c5210 tx=0x7f008402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7431363044f63af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0071ffb640 1 -- 172.21.15.45:0/2958002099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840bcad0 con 0x5606b38a0780 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0071ffb640 1 -- 172.21.15.45:0/2958002099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0084036040 con 0x5606b38a0780 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0071ffb640 1 -- 172.21.15.45:0/2958002099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840edfc0 con 0x5606b38a0780 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 -- 172.21.15.45:0/2958002099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 msgr2=0x5606b38a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f00840c5210 tx=0x7f008402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 -- 172.21.15.45:0/2958002099 shutdown_connections 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a2b70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 --2- 172.21.15.45:0/2958002099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.641+0000 7f0094d89740 1 -- 172.21.15.45:0/2958002099 >> 172.21.15.45:0/2958002099 conn(0x5606b388f8a0 msgr2=0x5606b388fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- 172.21.15.45:0/2958002099 shutdown_connections 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- 172.21.15.45:0/2958002099 wait complete. 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- start start 2024-04-25T00:31:50.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 Processor -- start 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b389baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38a0780 0x5606b389c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606b3895fd0 con 0x5606b38932d0 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5606b37b4050 con 0x5606b3893c80 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5606b3896150 con 0x5606b38a0780 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f0094d89740 10 client.0 _ll_register_callbacks cb 0x5606b350bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f008b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f008b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52366/0 (socket says 172.21.15.45:52366) 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f008b7fe640 1 -- 172.21.15.45:0/2101945009 learned_addr learned my addr 172.21.15.45:0/2101945009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f008affd640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b389baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.642+0000 7f008bfff640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38a0780 0x5606b389c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 -- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38a0780 msgr2=0x5606b389c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38a0780 0x5606b389c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 -- 172.21.15.45:0/2101945009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 msgr2=0x5606b389baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b389baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 -- 172.21.15.45:0/2101945009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b3785dc0 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f008b7fe640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0080003c20 tx=0x7f00800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f29911ca746df89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f006affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080003e50 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f0080005020 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0084036040 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0080005220 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f0094d89740 10 client.18513 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606b350c3f0 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.643+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0084036040 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f008001b6e0 con 0x5606b38932d0 2024-04-25T00:31:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 10 client.18513 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 10 client.18513 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 20 client.18513 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b38b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 4 client.18513 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606b3805bb0 con 0x5606b38b27d0 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f008affd640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b38b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f006affd640 10 client.18513 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.644+0000 7f008affd640 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b38b4c70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9a3913d3c9cedd4 server_cookie=a197bc5edb4381c2 in_seq=0 out_seq=0 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f006affd640 1 -- 172.21.15.45:0/2101945009 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606b3805bb0 con 0x5606b38b27d0 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f006affd640 10 client.18513 handle_command_reply: tid=0 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 client.18513 shutdown 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 msgr2=0x5606b389b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0080003c20 tx=0x7f00800117b0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 shutdown_connections 2024-04-25T00:31:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b38b4c70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38a0780 0x5606b389c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b389baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f006affd640 0 client.18513 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 --2- 172.21.15.45:0/2101945009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38932d0 0x5606b389b5b0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 >> 172.21.15.45:0/2101945009 conn(0x5606b388f8a0 msgr2=0x5606b3895710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 shutdown_connections 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 1 -- 172.21.15.45:0/2101945009 wait complete. 2024-04-25T00:31:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.645+0000 7f0094d89740 20 client.18513 trim_cache size 0 max 16384 2024-04-25T00:31:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 Processor -- start 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 -- start start 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ac920 con 0x5606b38a0780 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ae460 con 0x5606b3893c80 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f0094d89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1acff0 con 0x5606b38932d0 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52370/0 (socket says 172.21.15.45:52370) 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 -- 172.21.15.45:0/2685377562 learned_addr learned my addr 172.21.15.45:0/2685377562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 -- 172.21.15.45:0/2685377562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 msgr2=0x5606b38ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 -- 172.21.15.45:0/2685377562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 msgr2=0x5606b38baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 -- 172.21.15.45:0/2685377562 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b35d67d0 con 0x5606b38a0780 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f008bfff640 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f00840bc7d0 tx=0x7f00840b8ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52b5d10cf2c0510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f00717fa640 1 -- 172.21.15.45:0/2685377562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084031e60 con 0x5606b38a0780 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f00717fa640 1 -- 172.21.15.45:0/2685377562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0084036040 con 0x5606b38a0780 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.659+0000 7f00717fa640 1 -- 172.21.15.45:0/2685377562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840edde0 con 0x5606b38a0780 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- 172.21.15.45:0/2685377562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 msgr2=0x5606b3898810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f00840bc7d0 tx=0x7f00840b8ea0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- 172.21.15.45:0/2685377562 shutdown_connections 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b3898810 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b3893c80 0x5606b38baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 --2- 172.21.15.45:0/2685377562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b38932d0 0x5606b38ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- 172.21.15.45:0/2685377562 >> 172.21.15.45:0/2685377562 conn(0x5606b388f8a0 msgr2=0x5606b3896c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- 172.21.15.45:0/2685377562 shutdown_connections 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- 172.21.15.45:0/2685377562 wait complete. 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.660+0000 7f0094d89740 1 -- start start 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 Processor -- start 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 0x5606b38adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 0x5606b38ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1add90 con 0x5606b38a0780 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1ad6c0 con 0x5606b38932d0 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1aa920 con 0x5606b3893c80 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 10 client.0 _ll_register_callbacks cb 0x7f00800041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52386/0 (socket says 172.21.15.45:52386) 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 -- 172.21.15.45:0/3090469869 learned_addr learned my addr 172.21.15.45:0/3090469869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008b7fe640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 0x5606b38adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008affd640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 0x5606b38ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 -- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 msgr2=0x5606b38ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 0x5606b38ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 -- 172.21.15.45:0/3090469869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 msgr2=0x5606b38adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 0x5606b38adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 -- 172.21.15.45:0/3090469869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b38a9a00 con 0x5606b38a0780 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008affd640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 0x5606b38ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008b7fe640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 0x5606b38adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f008bfff640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f0084081fd0 tx=0x7f008404d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc565beaa202a14f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f006affd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840edfc0 con 0x5606b38a0780 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f00840f0440 con 0x5606b38a0780 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.661+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0084036040 con 0x5606b38a0780 2024-04-25T00:31:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00840d9030 con 0x5606b38a0780 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 10 client.18519 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606b3918a50 con 0x5606b38a0780 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0084036040 con 0x5606b38a0780 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f008404d020 con 0x5606b38a0780 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 10 client.18519 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 10 client.18519 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 20 client.18519 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b391adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 4 client.18519 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606b37fff00 con 0x5606b38b27d0 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.662+0000 7f008b7fe640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b391adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f006affd640 10 client.18519 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f008b7fe640 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b391adf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f05ac5fded91fe1a server_cookie=d5a9f3543aa7ff95 in_seq=0 out_seq=0 2024-04-25T00:31:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f006affd640 1 -- 172.21.15.45:0/3090469869 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606b37fff00 con 0x5606b38b27d0 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f006affd640 10 client.18519 handle_command_reply: tid=0 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 client.18519 shutdown 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 msgr2=0x5606b38a9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f0084081fd0 tx=0x7f008404d650 comp rx=0 tx=0).stop 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 shutdown_connections 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x5606b38b27d0 0x5606b391adf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606b38a0780 0x5606b38a9200 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606b3893c80 0x5606b38ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 --2- 172.21.15.45:0/3090469869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606b38932d0 0x5606b38adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 >> 172.21.15.45:0/3090469869 conn(0x5606b388f8a0 msgr2=0x5606b3894e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f006affd640 0 client.18519 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 shutdown_connections 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 1 -- 172.21.15.45:0/3090469869 wait complete. 2024-04-25T00:31:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.663+0000 7f0094d89740 20 client.18519 trim_cache size 0 max 16384 2024-04-25T00:31:50.671 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 msgr2=0x7f00740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 0x7f00740a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f007c0058c0 tx=0x7f007c0138f0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 msgr2=0x7f008c19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0084005cc0 tx=0x7f0084003970 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 shutdown_connections 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00740a2d00 0x7f00740a51c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f008c106630 0x7f008c19b6b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f008c100fe0 0x7f008c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 --2- 172.21.15.45:0/2039145471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100620 0x7f008c196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 >> 172.21.15.45:0/2039145471 conn(0x7f008c0776a0 msgr2=0x7f008c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 shutdown_connections 2024-04-25T00:31:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.664+0000 7f0092076640 1 -- 172.21.15.45:0/2039145471 wait complete. 2024-04-25T00:31:50.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:31:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 -- 172.21.15.45:0/4072526091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314102120 msgr2=0x7f731410e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 --2- 172.21.15.45:0/4072526091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314102120 0x7f731410e630 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f730c009930 tx=0x7f730c02a420 comp rx=0 tx=0).stop 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 -- 172.21.15.45:0/4072526091 shutdown_connections 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 --2- 172.21.15.45:0/4072526091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314102120 0x7f731410e630 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 --2- 172.21.15.45:0/4072526091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314101760 0x7f7314101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 --2- 172.21.15.45:0/4072526091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314107770 0x7f7314107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 -- 172.21.15.45:0/4072526091 >> 172.21.15.45:0/4072526091 conn(0x7f73140fd250 msgr2=0x7f73140ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 -- 172.21.15.45:0/4072526091 shutdown_connections 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.779+0000 7f7319f96640 1 -- 172.21.15.45:0/4072526091 wait complete. 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 Processor -- start 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 -- start start 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314101760 0x7f7314073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 0x7f7314073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7314111b40 con 0x7f7314107770 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73141119c0 con 0x7f7314102120 2024-04-25T00:31:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7319f96640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7314111cc0 con 0x7f7314101760 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7313fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7313fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52414/0 (socket says 172.21.15.45:52414) 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f7313fff640 1 -- 172.21.15.45:0/1135831107 learned_addr learned my addr 172.21.15.45:0/1135831107 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.780+0000 7f73137fe640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314101760 0x7f7314073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7312ffd640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 0x7f7314073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 -- 172.21.15.45:0/1135831107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314101760 msgr2=0x7f7314073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314101760 0x7f7314073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 -- 172.21.15.45:0/1135831107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 msgr2=0x7f7314073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 0x7f7314073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 -- 172.21.15.45:0/1135831107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73141a6560 con 0x7f7314107770 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7312ffd640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 0x7f7314073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7313fff640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f730c005ab0 tx=0x7f730c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7310ff9640 1 -- 172.21.15.45:0/1135831107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c03f030 con 0x7f7314107770 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7310ff9640 1 -- 172.21.15.45:0/1135831107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f730c003ce0 con 0x7f7314107770 2024-04-25T00:31:50.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a67f0 con 0x7f7314107770 2024-04-25T00:31:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7310ff9640 1 -- 172.21.15.45:0/1135831107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c035070 con 0x7f7314107770 2024-04-25T00:31:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.781+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a6ce0 con 0x7f7314107770 2024-04-25T00:31:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7310ff9640 1 -- 172.21.15.45:0/1135831107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f730c042030 con 0x7f7314107770 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 Processor -- start 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 -- start start 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc554f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc55449670 con 0x55fc554e4ea0 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fc553f1970 con 0x55fc554e4580 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f731cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc55445630 con 0x55fc554ef570 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52428/0 (socket says 172.21.15.45:52428) 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 -- 172.21.15.45:0/66767690 learned_addr learned my addr 172.21.15.45:0/66767690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7310ff9640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 0x7f72fc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7312ffd640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7312ffd640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc554f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 -- 172.21.15.45:0/66767690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 msgr2=0x55fc554f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc554f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 -- 172.21.15.45:0/66767690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 msgr2=0x55fc554e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554e4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f73137fe640 1 -- 172.21.15.45:0/66767690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc55415400 con 0x55fc554e4ea0 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7312ffd640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.784+0000 7f7313fff640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 0x7f72fc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f7310ff9640 1 -- 172.21.15.45:0/1135831107 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f730c0efcb0 con 0x7f7314107770 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f73137fe640 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f73040103d0 tx=0x7f7304010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6c53847c13c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f72f9ffb640 1 -- 172.21.15.45:0/66767690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304003b70 con 0x55fc554e4ea0 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f72f9ffb640 1 -- 172.21.15.45:0/66767690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7304015070 con 0x55fc554e4ea0 2024-04-25T00:31:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f72f9ffb640 1 -- 172.21.15.45:0/66767690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304011920 con 0x55fc554e4ea0 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f731cd76740 1 -- 172.21.15.45:0/66767690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 msgr2=0x55fc554ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f7313fff640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 0x7f72fc0a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ed0f0 tx=0x7f730c009260 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.785+0000 7f731cd76740 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f73040103d0 tx=0x7f7304010890 comp rx=0 tx=0).stop 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- 172.21.15.45:0/66767690 shutdown_connections 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc554f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4ea0 0x55fc554ef030 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- 172.21.15.45:0/66767690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- 172.21.15.45:0/66767690 >> 172.21.15.45:0/66767690 conn(0x55fc554e08a0 msgr2=0x55fc554e0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- 172.21.15.45:0/66767690 shutdown_connections 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- 172.21.15.45:0/66767690 wait complete. 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- start start 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 Processor -- start 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4580 0x55fc554eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554ec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc554f2360 con 0x55fc554ef570 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fc55405050 con 0x55fc554e4ea0 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc554f24e0 con 0x55fc554e4580 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.786+0000 7f731cd76740 10 client.0 _ll_register_callbacks cb 0x55fc5515cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52444/0 (socket says 172.21.15.45:52444) 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f73137fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 -- 172.21.15.45:0/2869453893 learned_addr learned my addr 172.21.15.45:0/2869453893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4580 0x55fc554eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 -- 172.21.15.45:0/2869453893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4580 msgr2=0x55fc554eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4580 0x55fc554eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 -- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 msgr2=0x55fc554ec230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554ec230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 -- 172.21.15.45:0/2869453893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc553d6dc0 con 0x55fc554ef570 2024-04-25T00:31:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f7312ffd640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f730800d590 tx=0x7f7308004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b8198d344bd3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f72f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308011c40 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f7308011de0 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7304015070 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7308004020 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f731cd76740 10 client.18531 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.787+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc5515d3f0 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7304015070 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73080048c0 con 0x55fc554ef570 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 10 client.18531 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 10 client.18531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 20 client.18531 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc554ff540 0x55fc55501970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 4 client.18531 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc55456bb0 con 0x55fc554ff540 2024-04-25T00:31:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.788+0000 7f73137fe640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc554ff540 0x55fc55501970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f73137fe640 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc554ff540 0x55fc55501970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=14b6363f3a6e83d server_cookie=108f48ca9baafcac in_seq=0 out_seq=0 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f72f2ffd640 10 client.18531 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f72f2ffd640 1 -- 172.21.15.45:0/2869453893 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc55456bb0 con 0x55fc554ff540 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f72f2ffd640 10 client.18531 handle_command_reply: tid=0 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f731cd76740 1 client.18531 shutdown 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 msgr2=0x55fc554ec840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f730800d590 tx=0x7f7308004730 comp rx=0 tx=0).stop 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.789+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 shutdown_connections 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc554ff540 0x55fc55501970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc554ec840 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554ec230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 --2- 172.21.15.45:0/2869453893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4580 0x55fc554eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f72f2ffd640 0 client.18531 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 >> 172.21.15.45:0/2869453893 conn(0x55fc554e08a0 msgr2=0x55fc554dfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 shutdown_connections 2024-04-25T00:31:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 1 -- 172.21.15.45:0/2869453893 wait complete. 2024-04-25T00:31:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.790+0000 7f731cd76740 20 client.18531 trim_cache size 0 max 16384 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 Processor -- start 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 -- start start 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 0x55fc554eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b0e30 con 0x55fc554ef570 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b2970 con 0x55fc554e4580 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f731cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b1500 con 0x55fc554e4ea0 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52458/0 (socket says 172.21.15.45:52458) 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 -- 172.21.15.45:0/21426666 learned_addr learned my addr 172.21.15.45:0/21426666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f73137fe640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 0x55fc554eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 -- 172.21.15.45:0/21426666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 msgr2=0x55fc554eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 0x55fc554eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 -- 172.21.15.45:0/21426666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 msgr2=0x55fc554fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.803+0000 7f7312ffd640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f7312ffd640 1 -- 172.21.15.45:0/21426666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc552277d0 con 0x55fc554ef570 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f73137fe640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 0x55fc554eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f7312ffd640 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f730800dc40 tx=0x7f7308013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c117ac876e244a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f72f97fa640 1 -- 172.21.15.45:0/21426666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308005e50 con 0x55fc554ef570 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f72f97fa640 1 -- 172.21.15.45:0/21426666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308017040 con 0x55fc554ef570 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f72f97fa640 1 -- 172.21.15.45:0/21426666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004420 con 0x55fc554ef570 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 -- 172.21.15.45:0/21426666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 msgr2=0x55fc55507240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f730800dc40 tx=0x7f7308013f90 comp rx=0 tx=0).stop 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 -- 172.21.15.45:0/21426666 shutdown_connections 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554ef570 0x55fc55507240 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554e4ea0 0x55fc554eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 --2- 172.21.15.45:0/21426666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4580 0x55fc554fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 -- 172.21.15.45:0/21426666 >> 172.21.15.45:0/21426666 conn(0x55fc554e08a0 msgr2=0x55fc554e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 -- 172.21.15.45:0/21426666 shutdown_connections 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.804+0000 7f731cd76740 1 -- 172.21.15.45:0/21426666 wait complete. 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 -- start start 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 Processor -- start 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc5555da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b22a0 con 0x55fc554e4580 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73141b1bd0 con 0x55fc554e4ea0 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73141aee30 con 0x55fc554ef570 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f731cd76740 10 client.0 _ll_register_callbacks cb 0x7f730800db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52474/0 (socket says 172.21.15.45:52474) 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.805+0000 7f7312ffd640 1 -- 172.21.15.45:0/714818781 learned_addr learned my addr 172.21.15.45:0/714818781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f73137fe640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 -- 172.21.15.45:0/714818781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 msgr2=0x55fc5555da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc5555da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 -- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 msgr2=0x55fc554f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554f2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 -- 172.21.15.45:0/714818781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5555e030 con 0x55fc554e4580 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f7312ffd640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7308004cc0 tx=0x7f730800d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b968b961af2587b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f72f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308005e50 con 0x55fc554e4580 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f7308004f30 con 0x55fc554e4580 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308017040 con 0x55fc554e4580 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7308005130 con 0x55fc554e4580 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f731cd76740 10 client.18537 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc55565310 con 0x55fc554e4580 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.806+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308017040 con 0x55fc554e4580 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f730802b460 con 0x55fc554e4580 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 10 client.18537 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 10 client.18537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 20 client.18537 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc555659b0 0x55fc55567dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 4 client.18537 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc55450f00 con 0x55fc555659b0 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f73137fe640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc555659b0 0x55fc55567dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f73137fe640 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc555659b0 0x55fc55567dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d24d5d22612a106 server_cookie=43fd51dfc8205571 in_seq=0 out_seq=0 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.807+0000 7f72f2ffd640 10 client.18537 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f72f2ffd640 1 -- 172.21.15.45:0/714818781 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc55450f00 con 0x55fc555659b0 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f72f2ffd640 10 client.18537 handle_command_reply: tid=0 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 client.18537 shutdown 2024-04-25T00:31:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 msgr2=0x55fc554f2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7308004cc0 tx=0x7f730800d390 comp rx=0 tx=0).stop 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 shutdown_connections 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55fc555659b0 0x55fc55567dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc554ef570 0x55fc5555da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f72f2ffd640 0 client.18537 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc554e4ea0 0x55fc554f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 --2- 172.21.15.45:0/714818781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc554e4580 0x55fc554f2980 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 >> 172.21.15.45:0/714818781 conn(0x55fc554e08a0 msgr2=0x55fc554e6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 shutdown_connections 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 1 -- 172.21.15.45:0/714818781 wait complete. 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f731cd76740 20 client.18537 trim_cache size 0 max 16384 2024-04-25T00:31:50.814 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 msgr2=0x7f72fc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 0x7f72fc0a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ed0f0 tx=0x7f730c009260 comp rx=0 tx=0).stop 2024-04-25T00:31:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 msgr2=0x7f73140744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f730c005ab0 tx=0x7f730c0042f0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 shutdown_connections 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f72fc0a2d00 0x7f72fc0a51c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7314107770 0x7f73140744c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7314102120 0x7f7314073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.808+0000 7f7319f96640 1 --2- 172.21.15.45:0/1135831107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7314101760 0x7f7314073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.809+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 >> 172.21.15.45:0/1135831107 conn(0x7f73140fd250 msgr2=0x7f73140fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.809+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 shutdown_connections 2024-04-25T00:31:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.809+0000 7f7319f96640 1 -- 172.21.15.45:0/1135831107 wait complete. 2024-04-25T00:31:50.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1049261092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60107720 msgr2=0x7f4a60107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1049261092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60107720 0x7f4a60107b00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009930 tx=0x7f4a5002a420 comp rx=0 tx=0).stop 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1049261092 shutdown_connections 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1049261092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a601020d0 0x7f4a6010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1049261092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60101710 0x7f4a60101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1049261092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a60107720 0x7f4a60107b00 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1049261092 >> 172.21.15.45:0/1049261092 conn(0x7f4a600fd180 msgr2=0x7f4a600ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1049261092 shutdown_connections 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1049261092 wait complete. 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.923+0000 7f4a66ec3640 1 Processor -- start 2024-04-25T00:31:50.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 -- start start 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60101710 0x7f4a601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 0x7f4a6019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111b50 con 0x7f4a601020d0 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601119d0 con 0x7f4a60101710 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a66ec3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60111cd0 con 0x7f4a60107720 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52500/0 (socket says 172.21.15.45:52500) 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 -- 172.21.15.45:0/1523308494 learned_addr learned my addr 172.21.15.45:0/1523308494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60101710 0x7f4a601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 -- 172.21.15.45:0/1523308494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 msgr2=0x7f4a6019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a65460640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 0x7f4a6019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 0x7f4a6019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 -- 172.21.15.45:0/1523308494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60101710 msgr2=0x7f4a601980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60101710 0x7f4a601980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 -- 172.21.15.45:0/1523308494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6019d0c0 con 0x7f4a601020d0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a65460640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 0x7f4a6019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.924+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d770 tx=0x7f4a5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.925+0000 7f4a5dffb640 1 -- 172.21.15.45:0/1523308494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54017070 con 0x7f4a601020d0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.925+0000 7f4a5dffb640 1 -- 172.21.15.45:0/1523308494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a54005500 con 0x7f4a601020d0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.925+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6019d3b0 con 0x7f4a601020d0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.925+0000 7f4a5dffb640 1 -- 172.21.15.45:0/1523308494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54004e00 con 0x7f4a601020d0 2024-04-25T00:31:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.925+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a4c40 con 0x7f4a601020d0 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.927+0000 7f4a5dffb640 1 -- 172.21.15.45:0/1523308494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a54004fa0 con 0x7f4a601020d0 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.927+0000 7f4a69bcc740 1 Processor -- start 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.927+0000 7f4a69bcc740 1 -- start start 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8580 0x55e6dd7d8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd73d670 con 0x55e6dd7e3570 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd6e5970 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd739630 con 0x55e6dd7d8580 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5dffb640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 0x7f4a480a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52506/0 (socket says 172.21.15.45:52506) 2024-04-25T00:31:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 -- 172.21.15.45:0/3237350438 learned_addr learned my addr 172.21.15.45:0/3237350438 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a65460640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 0x7f4a480a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a65460640 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8580 0x55e6dd7d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5dffb640 1 -- 172.21.15.45:0/1523308494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f4a540cf280 con 0x7f4a601020d0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 -- 172.21.15.45:0/3237350438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8580 msgr2=0x55e6dd7d8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8580 0x55e6dd7d8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 -- 172.21.15.45:0/3237350438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 msgr2=0x55e6dd7e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 -- 172.21.15.45:0/3237350438 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd709400 con 0x55e6dd7e3570 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.928+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005760 tx=0x7f4a54009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4d1345b325cb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a65460640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 0x7f4a480a5240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800afd0 tx=0x7f4a5800ad00 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a46ffd640 1 -- 172.21.15.45:0/3237350438 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54020070 con 0x55e6dd7e3570 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a46ffd640 1 -- 172.21.15.45:0/3237350438 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5409b380 con 0x55e6dd7e3570 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a46ffd640 1 -- 172.21.15.45:0/3237350438 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540cc830 con 0x55e6dd7e3570 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3237350438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 msgr2=0x55e6dd7e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4a54005760 tx=0x7f4a54009c30 comp rx=0 tx=0).stop 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3237350438 shutdown_connections 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e5960 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3237350438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8580 0x55e6dd7d8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3237350438 >> 172.21.15.45:0/3237350438 conn(0x55e6dd7d48a0 msgr2=0x55e6dd7d4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3237350438 shutdown_connections 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.929+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3237350438 wait complete. 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 -- start start 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 Processor -- start 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd7e6300 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd6f9050 con 0x55e6dd7d8580 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd7e6480 con 0x55e6dd7e3570 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a69bcc740 10 client.0 _ll_register_callbacks cb 0x55e6dd450be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52508/0 (socket says 172.21.15.45:52508) 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a5ffff640 1 -- 172.21.15.45:0/2688673788 learned_addr learned my addr 172.21.15.45:0/2688673788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a65460640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.930+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 -- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 msgr2=0x55e6dd7e0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 -- 172.21.15.45:0/2688673788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 msgr2=0x55e6dd7e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 -- 172.21.15.45:0/2688673788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd6cadc0 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a65460640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4a54097d50 tx=0x7f4a54009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5932f9e4dedad5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54020040 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f4a540cc830 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5409b380 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a540cca30 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a69bcc740 10 client.18549 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6dd4513f0 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5409b380 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a5409a240 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a69bcc740 10 client.18549 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.931+0000 7f4a69bcc740 10 client.18549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a69bcc740 20 client.18549 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd7f33f0 0x55e6dd7f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a69bcc740 4 client.18549 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6dd74abb0 con 0x55e6dd7f33f0 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd7f33f0 0x55e6dd7f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd7f33f0 0x55e6dd7f57e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0f82aeb65b60fd server_cookie=264042aff0b40169 in_seq=0 out_seq=0 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.932+0000 7f4a3ffff640 10 client.18549 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a3ffff640 1 -- 172.21.15.45:0/2688673788 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6dd74abb0 con 0x55e6dd7f33f0 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a3ffff640 10 client.18549 handle_command_reply: tid=0 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 client.18549 shutdown 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 msgr2=0x55e6dd7e01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4a54097d50 tx=0x7f4a54009e10 comp rx=0 tx=0).stop 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 shutdown_connections 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd7f33f0 0x55e6dd7f57e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e01d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a3ffff640 0 client.18549 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/2688673788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 >> 172.21.15.45:0/2688673788 conn(0x55e6dd7d48a0 msgr2=0x55e6dd7d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.933+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 shutdown_connections 2024-04-25T00:31:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.934+0000 7f4a69bcc740 1 -- 172.21.15.45:0/2688673788 wait complete. 2024-04-25T00:31:50.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.934+0000 7f4a69bcc740 20 client.18549 trim_cache size 0 max 16384 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 Processor -- start 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 -- start start 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7d9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601aeca0 con 0x55e6dd7e3570 2024-04-25T00:31:50.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b07e0 con 0x55e6dd7d8580 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601af370 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a65460640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a65460640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52524/0 (socket says 172.21.15.45:52524) 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a65460640 1 -- 172.21.15.45:0/698895395 learned_addr learned my addr 172.21.15.45:0/698895395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.947+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7d9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 -- 172.21.15.45:0/698895395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 msgr2=0x55e6dd7d9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7d9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 -- 172.21.15.45:0/698895395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 msgr2=0x55e6dd7ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 -- 172.21.15.45:0/698895395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd7d8cd0 con 0x55e6dd7e3570 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7d9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:50.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a65460640 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800a830 tx=0x7f4a5801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38890f987acfe17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a467fc640 1 -- 172.21.15.45:0/698895395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58024030 con 0x55e6dd7e3570 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a467fc640 1 -- 172.21.15.45:0/698895395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5800fda0 con 0x55e6dd7e3570 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a467fc640 1 -- 172.21.15.45:0/698895395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58020d50 con 0x55e6dd7e3570 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a69bcc740 1 -- 172.21.15.45:0/698895395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 msgr2=0x55e6dd7e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800a830 tx=0x7f4a5801c000 comp rx=0 tx=0).stop 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.948+0000 7f4a69bcc740 1 -- 172.21.15.45:0/698895395 shutdown_connections 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e2a10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7d9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/698895395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8580 0x55e6dd7ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- 172.21.15.45:0/698895395 >> 172.21.15.45:0/698895395 conn(0x55e6dd7d48a0 msgr2=0x55e6dd7df200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- 172.21.15.45:0/698895395 shutdown_connections 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- 172.21.15.45:0/698895395 wait complete. 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- start start 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 Processor -- start 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b0110 con 0x55e6dd7d8580 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601afa40 con 0x55e6dd7d8ea0 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601acca0 con 0x55e6dd7e3570 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a69bcc740 10 client.0 _ll_register_callbacks cb 0x7f4a58014600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a64c5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a64c5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52526/0 (socket says 172.21.15.45:52526) 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.949+0000 7f4a64c5f640 1 -- 172.21.15.45:0/3453989873 learned_addr learned my addr 172.21.15.45:0/3453989873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a65460640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 -- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 msgr2=0x55e6dd7e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 -- 172.21.15.45:0/3453989873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 msgr2=0x55e6dd7e7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 -- 172.21.15.45:0/3453989873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd51b7d0 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a64c5f640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006010 tx=0x7f4a50031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a649b32dc80b4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5004a070 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f4a50004020 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5800fda0 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a50055050 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a69bcc740 10 client.18555 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6dd859620 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.950+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5800fda0 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a50003860 con 0x55e6dd7d8580 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 10 client.18555 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 10 client.18555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 20 client.18555 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd859d20 0x55e6dd85c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:50.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 4 client.18555 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6dd744f00 con 0x55e6dd859d20 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd859d20 0x55e6dd85c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a5ffff640 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd859d20 0x55e6dd85c140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2c208021632aed7 server_cookie=c2b5898e2de0c542 in_seq=0 out_seq=0 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a3ffff640 10 client.18555 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a3ffff640 1 -- 172.21.15.45:0/3453989873 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6dd744f00 con 0x55e6dd859d20 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a3ffff640 10 client.18555 handle_command_reply: tid=0 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.951+0000 7f4a69bcc740 1 client.18555 shutdown 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 msgr2=0x55e6dd7e6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006010 tx=0x7f4a50031ad0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 shutdown_connections 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55e6dd859d20 0x55e6dd85c140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6dd7e3570 0x55e6dd7e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6dd7d8ea0 0x55e6dd7e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a3ffff640 0 client.18555 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:31:50.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 --2- 172.21.15.45:0/3453989873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6dd7d8580 0x55e6dd7e6ff0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 >> 172.21.15.45:0/3453989873 conn(0x55e6dd7d48a0 msgr2=0x55e6dd7d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 shutdown_connections 2024-04-25T00:31:50.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 1 -- 172.21.15.45:0/3453989873 wait complete. 2024-04-25T00:31:50.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a69bcc740 20 client.18555 trim_cache size 0 max 16384 2024-04-25T00:31:50.959 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 msgr2=0x7f4a480a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 0x7f4a480a5240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800afd0 tx=0x7f4a5800ad00 comp rx=0 tx=0).stop 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 msgr2=0x7f4a60198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d770 tx=0x7f4a5400dc30 comp rx=0 tx=0).stop 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 shutdown_connections 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f4a480a2d80 0x7f4a480a5240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a60107720 0x7f4a6019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a601020d0 0x7f4a60198610 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 --2- 172.21.15.45:0/1523308494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60101710 0x7f4a601980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:50.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.952+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 >> 172.21.15.45:0/1523308494 conn(0x7f4a600fd180 msgr2=0x7f4a600fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:50.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.953+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 shutdown_connections 2024-04-25T00:31:50.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:50.953+0000 7f4a66ec3640 1 -- 172.21.15.45:0/1523308494 wait complete. 2024-04-25T00:31:50.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:31:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 -- 172.21.15.45:0/3338676163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf001002e0 msgr2=0x7fcf001006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/3338676163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf001002e0 0x7fcf001006c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef402a410 comp rx=0 tx=0).stop 2024-04-25T00:31:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 -- 172.21.15.45:0/3338676163 shutdown_connections 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/3338676163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00106aa0 0x7fcf0010e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/3338676163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00100c90 0x7fcf001063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/3338676163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf001002e0 0x7fcf001006c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 -- 172.21.15.45:0/3338676163 >> 172.21.15.45:0/3338676163 conn(0x7fcf000758d0 msgr2=0x7fcf00075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 -- 172.21.15.45:0/3338676163 shutdown_connections 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.067+0000 7fcf06d7e640 1 -- 172.21.15.45:0/3338676163 wait complete. 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 Processor -- start 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 -- start start 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001002e0 0x7fcf001980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 0x7fcf00198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00111b90 con 0x7fcf0019c990 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00111a10 con 0x7fcf00106aa0 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf06d7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00111d10 con 0x7fcf001002e0 2024-04-25T00:31:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52552/0 (socket says 172.21.15.45:52552) 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 -- 172.21.15.45:0/1249140369 learned_addr learned my addr 172.21.15.45:0/1249140369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001002e0 0x7fcf001980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcefffff640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 0x7fcf00198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 -- 172.21.15.45:0/1249140369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001002e0 msgr2=0x7fcf001980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001002e0 0x7fcf001980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 -- 172.21.15.45:0/1249140369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 msgr2=0x7fcf00198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 0x7fcf00198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcf0531b640 1 -- 172.21.15.45:0/1249140369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0019d480 con 0x7fcf0019c990 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.068+0000 7fcefffff640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 0x7fcf00198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcf0531b640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009fd0 tx=0x7fcef800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcefdffb640 1 -- 172.21.15.45:0/1249140369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8017070 con 0x7fcf0019c990 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcefdffb640 1 -- 172.21.15.45:0/1249140369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8003e50 con 0x7fcf0019c990 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0019d770 con 0x7fcf0019c990 2024-04-25T00:31:51.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcefdffb640 1 -- 172.21.15.45:0/1249140369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80106c0 con 0x7fcf0019c990 2024-04-25T00:31:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.069+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf001a4f10 con 0x7fcf0019c990 2024-04-25T00:31:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.071+0000 7fcefdffb640 1 -- 172.21.15.45:0/1249140369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcef8003970 con 0x7fcf0019c990 2024-04-25T00:31:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.071+0000 7fcf09b69740 1 Processor -- start 2024-04-25T00:31:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.071+0000 7fcf09b69740 1 -- start start 2024-04-25T00:31:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcefdffb640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 0x7fcef00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73e580 0x55d23a73e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73eea0 0x55d23a749030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcefdffb640 1 -- 172.21.15.45:0/1249140369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fcef80cf690 con 0x7fcf0019c990 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcefffff640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 0x7fcef00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d23a6a3670 con 0x55d23a749570 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d23a64b970 con 0x55d23a73eea0 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf09b69740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d23a69f630 con 0x55d23a73e580 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52568/0 (socket says 172.21.15.45:52568) 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 -- 172.21.15.45:0/770468013 learned_addr learned my addr 172.21.15.45:0/770468013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73e580 0x55d23a73e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 -- 172.21.15.45:0/770468013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73e580 msgr2=0x55d23a73e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcefffff640 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73eea0 0x55d23a749030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73e580 0x55d23a73e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 -- 172.21.15.45:0/770468013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73eea0 msgr2=0x55d23a749030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73eea0 0x55d23a749030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 -- 172.21.15.45:0/770468013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23a66f400 con 0x55d23a749570 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcefffff640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 0x7fcef00a51c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf00106520 tx=0x7fceec009f20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fcf0531b640 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcef80a3ed0 tx=0x7fcef809b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570caa5ba4d7b27c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.072+0000 7fceeaffd640 1 -- 172.21.15.45:0/770468013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80cf070 con 0x55d23a749570 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fceeaffd640 1 -- 172.21.15.45:0/770468013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef809be90 con 0x55d23a749570 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fceeaffd640 1 -- 172.21.15.45:0/770468013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8020040 con 0x55d23a749570 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 -- 172.21.15.45:0/770468013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 msgr2=0x55d23a74b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcef80a3ed0 tx=0x7fcef809b390 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 -- 172.21.15.45:0/770468013 shutdown_connections 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a749570 0x55d23a74b960 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73eea0 0x55d23a749030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 --2- 172.21.15.45:0/770468013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73e580 0x55d23a73e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 -- 172.21.15.45:0/770468013 >> 172.21.15.45:0/770468013 conn(0x55d23a73a8a0 msgr2=0x55d23a73ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 -- 172.21.15.45:0/770468013 shutdown_connections 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.073+0000 7fcf09b69740 1 -- 172.21.15.45:0/770468013 wait complete. 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 -- start start 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 Processor -- start 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73e580 0x55d23a748f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a749570 0x55d23a746850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d23a74c300 con 0x55d23a73eea0 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d23a65f050 con 0x55d23a73e580 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d23a74c480 con 0x55d23a749570 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf09b69740 10 client.0 _ll_register_callbacks cb 0x55d23a3b6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52572/0 (socket says 172.21.15.45:52572) 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 -- 172.21.15.45:0/3634876794 learned_addr learned my addr 172.21.15.45:0/3634876794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf0531b640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a749570 0x55d23a746850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73e580 0x55d23a748f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 -- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a749570 msgr2=0x55d23a746850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a749570 0x55d23a746850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 -- 172.21.15.45:0/3634876794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73e580 msgr2=0x55d23a748f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73e580 0x55d23a748f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.074+0000 7fcefffff640 1 -- 172.21.15.45:0/3634876794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23a630dc0 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcefffff640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fceec012650 tx=0x7fceec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c158cf2655b9687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec022030 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fceec011870 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef809be90 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fceec02f050 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 10 client.18567 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d23a3b73f0 con 0x55d23a73eea0 2024-04-25T00:31:51.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef809be90 con 0x55d23a73eea0 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fceec00f040 con 0x55d23a73eea0 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 10 client.18567 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 10 client.18567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 20 client.18567 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a759640 0x55d23a75bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 4 client.18567 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.075+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d23a6b0bb0 con 0x55d23a759640 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.076+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a759640 0x55d23a75bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.076+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a759640 0x55d23a75bab0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6b3cf7f732b95aa server_cookie=8e6c70f42927e02a in_seq=0 out_seq=0 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.076+0000 7fcedffff640 10 client.18567 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcedffff640 1 -- 172.21.15.45:0/3634876794 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d23a6b0bb0 con 0x55d23a759640 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcedffff640 10 client.18567 handle_command_reply: tid=0 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 client.18567 shutdown 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 msgr2=0x55d23a746240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fceec012650 tx=0x7fceec004730 comp rx=0 tx=0).stop 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 shutdown_connections 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a759640 0x55d23a75bab0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a749570 0x55d23a746850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73eea0 0x55d23a746240 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcedffff640 0 client.18567 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3634876794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a73e580 0x55d23a748f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 >> 172.21.15.45:0/3634876794 conn(0x55d23a73a8a0 msgr2=0x55d23a739b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 shutdown_connections 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 1 -- 172.21.15.45:0/3634876794 wait complete. 2024-04-25T00:31:51.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.077+0000 7fcf09b69740 20 client.18567 trim_cache size 0 max 16384 2024-04-25T00:31:51.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 Processor -- start 2024-04-25T00:31:51.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 -- start start 2024-04-25T00:31:51.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a748a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a7516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001aec70 con 0x55d23a73e580 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b07b0 con 0x55d23a749570 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf09b69740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001af340 con 0x55d23a73eea0 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52578/0 (socket says 172.21.15.45:52578) 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3823341598 learned_addr learned my addr 172.21.15.45:0/3823341598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf0531b640 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a7516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3823341598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 msgr2=0x55d23a748a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a748a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3823341598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 msgr2=0x55d23a7516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a7516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3823341598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23a4817d0 con 0x55d23a73e580 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.091+0000 7fcf0531b640 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a7516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cdedf6a64c92ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fceea7fc640 1 -- 172.21.15.45:0/3823341598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4048070 con 0x55d23a73e580 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fceea7fc640 1 -- 172.21.15.45:0/3823341598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef4005520 con 0x55d23a73e580 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fceea7fc640 1 -- 172.21.15.45:0/3823341598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4035030 con 0x55d23a73e580 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 -- 172.21.15.45:0/3823341598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 msgr2=0x55d23a7453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4005a20 comp rx=0 tx=0).stop 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 -- 172.21.15.45:0/3823341598 shutdown_connections 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a7516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a748a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3823341598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a7453a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 -- 172.21.15.45:0/3823341598 >> 172.21.15.45:0/3823341598 conn(0x55d23a73a8a0 msgr2=0x55d23a7541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 -- 172.21.15.45:0/3823341598 shutdown_connections 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.092+0000 7fcf09b69740 1 -- 172.21.15.45:0/3823341598 wait complete. 2024-04-25T00:31:51.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 -- start start 2024-04-25T00:31:51.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 Processor -- start 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a74f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a74fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001b00e0 con 0x55d23a73e580 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001afa10 con 0x55d23a749570 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001acc70 con 0x55d23a73eea0 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf09b69740 10 client.0 _ll_register_callbacks cb 0x7fceec016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf04b1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf04b1a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52580/0 (socket says 172.21.15.45:52580) 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.093+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3617185758 learned_addr learned my addr 172.21.15.45:0/3617185758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcefffff640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a74f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 msgr2=0x55d23a74f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a74f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf0531b640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a74fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3617185758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 msgr2=0x55d23a74fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a74fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 -- 172.21.15.45:0/3617185758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23a7bef20 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf04b1a640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcef402a930 tx=0x7fcef4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f35a9695b40d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4048040 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7fcef40035f0 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4005520 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcef4035040 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 10 client.18573 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d23a7c0130 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef4005520 con 0x55d23a73e580 2024-04-25T00:31:51.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcef404b070 con 0x55d23a73e580 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 10 client.18573 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 10 client.18573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 20 client.18573 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a7c07e0 0x55d23a7c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 4 client.18573 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.094+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d23a6aaf00 con 0x55d23a7c07e0 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcefffff640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a7c07e0 0x55d23a7c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcefffff640 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a7c07e0 0x55d23a7c2bd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=286ef60f77fdbbc1 server_cookie=925c3fde035b71d8 in_seq=0 out_seq=0 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcedffff640 10 client.18573 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcedffff640 1 -- 172.21.15.45:0/3617185758 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fceec02c020 con 0x55d23a7c07e0 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcedffff640 10 client.18573 handle_command_reply: tid=0 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 client.18573 shutdown 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 msgr2=0x55d23a74f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcef402a930 tx=0x7fcef4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 shutdown_connections 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x55d23a7c07e0 0x55d23a7c2bd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d23a749570 0x55d23a74fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d23a73eea0 0x55d23a74f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcedffff640 0 client.18573 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 --2- 172.21.15.45:0/3617185758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d23a73e580 0x55d23a74f3c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.095+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 >> 172.21.15.45:0/3617185758 conn(0x55d23a73a8a0 msgr2=0x55d23a745220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 shutdown_connections 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf09b69740 1 -- 172.21.15.45:0/3617185758 wait complete. 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf09b69740 20 client.18573 trim_cache size 0 max 16384 2024-04-25T00:31:51.102 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 msgr2=0x7fcef00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 0x7fcef00a51c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf00106520 tx=0x7fceec009f20 comp rx=0 tx=0).stop 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 msgr2=0x7fcf0019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009fd0 tx=0x7fcef800b260 comp rx=0 tx=0).stop 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 shutdown_connections 2024-04-25T00:31:51.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fcef00a2d00 0x7fcef00a51c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf0019c990 0x7fcf0019ce40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf00106aa0 0x7fcf00198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 --2- 172.21.15.45:0/1249140369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001002e0 0x7fcf001980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 >> 172.21.15.45:0/1249140369 conn(0x7fcf000758d0 msgr2=0x7fcf00107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 shutdown_connections 2024-04-25T00:31:51.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.096+0000 7fcf06d7e640 1 -- 172.21.15.45:0/1249140369 wait complete. 2024-04-25T00:31:51.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 -- 172.21.15.45:0/3379970842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4100620 msgr2=0x7f9ff4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/3379970842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4100620 0x7f9ff4100aa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009930 tx=0x7f9fe402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 -- 172.21.15.45:0/3379970842 shutdown_connections 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/3379970842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 0x7f9ff410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/3379970842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4100620 0x7f9ff4100aa0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/3379970842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4106630 0x7f9ff4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 -- 172.21.15.45:0/3379970842 >> 172.21.15.45:0/3379970842 conn(0x7f9ff40776a0 msgr2=0x7f9ff4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 -- 172.21.15.45:0/3379970842 shutdown_connections 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.211+0000 7f9ffbba5640 1 -- 172.21.15.45:0/3379970842 wait complete. 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 Processor -- start 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 -- start start 2024-04-25T00:31:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4100620 0x7f9ff4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 0x7f9ff4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4110a90 con 0x7f9ff4106630 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4110910 con 0x7f9ff4100fe0 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffbba5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff4110c10 con 0x7f9ff4100620 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52602/0 (socket says 172.21.15.45:52602) 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 -- 172.21.15.45:0/637057136 learned_addr learned my addr 172.21.15.45:0/637057136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ff9941640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4100620 0x7f9ff4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ff9140640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 0x7f9ff4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 -- 172.21.15.45:0/637057136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4100620 msgr2=0x7f9ff4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4100620 0x7f9ff4198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 -- 172.21.15.45:0/637057136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 msgr2=0x7f9ff4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 0x7f9ff4198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 -- 172.21.15.45:0/637057136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff419d110 con 0x7f9ff4106630 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.212+0000 7f9ffa142640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009fd0 tx=0x7f9fec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.213+0000 7f9feaffd640 1 -- 172.21.15.45:0/637057136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec017070 con 0x7f9ff4106630 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.213+0000 7f9feaffd640 1 -- 172.21.15.45:0/637057136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec003e50 con 0x7f9ff4106630 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.213+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff419d400 con 0x7f9ff4106630 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.213+0000 7f9feaffd640 1 -- 172.21.15.45:0/637057136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0106c0 con 0x7f9ff4106630 2024-04-25T00:31:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.213+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff41a4c90 con 0x7f9ff4106630 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.215+0000 7f9feaffd640 1 -- 172.21.15.45:0/637057136 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9fec003970 con 0x7f9ff4106630 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 Processor -- start 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 -- start start 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc341d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557efc295670 con 0x557efc330480 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557efc23d970 con 0x557efc330da0 2024-04-25T00:31:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557efc291630 con 0x557efc33f980 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52610/0 (socket says 172.21.15.45:52610) 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.216+0000 7f9ff9140640 1 -- 172.21.15.45:0/1655472344 learned_addr learned my addr 172.21.15.45:0/1655472344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9feaffd640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 0x7f9fe00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9941640 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ffa142640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 0x7f9fe00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9feaffd640 1 -- 172.21.15.45:0/637057136 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f9fec0cf5d0 con 0x7f9ff4106630 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 -- 172.21.15.45:0/1655472344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 msgr2=0x557efc341d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc341d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 -- 172.21.15.45:0/1655472344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 msgr2=0x557efc33f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 -- 172.21.15.45:0/1655472344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efc261400 con 0x557efc330480 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9941640 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005c40 tx=0x7f9fe4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1f6cc855c0bc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9ffa142640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 0x7f9fe00a5240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9fec09b430 tx=0x7f9fec086040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9fdffff640 1 -- 172.21.15.45:0/1655472344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4040020 con 0x557efc330480 2024-04-25T00:31:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9fdffff640 1 -- 172.21.15.45:0/1655472344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4005520 con 0x557efc330480 2024-04-25T00:31:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.217+0000 7f9fdffff640 1 -- 172.21.15.45:0/1655472344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4035070 con 0x557efc330480 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1655472344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 msgr2=0x557efc330860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005c40 tx=0x7f9fe4009650 comp rx=0 tx=0).stop 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1655472344 shutdown_connections 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc341d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1655472344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc330860 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1655472344 >> 172.21.15.45:0/1655472344 conn(0x557efc32c8a0 msgr2=0x557efc32cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1655472344 shutdown_connections 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1655472344 wait complete. 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 -- start start 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.218+0000 7f9ffe98e740 1 Processor -- start 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc33cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557efc3426e0 con 0x557efc330480 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557efc251050 con 0x557efc330da0 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557efc342860 con 0x557efc33f980 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ffe98e740 10 client.0 _ll_register_callbacks cb 0x557efbfa8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52622/0 (socket says 172.21.15.45:52622) 2024-04-25T00:31:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 -- 172.21.15.45:0/1205658180 learned_addr learned my addr 172.21.15.45:0/1205658180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 -- 172.21.15.45:0/1205658180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 msgr2=0x557efc33cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc33cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 -- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 msgr2=0x557efc33c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9941640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 -- 172.21.15.45:0/1205658180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efc222dc0 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9ff9140640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009930 tx=0x7f9fe40052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134962ec20d77d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.219+0000 7f9fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4040070 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f9fe40315b0 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe4005520 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fe4057050 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 10 client.18585 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557efbfa93f0 con 0x557efc330480 2024-04-25T00:31:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4005520 con 0x557efc330480 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9fe4003960 con 0x557efc330480 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 10 client.18585 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 20 client.18585 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc34f730 0x557efc351b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 4 client.18585 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.220+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557efc2a2bb0 con 0x557efc34f730 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.221+0000 7f9ff9941640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc34f730 0x557efc351b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.221+0000 7f9ff9941640 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc34f730 0x557efc351b20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56715d82ffce159 server_cookie=a66ba1122e8b7d19 in_seq=0 out_seq=0 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.221+0000 7f9fdd7fa640 10 client.18585 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/1205658180 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557efc2a2bb0 con 0x557efc34f730 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9fdd7fa640 10 client.18585 handle_command_reply: tid=0 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 client.18585 shutdown 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 msgr2=0x557efc33f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009930 tx=0x7f9fe40052b0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 shutdown_connections 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc33cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc34f730 0x557efc351b20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/1205658180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc33f2a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 >> 172.21.15.45:0/1205658180 conn(0x557efc32c8a0 msgr2=0x557efc32bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9fdd7fa640 0 client.18585 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 shutdown_connections 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 1 -- 172.21.15.45:0/1205658180 wait complete. 2024-04-25T00:31:51.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.222+0000 7f9ffe98e740 20 client.18585 trim_cache size 0 max 16384 2024-04-25T00:31:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.235+0000 7f9ffe98e740 1 Processor -- start 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.235+0000 7f9ffe98e740 1 -- start start 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc357170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41ac910 con 0x557efc330480 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41ae450 con 0x557efc330da0 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41acfe0 con 0x557efc33f980 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52636/0 (socket says 172.21.15.45:52636) 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 -- 172.21.15.45:0/2322907588 learned_addr learned my addr 172.21.15.45:0/2322907588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9941640 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc357170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 -- 172.21.15.45:0/2322907588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 msgr2=0x557efc357170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc357170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 -- 172.21.15.45:0/2322907588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 msgr2=0x557efc33edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 -- 172.21.15.45:0/2322907588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efc0737d0 con 0x557efc330480 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9ff9140640 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40057c0 tx=0x7f9fe4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e256ec11e7f0ebdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9fdf7fe640 1 -- 172.21.15.45:0/2322907588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4040040 con 0x557efc330480 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9fdf7fe640 1 -- 172.21.15.45:0/2322907588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe404a030 con 0x557efc330480 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.236+0000 7f9fdf7fe640 1 -- 172.21.15.45:0/2322907588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4035040 con 0x557efc330480 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- 172.21.15.45:0/2322907588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 msgr2=0x557efc337c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40057c0 tx=0x7f9fe4001e70 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- 172.21.15.45:0/2322907588 shutdown_connections 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc33f980 0x557efc357170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc33edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/2322907588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc330480 0x557efc337c70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- 172.21.15.45:0/2322907588 >> 172.21.15.45:0/2322907588 conn(0x557efc32c8a0 msgr2=0x557efc331d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- 172.21.15.45:0/2322907588 shutdown_connections 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- 172.21.15.45:0/2322907588 wait complete. 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 -- start start 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.237+0000 7f9ffe98e740 1 Processor -- start 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc330480 0x557efc348060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc343420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41add80 con 0x557efc33f980 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41ad6b0 con 0x557efc330da0 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41aa910 con 0x557efc330480 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ffe98e740 10 client.0 _ll_register_callbacks cb 0x7f9fe4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52650/0 (socket says 172.21.15.45:52650) 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 -- 172.21.15.45:0/366769258 learned_addr learned my addr 172.21.15.45:0/366769258 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9941640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc343420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc330480 0x557efc348060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 -- 172.21.15.45:0/366769258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc330480 msgr2=0x557efc348060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc330480 0x557efc348060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 -- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 msgr2=0x557efc343420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc343420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 -- 172.21.15.45:0/366769258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efc3b49e0 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9ff9140640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005bd0 tx=0x7f9fe4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570d2a8b66950eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.238+0000 7f9fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4035020 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe404a030 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f9fe40052b0 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fe4040070 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 10 client.18591 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557efc3b5b80 con 0x557efc33f980 2024-04-25T00:31:51.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe404a030 con 0x557efc33f980 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9fe4051820 con 0x557efc33f980 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 10 client.18591 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 20 client.18591 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc3b6060 0x557efc3b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 4 client.18591 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557efc33ce60 con 0x557efc3b6060 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.239+0000 7f9ff9941640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc3b6060 0x557efc3b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9fdd7fa640 10 client.18591 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ff9941640 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc3b6060 0x557efc3b8480 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0a1cea13bb795f server_cookie=c5efdb0bf46fd0fe in_seq=0 out_seq=0 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9fdd7fa640 1 -- 172.21.15.45:0/366769258 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557efc33ce60 con 0x557efc3b6060 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9fdd7fa640 10 client.18591 handle_command_reply: tid=0 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 client.18591 shutdown 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 msgr2=0x557efc343b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005bd0 tx=0x7f9fe4009520 comp rx=0 tx=0).stop 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 shutdown_connections 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x557efc3b6060 0x557efc3b8480 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557efc33f980 0x557efc343b10 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557efc330da0 0x557efc343420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9fdd7fa640 0 client.18591 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 --2- 172.21.15.45:0/366769258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557efc330480 0x557efc348060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 >> 172.21.15.45:0/366769258 conn(0x557efc32c8a0 msgr2=0x557efc337af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 shutdown_connections 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 1 -- 172.21.15.45:0/366769258 wait complete. 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.240+0000 7f9ffe98e740 20 client.18591 trim_cache size 0 max 16384 2024-04-25T00:31:51.247 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:31:51.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 msgr2=0x7f9fe00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 0x7f9fe00a5240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9fec09b430 tx=0x7f9fec086040 comp rx=0 tx=0).stop 2024-04-25T00:31:51.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 msgr2=0x7f9ff419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009fd0 tx=0x7f9fec00b260 comp rx=0 tx=0).stop 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 shutdown_connections 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9fe00a2d80 0x7f9fe00a5240 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ff4106630 0x7f9ff419c9e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff4100fe0 0x7f9ff4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 --2- 172.21.15.45:0/637057136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ff4100620 0x7f9ff4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 >> 172.21.15.45:0/637057136 conn(0x7f9ff40776a0 msgr2=0x7f9ff406a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 shutdown_connections 2024-04-25T00:31:51.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:51.241+0000 7f9ffbba5640 1 -- 172.21.15.45:0/637057136 wait complete. 2024-04-25T00:31:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:51 smithi045 ceph-mon[29194]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:51.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:51 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1716465064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:51.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:51 smithi122 ceph-mon[32959]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:51.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:51 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1716465064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:51.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:51 smithi071 ceph-mon[32667]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:31:51.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:51 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1716465064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:31:52.253 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:31:52.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:31:52.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 -- 172.21.15.45:0/1322017598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c406a780 msgr2=0x7f43c410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1322017598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c406a780 0x7f43c410e5f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009cd0 tx=0x7f43bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 -- 172.21.15.45:0/1322017598 shutdown_connections 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1322017598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c406a780 0x7f43c410e5f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1322017598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c406a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1322017598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c4107730 0x7f43c4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 -- 172.21.15.45:0/1322017598 >> 172.21.15.45:0/1322017598 conn(0x7f43c40776a0 msgr2=0x7f43c4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 -- 172.21.15.45:0/1322017598 shutdown_connections 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.371+0000 7f43cb9db640 1 -- 172.21.15.45:0/1322017598 wait complete. 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 Processor -- start 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 -- start start 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c406a780 0x7f43c41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4111b50 con 0x7f43c4107730 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41119d0 con 0x7f43c4069dc0 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43cb9db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4111cd0 con 0x7f43c406a780 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43c9f78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43c9f78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52670/0 (socket says 172.21.15.45:52670) 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.372+0000 7f43c9f78640 1 -- 172.21.15.45:0/1086238909 learned_addr learned my addr 172.21.15.45:0/1086238909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9777640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c406a780 0x7f43c41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 -- 172.21.15.45:0/1086238909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c406a780 msgr2=0x7f43c41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c406a780 0x7f43c41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 -- 172.21.15.45:0/1086238909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 msgr2=0x7f43c4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 -- 172.21.15.45:0/1086238909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419cfa0 con 0x7f43c4107730 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9777640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43c9f78640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f43bc033040 tx=0x7f43bc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43ba7fc640 1 -- 172.21.15.45:0/1086238909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004040 con 0x7f43c4107730 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419d230 con 0x7f43c4107730 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43ba7fc640 1 -- 172.21.15.45:0/1086238909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43bc0046c0 con 0x7f43c4107730 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43ba7fc640 1 -- 172.21.15.45:0/1086238909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc046480 con 0x7f43c4107730 2024-04-25T00:31:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.373+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41a4b20 con 0x7f43c4107730 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.375+0000 7f43ba7fc640 1 -- 172.21.15.45:0/1086238909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f43bc0305a0 con 0x7f43c4107730 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 Processor -- start 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 -- start start 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ef980 0x55dee89edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dee89522c0 con 0x55dee89ee290 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dee88fa910 con 0x55dee89f9010 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ba7fc640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 0x7f43b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dee894e280 con 0x55dee89ef980 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52682/0 (socket says 172.21.15.45:52682) 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c9777640 1 -- 172.21.15.45:0/3479579515 learned_addr learned my addr 172.21.15.45:0/3479579515 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c9f78640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 0x7f43b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43ba7fc640 1 -- 172.21.15.45:0/1086238909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f43bc02e030 con 0x7f43c4107730 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c8f76640 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.376+0000 7f43c8f76640 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ef980 0x55dee89edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 -- 172.21.15.45:0/3479579515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ef980 msgr2=0x55dee89edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ef980 0x55dee89edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 -- 172.21.15.45:0/3479579515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 msgr2=0x55dee89fb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 -- 172.21.15.45:0/3479579515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee890bfb0 con 0x55dee89ee290 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9f78640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 0x7f43b00a51c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ee4a0 tx=0x7f43bc0dd040 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43c9777640 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cc20 tx=0x7f43c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7384920bbb101a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43af7fe640 1 -- 172.21.15.45:0/3479579515 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0005220 con 0x55dee89ee290 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43af7fe640 1 -- 172.21.15.45:0/3479579515 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c0016070 con 0x55dee89ee290 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.377+0000 7f43af7fe640 1 -- 172.21.15.45:0/3479579515 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0010720 con 0x55dee89ee290 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3479579515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 msgr2=0x55dee89ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f43c000cc20 tx=0x7f43c000e720 comp rx=0 tx=0).stop 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3479579515 shutdown_connections 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee89ee710 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3479579515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ef980 0x55dee89edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3479579515 >> 172.21.15.45:0/3479579515 conn(0x55dee89b6950 msgr2=0x55dee89ea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3479579515 shutdown_connections 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3479579515 wait complete. 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 -- start start 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.378+0000 7f43ce7bd740 1 Processor -- start 2024-04-25T00:31:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee89f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dee89fbea0 con 0x55dee89ef980 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55dee890de90 con 0x55dee89f9010 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dee89fc020 con 0x55dee89ee290 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 10 client.0 _ll_register_callbacks cb 0x55dee8949970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52684/0 (socket says 172.21.15.45:52684) 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 -- 172.21.15.45:0/575518085 learned_addr learned my addr 172.21.15.45:0/575518085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c8f76640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c8f76640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee89f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 -- 172.21.15.45:0/575518085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 msgr2=0x55dee89f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee89f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 -- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 msgr2=0x55dee89f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 -- 172.21.15.45:0/575518085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee872e7e0 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43c9777640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f43c0004920 tx=0x7f43c0004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb6ddeeb493a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43acff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c002a060 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f43c0004d00 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.379+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c0016070 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43c0031050 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 10 client.18603 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dee878ef70 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c0016070 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43c0010be0 con 0x55dee89ef980 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 10 client.18603 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 20 client.18603 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a09050 0x55dee8a0b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 4 client.18603 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee89265a0 con 0x55dee8a09050 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.380+0000 7f43c8f76640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a09050 0x55dee8a0b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.381+0000 7f43c8f76640 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a09050 0x55dee8a0b440 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4e7feb5a27e1db server_cookie=551bb398c6ea9320 in_seq=0 out_seq=0 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.381+0000 7f43acff9640 10 client.18603 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43acff9640 1 -- 172.21.15.45:0/575518085 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dee89265a0 con 0x55dee8a09050 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43acff9640 10 client.18603 handle_command_reply: tid=0 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 client.18603 shutdown 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 msgr2=0x55dee89f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f43c0004920 tx=0x7f43c0004950 comp rx=0 tx=0).stop 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 shutdown_connections 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a09050 0x55dee8a0b440 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee89f6450 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/575518085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee89f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 >> 172.21.15.45:0/575518085 conn(0x55dee89b6950 msgr2=0x55dee89f0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43acff9640 0 client.18603 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 shutdown_connections 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 1 -- 172.21.15.45:0/575518085 wait complete. 2024-04-25T00:31:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.382+0000 7f43ce7bd740 20 client.18603 trim_cache size 0 max 16384 2024-04-25T00:31:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 Processor -- start 2024-04-25T00:31:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 -- start start 2024-04-25T00:31:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89ef980 0x55dee8a10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89f9010 0x55dee8a10fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41aebb0 con 0x55dee89ee290 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41b06f0 con 0x55dee89ef980 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41af280 con 0x55dee89f9010 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43c8f76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43c8f76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52696/0 (socket says 172.21.15.45:52696) 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43c8f76640 1 -- 172.21.15.45:0/3675232917 learned_addr learned my addr 172.21.15.45:0/3675232917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.400+0000 7f43c9777640 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89ef980 0x55dee8a10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 -- 172.21.15.45:0/3675232917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89f9010 msgr2=0x55dee8a10fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89f9010 0x55dee8a10fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 -- 172.21.15.45:0/3675232917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89ef980 msgr2=0x55dee8a10a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89ef980 0x55dee8a10a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 -- 172.21.15.45:0/3675232917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee8626170 con 0x55dee89ee290 2024-04-25T00:31:52.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43c8f76640 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55dee8a0ba00 tx=0x7f43b400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b4dfb742c8300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43aeffd640 1 -- 172.21.15.45:0/3675232917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4029070 con 0x55dee89ee290 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43aeffd640 1 -- 172.21.15.45:0/3675232917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b4013c70 con 0x55dee89ee290 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.401+0000 7f43aeffd640 1 -- 172.21.15.45:0/3675232917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4012820 con 0x55dee89ee290 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3675232917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 msgr2=0x55dee8a04dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x55dee8a0ba00 tx=0x7f43b400cf00 comp rx=0 tx=0).stop 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3675232917 shutdown_connections 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89f9010 0x55dee8a10fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89ef980 0x55dee8a10a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/3675232917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ee290 0x55dee8a04dd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3675232917 >> 172.21.15.45:0/3675232917 conn(0x55dee89b6950 msgr2=0x55dee89e9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3675232917 shutdown_connections 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- 172.21.15.45:0/3675232917 wait complete. 2024-04-25T00:31:52.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 -- start start 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.402+0000 7f43ce7bd740 1 Processor -- start 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee8a012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41b0020 con 0x55dee89ef980 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41af950 con 0x55dee89f9010 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41acbb0 con 0x55dee89ee290 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 10 client.0 _ll_register_callbacks cb 0x7f43c0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52698/0 (socket says 172.21.15.45:52698) 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 -- 172.21.15.45:0/1381157054 learned_addr learned my addr 172.21.15.45:0/1381157054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee8a012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 -- 172.21.15.45:0/1381157054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 msgr2=0x55dee8a012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee8a012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 -- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 msgr2=0x55dee89fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 -- 172.21.15.45:0/1381157054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee89fea80 con 0x55dee89ef980 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43c9777640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000c00 tx=0x7f43c000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd5da33fa7c1a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43acff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c002a030 con 0x55dee89ef980 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f43c0005260 con 0x55dee89ef980 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b4013c70 con 0x55dee89ef980 2024-04-25T00:31:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43c000c7f0 con 0x55dee89ef980 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 10 client.18609 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.403+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dee8a6fde0 con 0x55dee89ef980 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b4013c70 con 0x55dee89ef980 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43c0010430 con 0x55dee89ef980 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 10 client.18609 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 20 client.18609 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a704f0 0x55dee8a72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 4 client.18609 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dee895f800 con 0x55dee8a704f0 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a704f0 0x55dee8a72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43c8f76640 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a704f0 0x55dee8a72910 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482cb2b63c601b3d server_cookie=543499f74202da8 in_seq=0 out_seq=0 2024-04-25T00:31:52.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.404+0000 7f43acff9640 10 client.18609 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43acff9640 1 -- 172.21.15.45:0/1381157054 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dee895f800 con 0x55dee8a704f0 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43acff9640 10 client.18609 handle_command_reply: tid=0 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 client.18609 shutdown 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 msgr2=0x55dee8a01820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000c00 tx=0x7f43c000eed0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 shutdown_connections 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dee89f9010 0x55dee89fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55dee8a704f0 0x55dee8a72910 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dee89ef980 0x55dee8a01820 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 --2- 172.21.15.45:0/1381157054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dee89ee290 0x55dee8a012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43acff9640 0 client.18609 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 >> 172.21.15.45:0/1381157054 conn(0x55dee89b6950 msgr2=0x55dee89efd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 shutdown_connections 2024-04-25T00:31:52.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 1 -- 172.21.15.45:0/1381157054 wait complete. 2024-04-25T00:31:52.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.406+0000 7f43ce7bd740 20 client.18609 trim_cache size 0 max 16384 2024-04-25T00:31:52.415 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:52.415 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T00:31:52.415 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "37f3ea5d-cc46-486c-ba68-f600c59a09e0", 2024-04-25T00:31:52.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T00:31:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:52.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 msgr2=0x7f43b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 0x7f43b00a51c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0ee4a0 tx=0x7f43bc0dd040 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 msgr2=0x7f43c419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f43bc033040 tx=0x7f43bc003970 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 shutdown_connections 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f43b00a2d00 0x7f43b00a51c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43c4107730 0x7f43c419c870 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c406a780 0x7f43c41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 --2- 172.21.15.45:0/1086238909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c4069dc0 0x7f43c4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 >> 172.21.15.45:0/1086238909 conn(0x7f43c40776a0 msgr2=0x7f43c40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 shutdown_connections 2024-04-25T00:31:52.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.407+0000 7f43cb9db640 1 -- 172.21.15.45:0/1086238909 wait complete. 2024-04-25T00:31:52.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:31:52.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 -- 172.21.15.45:0/2005135817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100620 msgr2=0x7feee4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 --2- 172.21.15.45:0/2005135817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100620 0x7feee4100aa0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feed4009930 tx=0x7feed402a420 comp rx=0 tx=0).stop 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 -- 172.21.15.45:0/2005135817 shutdown_connections 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 --2- 172.21.15.45:0/2005135817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100fe0 0x7feee410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 --2- 172.21.15.45:0/2005135817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100620 0x7feee4100aa0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 --2- 172.21.15.45:0/2005135817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 -- 172.21.15.45:0/2005135817 >> 172.21.15.45:0/2005135817 conn(0x7feee40776a0 msgr2=0x7feee4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 -- 172.21.15.45:0/2005135817 shutdown_connections 2024-04-25T00:31:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 -- 172.21.15.45:0/2005135817 wait complete. 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.522+0000 7feeebcf5640 1 Processor -- start 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 -- start start 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100620 0x7feee4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee41a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feee4110a40 con 0x7feee4100fe0 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feee41108c0 con 0x7feee4106630 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeebcf5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feee4110bc0 con 0x7feee4100620 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52724/0 (socket says 172.21.15.45:52724) 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 -- 172.21.15.45:0/1535464450 learned_addr learned my addr 172.21.15.45:0/1535464450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9a91640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100620 0x7feee4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeea292640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee41a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 -- 172.21.15.45:0/1535464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100620 msgr2=0x7feee4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100620 0x7feee4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 -- 172.21.15.45:0/1535464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 msgr2=0x7feee41a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee41a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 -- 172.21.15.45:0/1535464450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee41a4420 con 0x7feee4100fe0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feeea292640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee41a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.523+0000 7feee9290640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7feed4009230 tx=0x7feed4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.524+0000 7feedaffd640 1 -- 172.21.15.45:0/1535464450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed403f030 con 0x7feee4100fe0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.524+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee41a45f0 con 0x7feee4100fe0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.524+0000 7feedaffd640 1 -- 172.21.15.45:0/1535464450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed4003ad0 con 0x7feee4100fe0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.524+0000 7feedaffd640 1 -- 172.21.15.45:0/1535464450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4035070 con 0x7feee4100fe0 2024-04-25T00:31:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.524+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a4a60 con 0x7feee4100fe0 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.526+0000 7feedaffd640 1 -- 172.21.15.45:0/1535464450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feed4042030 con 0x7feee4100fe0 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.526+0000 7feeeea04740 1 Processor -- start 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.526+0000 7feeeea04740 1 -- start start 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5ae00f6b0 con 0x55f5ae0aa530 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f5adfb7880 con 0x55f5ae0b9a30 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeeea04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f5ae00b670 con 0x55f5ae0aae50 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeea292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeea292640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52738/0 (socket says 172.21.15.45:52738) 2024-04-25T00:31:52.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeea292640 1 -- 172.21.15.45:0/698793422 learned_addr learned my addr 172.21.15.45:0/698793422 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9290640 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feedaffd640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 0x7feed00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 -- 172.21.15.45:0/698793422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 msgr2=0x55f5ae0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 -- 172.21.15.45:0/698793422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 msgr2=0x55f5ae0aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 -- 172.21.15.45:0/698793422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5added7d0 con 0x55f5ae0aae50 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeea292640 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feeea292640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 0x7feed00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feedaffd640 1 -- 172.21.15.45:0/1535464450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7feed40efdb0 con 0x7feee4100fe0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.527+0000 7feee9a91640 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b94f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feee000bb90 tx=0x7feee000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f085ccd9fe84cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeea292640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 0x7feed00a51c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feedc013c60 tx=0x7feedc004ab0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feecffff640 1 -- 172.21.15.45:0/698793422 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0015070 con 0x55f5ae0aae50 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feecffff640 1 -- 172.21.15.45:0/698793422 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00035f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feecffff640 1 -- 172.21.15.45:0/698793422 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00116a0 con 0x55f5ae0aae50 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 -- 172.21.15.45:0/698793422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 msgr2=0x55f5ae0b94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b94f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feee000bb90 tx=0x7feee000f730 comp rx=0 tx=0).stop 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 -- 172.21.15.45:0/698793422 shutdown_connections 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b94f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 --2- 172.21.15.45:0/698793422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.528+0000 7feeeea04740 1 -- 172.21.15.45:0/698793422 >> 172.21.15.45:0/698793422 conn(0x55f5ae0a6950 msgr2=0x55f5ae0a6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- 172.21.15.45:0/698793422 shutdown_connections 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- 172.21.15.45:0/698793422 wait complete. 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- start start 2024-04-25T00:31:52.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 Processor -- start 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5ae0bc7f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f5adfcaf50 con 0x55f5ae0b9a30 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f5ae0bc970 con 0x55f5ae0aa530 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feeeea04740 10 client.0 _ll_register_callbacks cb 0x55f5ae006000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.529+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9a91640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52750/0 (socket says 172.21.15.45:52750) 2024-04-25T00:31:52.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 -- 172.21.15.45:0/2222134626 learned_addr learned my addr 172.21.15.45:0/2222134626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feeea292640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 -- 172.21.15.45:0/2222134626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 msgr2=0x55f5ae0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 -- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 msgr2=0x55f5ae0b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 -- 172.21.15.45:0/2222134626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5add233f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9a91640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feeea292640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feee9290640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7feed4009a50 tx=0x7feed40f34f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdee274a651a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4049070 con 0x55f5ae0aae50 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7feed40b8580 con 0x55f5ae0aae50 2024-04-25T00:31:52.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee00035f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feed40bb870 con 0x55f5ae0aae50 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feeeea04740 10 client.18621 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5adce30f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.530+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00035f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feed4003c80 con 0x55f5ae0aae50 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 10 client.18621 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 20 client.18621 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae0c9c20 0x55f5ae0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 4 client.18621 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5ae01cbf0 con 0x55f5ae0c9c20 2024-04-25T00:31:52.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feee9a91640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae0c9c20 0x55f5ae0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feecd7fa640 10 client.18621 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.531+0000 7feee9a91640 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae0c9c20 0x55f5ae0cc070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ae8f4e97ccfd45 server_cookie=603a8da7c76b53de in_seq=0 out_seq=0 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feecd7fa640 1 -- 172.21.15.45:0/2222134626 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5ae01cbf0 con 0x55f5ae0c9c20 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feecd7fa640 10 client.18621 handle_command_reply: tid=0 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 client.18621 shutdown 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 msgr2=0x55f5ae0b6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7feed4009a50 tx=0x7feed40f34f0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 shutdown_connections 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae0c9c20 0x55f5ae0cc070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0b6f10 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 --2- 172.21.15.45:0/2222134626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 >> 172.21.15.45:0/2222134626 conn(0x55f5ae0a6950 msgr2=0x55f5ae0ac7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feecd7fa640 0 client.18621 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 shutdown_connections 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.532+0000 7feeeea04740 1 -- 172.21.15.45:0/2222134626 wait complete. 2024-04-25T00:31:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.533+0000 7feeeea04740 20 client.18621 trim_cache size 0 max 16384 2024-04-25T00:31:52.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 Processor -- start 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 -- start start 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ac950 con 0x55f5ae0aae50 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ae490 con 0x55f5ae0b9a30 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feeeea04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ad020 con 0x55f5ae0aa530 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.550+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52764/0 (socket says 172.21.15.45:52764) 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 -- 172.21.15.45:0/1611111825 learned_addr learned my addr 172.21.15.45:0/1611111825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9a91640 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feeea292640 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 -- 172.21.15.45:0/1611111825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 msgr2=0x55f5ae0c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 -- 172.21.15.45:0/1611111825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 msgr2=0x55f5ae0d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 -- 172.21.15.45:0/1611111825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ae0a7b60 con 0x55f5ae0aae50 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feee9290640 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7feed40096c0 tx=0x7feed40ba420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f325fc50b938a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feecf7fe640 1 -- 172.21.15.45:0/1611111825 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4049070 con 0x55f5ae0aae50 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feecf7fe640 1 -- 172.21.15.45:0/1611111825 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed40c4b80 con 0x55f5ae0aae50 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.551+0000 7feecf7fe640 1 -- 172.21.15.45:0/1611111825 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40c46f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- 172.21.15.45:0/1611111825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 msgr2=0x55f5ae0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7feed40096c0 tx=0x7feed40ba420 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- 172.21.15.45:0/1611111825 shutdown_connections 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aae50 0x55f5ae0d1950 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 --2- 172.21.15.45:0/1611111825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- 172.21.15.45:0/1611111825 >> 172.21.15.45:0/1611111825 conn(0x55f5ae0a6950 msgr2=0x55f5ae0b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- 172.21.15.45:0/1611111825 shutdown_connections 2024-04-25T00:31:52.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- 172.21.15.45:0/1611111825 wait complete. 2024-04-25T00:31:52.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 -- start start 2024-04-25T00:31:52.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 Processor -- start 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.552+0000 7feeeea04740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0aae50 0x55f5ae0bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feee41addc0 con 0x55f5ae0aa530 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ad6f0 con 0x55f5ae0aae50 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feee41aa950 con 0x55f5ae0b9a30 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 10 client.0 _ll_register_callbacks cb 0x7feed4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52776/0 (socket says 172.21.15.45:52776) 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 -- 172.21.15.45:0/2335863194 learned_addr learned my addr 172.21.15.45:0/2335863194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:31:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9290640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0aae50 0x55f5ae0bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeea292640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 -- 172.21.15.45:0/2335863194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0b9a30 msgr2=0x55f5ae0bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 -- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0aae50 msgr2=0x55f5ae0bcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0aae50 0x55f5ae0bcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 -- 172.21.15.45:0/2335863194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ae0bda50 con 0x55f5ae0aa530 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feee9a91640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7feee000ba80 tx=0x7feee000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c33339f3a39a450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee002f060 con 0x55f5ae0aa530 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed40c4b80 con 0x55f5ae0aa530 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.553+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7feee0004230 con 0x55f5ae0aa530 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feee0036050 con 0x55f5ae0aa530 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 10 client.18627 fetch_fsmap learned FSMap version 28 2024-04-25T00:31:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5ae12fee0 con 0x55f5ae0aa530 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed40c4b80 con 0x55f5ae0aa530 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feee0029580 con 0x55f5ae0aa530 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 10 client.18627 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 20 client.18627 populate_metadata read hostname 'smithi045' 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae130380 0x55f5ae132770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 4 client.18627 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5ae016f40 con 0x55f5ae130380 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feee9290640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae130380 0x55f5ae132770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feee9290640 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae130380 0x55f5ae132770 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f74f8f533376273 server_cookie=b1cef1f0cba3063e in_seq=0 out_seq=0 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.554+0000 7feecd7fa640 10 client.18627 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feecd7fa640 1 -- 172.21.15.45:0/2335863194 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f5ae016f40 con 0x55f5ae130380 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feecd7fa640 10 client.18627 handle_command_reply: tid=0 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 client.18627 shutdown 2024-04-25T00:31:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 msgr2=0x55f5ae0c1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7feee000ba80 tx=0x7feee000b540 comp rx=0 tx=0).stop 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 shutdown_connections 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5ae0b9a30 0x55f5ae0bd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5ae0aae50 0x55f5ae0bcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55f5ae130380 0x55f5ae132770 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 --2- 172.21.15.45:0/2335863194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5ae0aa530 0x55f5ae0c1c60 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feecd7fa640 0 client.18627 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 >> 172.21.15.45:0/2335863194 conn(0x55f5ae0a6950 msgr2=0x55f5ae0b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 shutdown_connections 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 1 -- 172.21.15.45:0/2335863194 wait complete. 2024-04-25T00:31:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeeea04740 20 client.18627 trim_cache size 0 max 16384 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "37f3ea5d-cc46-486c-ba68-f600c59a09e0": { 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "37f3ea5d-cc46-486c-ba68-f600c59a09e0", 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:31:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 msgr2=0x7feed00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 0x7feed00a51c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feedc013c60 tx=0x7feedc004ab0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 msgr2=0x7feee41993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.555+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7feed4009230 tx=0x7feed4004270 comp rx=0 tx=0).stop 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 shutdown_connections 2024-04-25T00:31:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7feed00a2d00 0x7feed00a51c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee4106630 0x7feee41a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee4100fe0 0x7feee41993a0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 --2- 172.21.15.45:0/1535464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feee4100620 0x7feee4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 >> 172.21.15.45:0/1535464450 conn(0x7feee40776a0 msgr2=0x7feee41031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 shutdown_connections 2024-04-25T00:31:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:31:52.556+0000 7feeebcf5640 1 -- 172.21.15.45:0/1535464450 wait complete. 2024-04-25T00:31:52.568 INFO:tasks.cephfs.filesystem:scrub status for tag:37f3ea5d-cc46-486c-ba68-f600c59a09e0 - {'path': '/', 'tag': '37f3ea5d-cc46-486c-ba68-f600c59a09e0', 'options': 'recursive,force'} 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: scrub queued for path: / 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: scrub summary: active paths [/] 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:53 smithi045 ceph-mon[29194]: scrub summary: idle 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: scrub queued for path: / 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: scrub summary: active paths [/] 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:53 smithi122 ceph-mon[32959]: scrub summary: idle 2024-04-25T00:31:53.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:31:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: scrub queued for path: / 2024-04-25T00:31:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: scrub summary: active paths [/] 2024-04-25T00:31:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: scrub summary: idle+waiting paths [/] 2024-04-25T00:31:53.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:53 smithi071 ceph-mon[32667]: scrub summary: idle 2024-04-25T00:31:55.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:55 smithi045 ceph-mon[29194]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:31:55.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:55 smithi122 ceph-mon[32959]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:31:55.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:55 smithi071 ceph-mon[32667]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-25T00:31:56.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:56 smithi122 ceph-mon[32959]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:56.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:56 smithi071 ceph-mon[32667]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:56.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:56 smithi045 ceph-mon[29194]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T00:31:58.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:31:58 smithi071 ceph-mon[32667]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:58.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:31:58 smithi045 ceph-mon[29194]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:31:59.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:31:58 smithi122 ceph-mon[32959]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:32:00.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:00 smithi071 ceph-mon[32667]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:32:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:00 smithi045 ceph-mon[29194]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:32:01.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:00 smithi122 ceph-mon[32959]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-25T00:32:02.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:02 smithi071 ceph-mon[32667]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:32:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:02 smithi045 ceph-mon[29194]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:32:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:02 smithi122 ceph-mon[32959]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T00:32:04.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 -- 172.21.15.45:0/1137494944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 msgr2=0x7f0110106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 --2- 172.21.15.45:0/1137494944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f0110106a10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0100009930 tx=0x7f010002a420 comp rx=0 tx=0).stop 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 -- 172.21.15.45:0/1137494944 shutdown_connections 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 --2- 172.21.15.45:0/1137494944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 0x7f011010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 --2- 172.21.15.45:0/1137494944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 0x7f0110100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 --2- 172.21.15.45:0/1137494944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f0110106a10 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 -- 172.21.15.45:0/1137494944 >> 172.21.15.45:0/1137494944 conn(0x7f01100777c0 msgr2=0x7f0110077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 -- 172.21.15.45:0/1137494944 shutdown_connections 2024-04-25T00:32:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 -- 172.21.15.45:0/1137494944 wait complete. 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.668+0000 7f0115dd5640 1 Processor -- start 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 -- start start 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 0x7f0110198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 0x7f0110199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110a00 con 0x7f0110106630 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110880 con 0x7f0110100670 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f0115dd5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110b80 con 0x7f0110101030 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57712/0 (socket says 172.21.15.45:57712) 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 -- 172.21.15.45:0/4207384428 learned_addr learned my addr 172.21.15.45:0/4207384428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010effd640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 0x7f0110199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010f7fe640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 0x7f0110198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 -- 172.21.15.45:0/4207384428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 msgr2=0x7f0110199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 0x7f0110199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 -- 172.21.15.45:0/4207384428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 msgr2=0x7f0110198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 0x7f0110198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.669+0000 7f010ffff640 1 -- 172.21.15.45:0/4207384428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01101a43a0 con 0x7f0110106630 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f010ffff640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f010800eaa0 tx=0x7f010800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010800cdb0 con 0x7f0110106630 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0108005020 con 0x7f0110106630 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01101a4570 con 0x7f0110106630 2024-04-25T00:32:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01080116b0 con 0x7f0110106630 2024-04-25T00:32:04.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.670+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a4a40 con 0x7f0110106630 2024-04-25T00:32:04.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.671+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011004f500 con 0x7f0110106630 2024-04-25T00:32:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.672+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0108005560 con 0x7f0110106630 2024-04-25T00:32:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.673+0000 7f010cff9640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 0x7f00f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.673+0000 7f010f7fe640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 0x7f00f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.673+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f01080cf970 con 0x7f0110106630 2024-04-25T00:32:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.673+0000 7f010f7fe640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 0x7f00f80a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0100009900 tx=0x7f01000058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.677+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0108016030 con 0x7f0110106630 2024-04-25T00:32:04.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:04 smithi071 ceph-mon[32667]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:32:04.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.883+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0110061b80 con 0x7f00f80a2da0 2024-04-25T00:32:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.890+0000 7f010cff9640 1 -- 172.21.15.45:0/4207384428 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f0110061b80 con 0x7f00f80a2da0 2024-04-25T00:32:04.891 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T00:22:03", "first": "2024-04-25T00:23:00", "last": "2024-04-25T00:32:00", "last_pruned": "2024-04-25T00:32:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T00:32:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 msgr2=0x7f00f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 0x7f00f80a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0100009900 tx=0x7f01000058d0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 msgr2=0x7f01101a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f010800eaa0 tx=0x7f010800ef60 comp rx=0 tx=0).stop 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 shutdown_connections 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00f80a2da0 0x7f00f80a5260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0110106630 0x7f01101a3e60 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0110101030 0x7f0110199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 --2- 172.21.15.45:0/4207384428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0110100670 0x7f0110198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.892+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 >> 172.21.15.45:0/4207384428 conn(0x7f01100777c0 msgr2=0x7f01101031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.893+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 shutdown_connections 2024-04-25T00:32:04.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:04.893+0000 7f0115dd5640 1 -- 172.21.15.45:0/4207384428 wait complete. 2024-04-25T00:32:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:04 smithi045 ceph-mon[29194]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:32:04.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 -- 172.21.15.45:0/1321280317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 msgr2=0x7f7a6010d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 --2- 172.21.15.45:0/1321280317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a6010d530 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009cd0 tx=0x7f7a5802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 -- 172.21.15.45:0/1321280317 shutdown_connections 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 --2- 172.21.15.45:0/1321280317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a6010d530 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 --2- 172.21.15.45:0/1321280317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 --2- 172.21.15.45:0/1321280317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a60106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 -- 172.21.15.45:0/1321280317 >> 172.21.15.45:0/1321280317 conn(0x7f7a600769c0 msgr2=0x7f7a60076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 -- 172.21.15.45:0/1321280317 shutdown_connections 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.008+0000 7f7a68676640 1 -- 172.21.15.45:0/1321280317 wait complete. 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 Processor -- start 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 -- start start 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a6019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60110a80 con 0x7f7a60101020 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60110900 con 0x7f7a60106670 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a68676640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a60110c00 con 0x7f7a60100660 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57730/0 (socket says 172.21.15.45:57730) 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 -- 172.21.15.45:0/1795924990 learned_addr learned my addr 172.21.15.45:0/1795924990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a66412640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a66c13640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a6019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 -- 172.21.15.45:0/1795924990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 msgr2=0x7f7a60197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 -- 172.21.15.45:0/1795924990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 msgr2=0x7f7a6019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a6019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a65c11640 1 -- 172.21.15.45:0/1795924990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6019cee0 con 0x7f7a60101020 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a66412640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.009+0000 7f7a66c13640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a6019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a65c11640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d950 tx=0x7f7a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50017030 con 0x7f7a60101020 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a50004020 con 0x7f7a60101020 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6019d1d0 con 0x7f7a60101020 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500047a0 con 0x7f7a60101020 2024-04-25T00:32:05.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.010+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601a4a60 con 0x7f7a60101020 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.011+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6004f500 con 0x7f7a60101020 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.012+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7a500041d0 con 0x7f7a60101020 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.012+0000 7f7a577fe640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 0x7f7a4c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.013+0000 7f7a66412640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 0x7f7a4c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.013+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9475+0+0 (secure 0 0 0) 0x7f7a500cf8f0 con 0x7f7a60101020 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.013+0000 7f7a66412640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 0x7f7a4c0a52e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005fd0 tx=0x7f7a5c005850 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.016+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7a5009c170 con 0x7f7a60101020 2024-04-25T00:32:05.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:04 smithi122 ceph-mon[32959]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-25T00:32:05.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.222+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 --> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7a60061b80 con 0x7f7a4c0a2e20 2024-04-25T00:32:05.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.229+0000 7f7a577fe640 1 -- 172.21.15.45:0/1795924990 <== mgr.25163 v2:172.21.15.45:6800/1314659631 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7a60061b80 con 0x7f7a4c0a2e20 2024-04-25T00:32:05.230 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-25T00:32:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.231+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 msgr2=0x7f7a4c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 0x7f7a4c0a52e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005fd0 tx=0x7f7a5c005850 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 msgr2=0x7f7a60198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d950 tx=0x7f7a5000de10 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 shutdown_connections 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f7a4c0a2e20 0x7f7a4c0a52e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a60106670 0x7f7a6019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a60101020 0x7f7a60198430 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 --2- 172.21.15.45:0/1795924990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a60100660 0x7f7a60197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 >> 172.21.15.45:0/1795924990 conn(0x7f7a600769c0 msgr2=0x7f7a6006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 shutdown_connections 2024-04-25T00:32:05.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:05.232+0000 7f7a68676640 1 -- 172.21.15.45:0/1795924990 wait complete. 2024-04-25T00:32:05.243 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T00:32:05.255 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T00:32:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:05 smithi045 ceph-mon[29194]: from='client.18633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:05 smithi122 ceph-mon[32959]: from='client.18633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:06.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:05 smithi071 ceph-mon[32667]: from='client.18633 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:06 smithi045 ceph-mon[29194]: from='client.18639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:06 smithi045 ceph-mon[29194]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:32:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:06 smithi122 ceph-mon[32959]: from='client.18639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:06 smithi122 ceph-mon[32959]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:32:07.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:06 smithi071 ceph-mon[32667]: from='client.18639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:32:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:06 smithi071 ceph-mon[32667]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-25T00:32:08.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:08 smithi045 ceph-mon[29194]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:32:08.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:08 smithi045 ceph-mon[29194]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:32:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:08 smithi122 ceph-mon[32959]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:32:09.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:08 smithi122 ceph-mon[32959]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:32:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:08 smithi071 ceph-mon[32667]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T00:32:09.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:08 smithi071 ceph-mon[32667]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:32:10.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:10 smithi045 ceph-mon[29194]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:11.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:10 smithi122 ceph-mon[32959]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:11.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:10 smithi071 ceph-mon[32667]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:11.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:11 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:11.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:11 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:11.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:11 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:13 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:13 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: pgmap v317: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-04-25T00:32:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:32:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:13 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:32:14.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:14.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:14 smithi122 ceph-mon[32959]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.370 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:32:14.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:14.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:14.372 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:14 smithi071 ceph-mon[32667]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:14.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi071 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:32:14.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi045 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: Unable to set osd_memory_target on smithi122 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:14.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:14 smithi045 ceph-mon[29194]: from='mgr.25163 ' entity='mgr.x' 2024-04-25T00:32:15.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:15 smithi071 ceph-mon[32667]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-25T00:32:15.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:15 smithi045 ceph-mon[29194]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-25T00:32:15.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:15 smithi122 ceph-mon[32959]: pgmap v319: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-25T00:32:17.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:17 smithi071 ceph-mon[32667]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T00:32:17.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:17 smithi045 ceph-mon[29194]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T00:32:17.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:17 smithi122 ceph-mon[32959]: pgmap v320: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T00:32:19.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:19 smithi071 ceph-mon[32667]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 206 B/s wr, 13 op/s 2024-04-25T00:32:19.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:19 smithi045 ceph-mon[29194]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 206 B/s wr, 13 op/s 2024-04-25T00:32:19.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:19 smithi122 ceph-mon[32959]: pgmap v321: 129 pgs: 129 active+clean; 2.2 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 206 B/s wr, 13 op/s 2024-04-25T00:32:21.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:21 smithi071 ceph-mon[32667]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:21.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:21 smithi045 ceph-mon[29194]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:21.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:21 smithi122 ceph-mon[32959]: pgmap v322: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:22.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:32:22.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 -- 172.21.15.45:0/8516587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 msgr2=0x7fa71010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 --2- 172.21.15.45:0/8516587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71010dc10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa700009900 tx=0x7fa70002a410 comp rx=0 tx=0).stop 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 -- 172.21.15.45:0/8516587 shutdown_connections 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 --2- 172.21.15.45:0/8516587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 0x7fa710110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 --2- 172.21.15.45:0/8516587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71010dc10 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 --2- 172.21.15.45:0/8516587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 0x7fa71006a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.688+0000 7fa7172b3640 1 -- 172.21.15.45:0/8516587 >> 172.21.15.45:0/8516587 conn(0x7fa7100776a0 msgr2=0x7fa710077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- 172.21.15.45:0/8516587 shutdown_connections 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- 172.21.15.45:0/8516587 wait complete. 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 Processor -- start 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- start start 2024-04-25T00:32:22.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 0x7fa71019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 0x7fa71019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa710113d10 con 0x7fa71006a6e0 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa710113b90 con 0x7fa710069dc0 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.689+0000 7fa7172b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa710113e90 con 0x7fa71010e150 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52710/0 (socket says 172.21.15.45:52710) 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 -- 172.21.15.45:0/1901945782 learned_addr learned my addr 172.21.15.45:0/1901945782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa715850640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 0x7fa71019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71504f640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 0x7fa71019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 -- 172.21.15.45:0/1901945782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 msgr2=0x7fa71019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 0x7fa71019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 -- 172.21.15.45:0/1901945782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 msgr2=0x7fa71019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 0x7fa71019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 -- 172.21.15.45:0/1901945782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71019f2c0 con 0x7fa71006a6e0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa71484e640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa7000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa7067fc640 1 -- 172.21.15.45:0/1901945782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70003f030 con 0x7fa71006a6e0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa7067fc640 1 -- 172.21.15.45:0/1901945782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa700003ce0 con 0x7fa71006a6e0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71019f550 con 0x7fa71006a6e0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa7067fc640 1 -- 172.21.15.45:0/1901945782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700035070 con 0x7fa71006a6e0 2024-04-25T00:32:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.690+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101a6e40 con 0x7fa71006a6e0 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.693+0000 7fa7067fc640 1 -- 172.21.15.45:0/1901945782 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa700042030 con 0x7fa71006a6e0 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.693+0000 7fa719fc1740 1 Processor -- start 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.693+0000 7fa719fc1740 1 -- start start 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8f50 0x55977a6d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55977a62d6b0 con 0x55977a6c8630 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55977a5d5880 con 0x55977a6c8f50 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa719fc1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55977a629670 con 0x55977a6d3620 2024-04-25T00:32:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa7067fc640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 0x7fa6fc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52714/0 (socket says 172.21.15.45:52714) 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 -- 172.21.15.45:0/1721958215 learned_addr learned my addr 172.21.15.45:0/1721958215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 0x7fa6fc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa71504f640 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8f50 0x55977a6d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa7067fc640 1 -- 172.21.15.45:0/1901945782 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fa7000efd80 con 0x7fa71006a6e0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa71484e640 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 -- 172.21.15.45:0/1721958215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 msgr2=0x55977a6d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 -- 172.21.15.45:0/1721958215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8f50 msgr2=0x55977a6d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8f50 0x55977a6d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.694+0000 7fa715850640 1 -- 172.21.15.45:0/1721958215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977a40b7d0 con 0x55977a6c8630 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa715850640 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa708012ce0 tx=0x7fa708010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf353e95246dbb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa715850640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 0x7fa6fc0a51c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa70800c000 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa6fb7fe640 1 -- 172.21.15.45:0/1721958215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708023070 con 0x55977a6c8630 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa6fb7fe640 1 -- 172.21.15.45:0/1721958215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708009ca0 con 0x55977a6c8630 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa6fb7fe640 1 -- 172.21.15.45:0/1721958215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708005020 con 0x55977a6c8630 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa719fc1740 1 -- 172.21.15.45:0/1721958215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 msgr2=0x55977a6c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.695+0000 7fa719fc1740 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa708012ce0 tx=0x7fa708010740 comp rx=0 tx=0).stop 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- 172.21.15.45:0/1721958215 shutdown_connections 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8f50 0x55977a6d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- 172.21.15.45:0/1721958215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6c8a10 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- 172.21.15.45:0/1721958215 >> 172.21.15.45:0/1721958215 conn(0x55977a6c4950 msgr2=0x55977a6c4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- 172.21.15.45:0/1721958215 shutdown_connections 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- 172.21.15.45:0/1721958215 wait complete. 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- start start 2024-04-25T00:32:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 Processor -- start 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55977a6d63b0 con 0x55977a6d3620 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55977a5e8f50 con 0x55977a6c8630 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55977a6d6530 con 0x55977a6c8f50 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.696+0000 7fa719fc1740 10 client.0 _ll_register_callbacks cb 0x55977a624000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52720/0 (socket says 172.21.15.45:52720) 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 -- 172.21.15.45:0/2100384660 learned_addr learned my addr 172.21.15.45:0/2100384660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa71484e640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa71504f640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 -- 172.21.15.45:0/2100384660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 msgr2=0x55977a6d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 -- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 msgr2=0x55977a6d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 -- 172.21.15.45:0/2100384660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977a3413f0 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa715850640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa708012090 tx=0x7fa708005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e2caa621444dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa6f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70802b030 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fa70801b070 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa708009ca0 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa708038050 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa719fc1740 10 client.18651 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.697+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55977a3010f0 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708009ca0 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa70800f040 con 0x55977a6d3620 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 10 client.18651 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:22.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 10 client.18651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 20 client.18651 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a6e37e0 0x55977a6e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 4 client.18651 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977a63abf0 con 0x55977a6e37e0 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.698+0000 7fa71504f640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a6e37e0 0x55977a6e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa71504f640 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a6e37e0 0x55977a6e5bd0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ad31df2c9842b2 server_cookie=df7910180c3ea64d in_seq=0 out_seq=0 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa6f8ff9640 10 client.18651 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2100384660 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55977a63abf0 con 0x55977a6e37e0 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa6f8ff9640 10 client.18651 handle_command_reply: tid=0 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 client.18651 shutdown 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 msgr2=0x55977a6d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa708012090 tx=0x7fa708005c40 comp rx=0 tx=0).stop 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 shutdown_connections 2024-04-25T00:32:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a6e37e0 0x55977a6e5bd0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6d3620 0x55977a6d1090 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa6f8ff9640 0 client.18651 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2100384660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.699+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 >> 172.21.15.45:0/2100384660 conn(0x55977a6c4950 msgr2=0x55977a6ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.700+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 shutdown_connections 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.700+0000 7fa719fc1740 1 -- 172.21.15.45:0/2100384660 wait complete. 2024-04-25T00:32:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.700+0000 7fa719fc1740 20 client.18651 trim_cache size 0 max 16384 2024-04-25T00:32:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.717+0000 7fa719fc1740 1 Processor -- start 2024-04-25T00:32:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.717+0000 7fa719fc1740 1 -- start start 2024-04-25T00:32:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b0e90 con 0x55977a6c8f50 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b29d0 con 0x55977a6c8630 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa719fc1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b1560 con 0x55977a6d3620 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52734/0 (socket says 172.21.15.45:52734) 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 -- 172.21.15.45:0/2521856217 learned_addr learned my addr 172.21.15.45:0/2521856217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71504f640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa715850640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 -- 172.21.15.45:0/2521856217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 msgr2=0x55977a6cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 -- 172.21.15.45:0/2521856217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 msgr2=0x55977a6c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 -- 172.21.15.45:0/2521856217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977a6b6090 con 0x55977a6c8f50 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa715850640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71504f640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa71484e640 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa7000ed110 tx=0x7fa7000bba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f77ff88c09dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa6faffd640 1 -- 172.21.15.45:0/2521856217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700049040 con 0x55977a6c8f50 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa6faffd640 1 -- 172.21.15.45:0/2521856217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7000d8020 con 0x55977a6c8f50 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.718+0000 7fa6faffd640 1 -- 172.21.15.45:0/2521856217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000c4bb0 con 0x55977a6c8f50 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- 172.21.15.45:0/2521856217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 msgr2=0x55977a6df270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa7000ed110 tx=0x7fa7000bba80 comp rx=0 tx=0).stop 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- 172.21.15.45:0/2521856217 shutdown_connections 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6d3620 0x55977a6cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8f50 0x55977a6df270 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2521856217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6c8630 0x55977a6c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- 172.21.15.45:0/2521856217 >> 172.21.15.45:0/2521856217 conn(0x55977a6c4950 msgr2=0x55977a6d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- 172.21.15.45:0/2521856217 shutdown_connections 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- 172.21.15.45:0/2521856217 wait complete. 2024-04-25T00:32:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 -- start start 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.719+0000 7fa719fc1740 1 Processor -- start 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6d3620 0x55977a748a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b2300 con 0x55977a6c8630 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101b1c30 con 0x55977a6d3620 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101aee90 con 0x55977a6c8f50 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa719fc1740 10 client.0 _ll_register_callbacks cb 0x7fa70c00d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52752/0 (socket says 172.21.15.45:52752) 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 -- 172.21.15.45:0/2285296455 learned_addr learned my addr 172.21.15.45:0/2285296455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa715850640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6d3620 0x55977a748a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71484e640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 -- 172.21.15.45:0/2285296455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 msgr2=0x55977a6cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 -- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6d3620 msgr2=0x55977a748a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6d3620 0x55977a748a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 -- 172.21.15.45:0/2285296455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977a748fb0 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa71504f640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x55977a6e6170 tx=0x7fa70c00dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e3fd43c8815aca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.720+0000 7fa6f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c01d3b0 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fa70c01d550 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7000d8020 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa70c02c050 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 10 client.18657 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55977a74a1a0 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7000d8020 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa70c0103d0 con 0x55977a6c8630 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 10 client.18657 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:22.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 10 client.18657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 20 client.18657 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a74a8a0 0x55977a74ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 4 client.18657 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.721+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55977a634f40 con 0x55977a74a8a0 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa71484e640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a74a8a0 0x55977a74ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa71484e640 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a74a8a0 0x55977a74ccc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2aee1c3042d0ee server_cookie=68b3e2c2d10ac75d in_seq=0 out_seq=0 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa6f8ff9640 10 client.18657 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa6f8ff9640 1 -- 172.21.15.45:0/2285296455 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55977a634f40 con 0x55977a74a8a0 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa6f8ff9640 10 client.18657 handle_command_reply: tid=0 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 client.18657 shutdown 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 msgr2=0x55977a6cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x55977a6e6170 tx=0x7fa70c00dd60 comp rx=0 tx=0).stop 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 shutdown_connections 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977a6d3620 0x55977a748a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55977a6c8f50 0x55977a6cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x55977a74a8a0 0x55977a74ccc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 --2- 172.21.15.45:0/2285296455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55977a6c8630 0x55977a6cd920 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa6f8ff9640 0 client.18657 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 >> 172.21.15.45:0/2285296455 conn(0x55977a6c4950 msgr2=0x55977a6d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.722+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 shutdown_connections 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa719fc1740 1 -- 172.21.15.45:0/2285296455 wait complete. 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa719fc1740 20 client.18657 trim_cache size 0 max 16384 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T00:32:22.730 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 msgr2=0x7fa6fc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 0x7fa6fc0a51c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa70800c000 comp rx=0 tx=0).stop 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 msgr2=0x7fa71019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa700006010 tx=0x7fa7000042f0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 shutdown_connections 2024-04-25T00:32:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fa6fc0a2d00 0x7fa6fc0a51c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71010e150 0x7fa71019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa71006a6e0 0x7fa71019a810 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 --2- 172.21.15.45:0/1901945782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa710069dc0 0x7fa71019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 >> 172.21.15.45:0/1901945782 conn(0x7fa7100776a0 msgr2=0x7fa71010f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 shutdown_connections 2024-04-25T00:32:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.723+0000 7fa7172b3640 1 -- 172.21.15.45:0/1901945782 wait complete. 2024-04-25T00:32:22.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:32:22.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 -- 172.21.15.45:0/671230345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74101050 msgr2=0x7ffb7410d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 --2- 172.21.15.45:0/671230345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74101050 0x7ffb7410d510 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009cd0 tx=0x7ffb6c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 -- 172.21.15.45:0/671230345 shutdown_connections 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 --2- 172.21.15.45:0/671230345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74101050 0x7ffb7410d510 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 --2- 172.21.15.45:0/671230345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74100690 0x7ffb74100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 --2- 172.21.15.45:0/671230345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74106650 0x7ffb74106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 -- 172.21.15.45:0/671230345 >> 172.21.15.45:0/671230345 conn(0x7ffb740777a0 msgr2=0x7ffb74077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 -- 172.21.15.45:0/671230345 shutdown_connections 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.837+0000 7ffb79637640 1 -- 172.21.15.45:0/671230345 wait complete. 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 Processor -- start 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 -- start start 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74101050 0x7ffb740fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74106650 0x7ffb740fe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74110a60 con 0x7ffb74100690 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffb741108e0 con 0x7ffb74106650 2024-04-25T00:32:22.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb79637640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74110be0 con 0x7ffb74101050 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52772/0 (socket says 172.21.15.45:52772) 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 -- 172.21.15.45:0/2744353376 learned_addr learned my addr 172.21.15.45:0/2744353376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb737fe640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74106650 0x7ffb740fe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb6bfff640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74101050 0x7ffb740fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 -- 172.21.15.45:0/2744353376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74101050 msgr2=0x7ffb740fdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74101050 0x7ffb740fdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 -- 172.21.15.45:0/2744353376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74106650 msgr2=0x7ffb740fe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74106650 0x7ffb740fe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 -- 172.21.15.45:0/2744353376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb741a6530 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.838+0000 7ffb72ffd640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9f0 tx=0x7ffb6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.839+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400cd80 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.839+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb64005500 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.839+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb741a6820 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.839+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64010690 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.839+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb741a6d20 con 0x7ffb74100690 2024-04-25T00:32:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.840+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb74101d80 con 0x7ffb74100690 2024-04-25T00:32:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.841+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb640046c0 con 0x7ffb74100690 2024-04-25T00:32:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.841+0000 7ffb70ff9640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 0x7ffb600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.841+0000 7ffb6bfff640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 0x7ffb600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.841+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7ffb640cfd10 con 0x7ffb74100690 2024-04-25T00:32:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.842+0000 7ffb6bfff640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 0x7ffb600a51c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb740ff0e0 tx=0x7ffb5c005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:22.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:22.846+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb6409c570 con 0x7ffb74100690 2024-04-25T00:32:23.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.070+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb74109bc0 con 0x7ffb74100690 2024-04-25T00:32:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.071+0000 7ffb70ff9640 1 -- 172.21.15.45:0/2744353376 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7ffb6409be70 con 0x7ffb74100690 2024-04-25T00:32:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:32:23.072 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:32:23.072 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 msgr2=0x7ffb600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 0x7ffb600a51c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb740ff0e0 tx=0x7ffb5c005f90 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 msgr2=0x7ffb740fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9f0 tx=0x7ffb6400edf0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 shutdown_connections 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ffb600a2d00 0x7ffb600a51c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb74106650 0x7ffb740fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb74101050 0x7ffb740fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 --2- 172.21.15.45:0/2744353376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74100690 0x7ffb740fd650 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 >> 172.21.15.45:0/2744353376 conn(0x7ffb740777a0 msgr2=0x7ffb7406a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 shutdown_connections 2024-04-25T00:32:23.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.073+0000 7ffb79637640 1 -- 172.21.15.45:0/2744353376 wait complete. 2024-04-25T00:32:23.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 -- 172.21.15.45:0/1739755538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 msgr2=0x7f17bc101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 --2- 172.21.15.45:0/1739755538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc101b90 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009930 tx=0x7f17a402a420 comp rx=0 tx=0).stop 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 -- 172.21.15.45:0/1739755538 shutdown_connections 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 --2- 172.21.15.45:0/1739755538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 0x7f17bc10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 --2- 172.21.15.45:0/1739755538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc101b90 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 --2- 172.21.15.45:0/1739755538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 0x7f17bc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 -- 172.21.15.45:0/1739755538 >> 172.21.15.45:0/1739755538 conn(0x7f17bc0fd180 msgr2=0x7f17bc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 -- 172.21.15.45:0/1739755538 shutdown_connections 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.190+0000 7f17c0f63640 1 -- 172.21.15.45:0/1739755538 wait complete. 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 Processor -- start 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 -- start start 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 0x7f17bc073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 0x7f17bc0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111b30 con 0x7f17bc101710 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1119b0 con 0x7f17bc107720 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17c0f63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111cb0 con 0x7f17bc1020d0 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52788/0 (socket says 172.21.15.45:52788) 2024-04-25T00:32:23.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 -- 172.21.15.45:0/2834405108 learned_addr learned my addr 172.21.15.45:0/2834405108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17b3fff640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 0x7f17bc073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17bad9d640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 0x7f17bc0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 -- 172.21.15.45:0/2834405108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 msgr2=0x7f17bc073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 0x7f17bc073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 -- 172.21.15.45:0/2834405108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 msgr2=0x7f17bc0744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 0x7f17bc0744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.191+0000 7f17ba59c640 1 -- 172.21.15.45:0/2834405108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc1a6470 con 0x7f17bc101710 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c920 tx=0x7f17ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17b37fe640 1 -- 172.21.15.45:0/2834405108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac007c70 con 0x7f17bc101710 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17b37fe640 1 -- 172.21.15.45:0/2834405108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac007e10 con 0x7f17bc101710 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc1a6760 con 0x7f17bc101710 2024-04-25T00:32:23.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17b37fe640 1 -- 172.21.15.45:0/2834405108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac018890 con 0x7f17bc101710 2024-04-25T00:32:23.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.192+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a6c80 con 0x7f17bc101710 2024-04-25T00:32:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.194+0000 7f17b37fe640 1 -- 172.21.15.45:0/2834405108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f17ac018a30 con 0x7f17bc101710 2024-04-25T00:32:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.194+0000 7f17c3c77740 1 Processor -- start 2024-04-25T00:32:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.194+0000 7f17c3c77740 1 -- start start 2024-04-25T00:32:23.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e256960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56102e1ae670 con 0x56102e249ea0 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56102e156970 con 0x56102e249580 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17c3c77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56102e1aa630 con 0x56102e254570 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52798/0 (socket says 172.21.15.45:52798) 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:56702/0 (socket says 172.21.15.45:56702) 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 -- 172.21.15.45:0/96276909 learned_addr learned my addr 172.21.15.45:0/96276909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17b37fe640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 0x7f17a80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17b3fff640 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e256960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17bad9d640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 0x7f17a80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17b37fe640 1 -- 172.21.15.45:0/2834405108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f17ac0cf920 con 0x7f17bc101710 2024-04-25T00:32:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 -- 172.21.15.45:0/96276909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 msgr2=0x56102e256960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e256960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 -- 172.21.15.45:0/96276909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 msgr2=0x56102e249960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 -- 172.21.15.45:0/96276909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102e17a400 con 0x56102e249ea0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17bad9d640 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.195+0000 7f17ba59c640 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f17ac09b720 tx=0x7f17ac00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f5b7bff610b274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17bad9d640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 0x7f17a80a51f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17b4005fd0 tx=0x7f17b4004ab0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17b0ff9640 1 -- 172.21.15.45:0/96276909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac09ba20 con 0x56102e249ea0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17b0ff9640 1 -- 172.21.15.45:0/96276909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0d1e50 con 0x56102e249ea0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17b0ff9640 1 -- 172.21.15.45:0/96276909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac098090 con 0x56102e249ea0 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 -- 172.21.15.45:0/96276909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 msgr2=0x56102e254030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f17ac09b720 tx=0x7f17ac00ff30 comp rx=0 tx=0).stop 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 -- 172.21.15.45:0/96276909 shutdown_connections 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e256960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e254030 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 --2- 172.21.15.45:0/96276909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e249960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 -- 172.21.15.45:0/96276909 >> 172.21.15.45:0/96276909 conn(0x56102e2458a0 msgr2=0x56102e245c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.196+0000 7f17c3c77740 1 -- 172.21.15.45:0/96276909 shutdown_connections 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 -- 172.21.15.45:0/96276909 wait complete. 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 -- start start 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 Processor -- start 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e253f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e251840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56102e257300 con 0x56102e249ea0 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56102e16a050 con 0x56102e249580 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56102e257480 con 0x56102e254570 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17c3c77740 10 client.0 _ll_register_callbacks cb 0x56102dec1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52812/0 (socket says 172.21.15.45:52812) 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17b3fff640 1 -- 172.21.15.45:0/647993691 learned_addr learned my addr 172.21.15.45:0/647993691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17ba59c640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e253f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17bad9d640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e251840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17b3fff640 1 -- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 msgr2=0x56102e251840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.197+0000 7f17b3fff640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e251840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17b3fff640 1 -- 172.21.15.45:0/647993691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 msgr2=0x56102e253f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17b3fff640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e253f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17b3fff640 1 -- 172.21.15.45:0/647993691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102e13bdc0 con 0x56102e249ea0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17ba59c640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e253f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17b3fff640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000c00 tx=0x7f17a40319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccca57d6bda7157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4035040 con 0x56102e249ea0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f17a4004300 con 0x56102e249ea0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac0d1e50 con 0x56102e249ea0 2024-04-25T00:32:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17a404f050 con 0x56102e249ea0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 10 client.18675 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56102dec23f0 con 0x56102e249ea0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0d1e50 con 0x56102e249ea0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f17a4004c90 con 0x56102e249ea0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 10 client.18675 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 10 client.18675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 20 client.18675 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2645c0 0x56102e2669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 4 client.18675 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.198+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56102e1bbbb0 con 0x56102e2645c0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.199+0000 7f17ba59c640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2645c0 0x56102e2669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.199+0000 7f17ba59c640 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2645c0 0x56102e2669d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6b922b58e4ab82d server_cookie=32fbf8e12d31cba2 in_seq=0 out_seq=0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.199+0000 7f17a1ffb640 10 client.18675 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17a1ffb640 1 -- 172.21.15.45:0/647993691 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56102e1bbbb0 con 0x56102e2645c0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17a1ffb640 10 client.18675 handle_command_reply: tid=0 2024-04-25T00:32:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 client.18675 shutdown 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 msgr2=0x56102e251230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f17a4000c00 tx=0x7f17a40319b0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 shutdown_connections 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2645c0 0x56102e2669d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e254570 0x56102e251840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e251230 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17a1ffb640 0 client.18675 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 --2- 172.21.15.45:0/647993691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249580 0x56102e253f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 >> 172.21.15.45:0/647993691 conn(0x56102e2458a0 msgr2=0x56102e24b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 shutdown_connections 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 1 -- 172.21.15.45:0/647993691 wait complete. 2024-04-25T00:32:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.200+0000 7f17c3c77740 20 client.18675 trim_cache size 0 max 16384 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 Processor -- start 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 -- start start 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e2601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249ea0 0x56102e26c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b0d10 con 0x56102e254570 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b2850 con 0x56102e249ea0 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.213+0000 7f17c3c77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b13e0 con 0x56102e249580 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52818/0 (socket says 172.21.15.45:52818) 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 -- 172.21.15.45:0/4059554114 learned_addr learned my addr 172.21.15.45:0/4059554114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17b3fff640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249ea0 0x56102e26c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17ba59c640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e2601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 -- 172.21.15.45:0/4059554114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 msgr2=0x56102e2601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e2601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 -- 172.21.15.45:0/4059554114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249ea0 msgr2=0x56102e26c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249ea0 0x56102e26c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 -- 172.21.15.45:0/4059554114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102df8c7d0 con 0x56102e254570 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17ba59c640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e2601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17bad9d640 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f17b4012990 tx=0x7f17b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfeed95d545d559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17a3fff640 1 -- 172.21.15.45:0/4059554114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b401a070 con 0x56102e254570 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17a3fff640 1 -- 172.21.15.45:0/4059554114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b4025070 con 0x56102e254570 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.214+0000 7f17a3fff640 1 -- 172.21.15.45:0/4059554114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4020400 con 0x56102e254570 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- 172.21.15.45:0/4059554114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 msgr2=0x56102e26c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f17b4012990 tx=0x7f17b4009840 comp rx=0 tx=0).stop 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- 172.21.15.45:0/4059554114 shutdown_connections 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e254570 0x56102e26c7e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e249ea0 0x56102e26c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- 172.21.15.45:0/4059554114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e2601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- 172.21.15.45:0/4059554114 >> 172.21.15.45:0/4059554114 conn(0x56102e2458a0 msgr2=0x56102e244b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- 172.21.15.45:0/4059554114 shutdown_connections 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- 172.21.15.45:0/4059554114 wait complete. 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- start start 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 Processor -- start 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e25cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e254570 0x56102e25a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b2180 con 0x56102e249ea0 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1b1ab0 con 0x56102e254570 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1aed10 con 0x56102e249580 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.215+0000 7f17c3c77740 10 client.0 _ll_register_callbacks cb 0x7f17a4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52826/0 (socket says 172.21.15.45:52826) 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 -- 172.21.15.45:0/2646344251 learned_addr learned my addr 172.21.15.45:0/2646344251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e25cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17bad9d640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e254570 0x56102e25a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 -- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 msgr2=0x56102e25cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e25cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 -- 172.21.15.45:0/2646344251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e254570 msgr2=0x56102e25a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e254570 0x56102e25a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 -- 172.21.15.45:0/2646344251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56102e2c9b20 con 0x56102e249ea0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17b3fff640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f17a4006010 tx=0x7f17a4003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f05df9d5938a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4035070 con 0x56102e249ea0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17b4025070 con 0x56102e249ea0 2024-04-25T00:32:23.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f17a4004100 con 0x56102e249ea0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17a4061050 con 0x56102e249ea0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17c3c77740 10 client.18681 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.216+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56102e2cac30 con 0x56102e249ea0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b4025070 con 0x56102e249ea0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f17a4004770 con 0x56102e249ea0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 10 client.18681 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 10 client.18681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 20 client.18681 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2cb370 0x56102e2cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 4 client.18681 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 --> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56102e1b5f00 con 0x56102e2cb370 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2cb370 0x56102e2cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17a1ffb640 10 client.18681 ms_handle_connect on v2:172.21.15.122:6834/1025393130 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.217+0000 7f17ba59c640 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2cb370 0x56102e2cd790 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2da2a207e9eda06c server_cookie=12e1f35214d3a0b9 in_seq=0 out_seq=0 2024-04-25T00:32:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17a1ffb640 1 -- 172.21.15.45:0/2646344251 <== mds.4 v2:172.21.15.122:6834/1025393130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56102e1b5f00 con 0x56102e2cb370 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17a1ffb640 10 client.18681 handle_command_reply: tid=0 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 client.18681 shutdown 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 msgr2=0x56102e259fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f17a4006010 tx=0x7f17a4003a30 comp rx=0 tx=0).stop 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 shutdown_connections 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:6834/1025393130,v1:172.21.15.122:6835/1025393130] conn(0x56102e2cb370 0x56102e2cd790 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56102e254570 0x56102e25a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56102e249ea0 0x56102e259fa0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 --2- 172.21.15.45:0/2646344251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56102e249580 0x56102e25cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17a1ffb640 0 client.18681 ms_handle_reset on v2:172.21.15.122:6834/1025393130 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 >> 172.21.15.45:0/2646344251 conn(0x56102e2458a0 msgr2=0x56102e25f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 shutdown_connections 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 1 -- 172.21.15.45:0/2646344251 wait complete. 2024-04-25T00:32:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c3c77740 20 client.18681 trim_cache size 0 max 16384 2024-04-25T00:32:23.226 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:32:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 msgr2=0x7f17a80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 0x7f17a80a51f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17b4005fd0 tx=0x7f17b4004ab0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 msgr2=0x7f17bc073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c920 tx=0x7f17ac00cde0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 shutdown_connections 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f17a80a2d30 0x7f17a80a51f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17bc107720 0x7f17bc0744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17bc1020d0 0x7f17bc073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 --2- 172.21.15.45:0/2834405108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17bc101710 0x7f17bc073a40 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 >> 172.21.15.45:0/2834405108 conn(0x7f17bc0fd180 msgr2=0x7f17bc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 shutdown_connections 2024-04-25T00:32:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.218+0000 7f17c0f63640 1 -- 172.21.15.45:0/2834405108 wait complete. 2024-04-25T00:32:23.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 -- 172.21.15.45:0/1646060692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 msgr2=0x7fb418100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 --2- 172.21.15.45:0/1646060692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418100a90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c02a420 comp rx=0 tx=0).stop 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 -- 172.21.15.45:0/1646060692 shutdown_connections 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 --2- 172.21.15.45:0/1646060692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb418100fd0 0x7fb41810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 --2- 172.21.15.45:0/1646060692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418100a90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.332+0000 7fb41d058640 1 --2- 172.21.15.45:0/1646060692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4181066b0 0x7fb418106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.333+0000 7fb41d058640 1 -- 172.21.15.45:0/1646060692 >> 172.21.15.45:0/1646060692 conn(0x7fb418077740 msgr2=0x7fb418077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.333+0000 7fb41d058640 1 -- 172.21.15.45:0/1646060692 shutdown_connections 2024-04-25T00:32:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.333+0000 7fb41d058640 1 -- 172.21.15.45:0/1646060692 wait complete. 2024-04-25T00:32:23.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.333+0000 7fb41d058640 1 Processor -- start 2024-04-25T00:32:23.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.333+0000 7fb41d058640 1 -- start start 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb418100fd0 0x7fb418199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 0x7fb4181a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb418110a30 con 0x7fb418100610 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181108b0 con 0x7fb418100fd0 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb418110bb0 con 0x7fb4181066b0 2024-04-25T00:32:23.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52834/0 (socket says 172.21.15.45:52834) 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 -- 172.21.15.45:0/3217134518 learned_addr learned my addr 172.21.15.45:0/3217134518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41659c640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb418100fd0 0x7fb418199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41759e640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 0x7fb4181a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 -- 172.21.15.45:0/3217134518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 msgr2=0x7fb4181a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 0x7fb4181a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 -- 172.21.15.45:0/3217134518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb418100fd0 msgr2=0x7fb418199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb418100fd0 0x7fb418199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 -- 172.21.15.45:0/3217134518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181a43b0 con 0x7fb418100610 2024-04-25T00:32:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41759e640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 0x7fb4181a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9c0 tx=0x7fb40800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb3fffff640 1 -- 172.21.15.45:0/3217134518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800cd80 con 0x7fb418100610 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181a4580 con 0x7fb418100610 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb3fffff640 1 -- 172.21.15.45:0/3217134518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4080035f0 con 0x7fb418100610 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb3fffff640 1 -- 172.21.15.45:0/3217134518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408010690 con 0x7fb418100610 2024-04-25T00:32:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.334+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a4a90 con 0x7fb418100610 2024-04-25T00:32:23.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.337+0000 7fb3fffff640 1 -- 172.21.15.45:0/3217134518 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4080037a0 con 0x7fb418100610 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.337+0000 7fb41fd65740 1 Processor -- start 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 -- start start 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5b9570 0x55933d5bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55933d513670 con 0x55933d5aeea0 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55933d4bb970 con 0x55933d5ae580 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41fd65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55933d50f630 con 0x55933d5b9570 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52838/0 (socket says 172.21.15.45:52838) 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 -- 172.21.15.45:0/3945108931 learned_addr learned my addr 172.21.15.45:0/3945108931 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb3fffff640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 0x7fb4000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41759e640 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41659c640 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5b9570 0x55933d5bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 -- 172.21.15.45:0/3945108931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5b9570 msgr2=0x55933d5bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5b9570 0x55933d5bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 -- 172.21.15.45:0/3945108931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 msgr2=0x55933d5ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb3fffff640 1 -- 172.21.15.45:0/3217134518 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fb4080cfbf0 con 0x7fb418100610 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 -- 172.21.15.45:0/3945108931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933d4df400 con 0x55933d5aeea0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb41759e640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 0x7fb4000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.338+0000 7fb416d9d640 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb4080cce60 tx=0x7fb408003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88d695335c9fdba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb3fd7fa640 1 -- 172.21.15.45:0/3945108931 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408052070 con 0x55933d5aeea0 2024-04-25T00:32:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb3fd7fa640 1 -- 172.21.15.45:0/3945108931 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40809c3f0 con 0x55933d5aeea0 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb3fd7fa640 1 -- 172.21.15.45:0/3945108931 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080d3050 con 0x55933d5aeea0 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 -- 172.21.15.45:0/3945108931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 msgr2=0x55933d5b9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41759e640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 0x7fb4000a51c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb410005fd0 tx=0x7fb410004a40 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb4080cce60 tx=0x7fb408003e30 comp rx=0 tx=0).stop 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 -- 172.21.15.45:0/3945108931 shutdown_connections 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5b9570 0x55933d5bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b9030 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 --2- 172.21.15.45:0/3945108931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 -- 172.21.15.45:0/3945108931 >> 172.21.15.45:0/3945108931 conn(0x55933d5aa8a0 msgr2=0x55933d5aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 -- 172.21.15.45:0/3945108931 shutdown_connections 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.339+0000 7fb41fd65740 1 -- 172.21.15.45:0/3945108931 wait complete. 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 -- start start 2024-04-25T00:32:23.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 Processor -- start 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5b9570 0x55933d5b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55933d5bc300 con 0x55933d5aeea0 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55933d4cf050 con 0x55933d5b9570 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55933d5bc480 con 0x55933d5ae580 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41fd65740 10 client.0 _ll_register_callbacks cb 0x55933d226be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52848/0 (socket says 172.21.15.45:52848) 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 -- 172.21.15.45:0/1269376658 learned_addr learned my addr 172.21.15.45:0/1269376658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb416d9d640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41759e640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5b9570 0x55933d5b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 -- 172.21.15.45:0/1269376658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 msgr2=0x55933d5b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 -- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5b9570 msgr2=0x55933d5b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5b9570 0x55933d5b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.340+0000 7fb41659c640 1 -- 172.21.15.45:0/1269376658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933d4a0dc0 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41659c640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0e2da59d78f8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c035040 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fb40c004240 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40809c3f0 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb40c004440 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 10 client.18693 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55933d2273f0 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40809c3f0 con 0x55933d5aeea0 2024-04-25T00:32:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb40c03c540 con 0x55933d5aeea0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 10 client.18693 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 10 client.18693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 20 client.18693 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d5c95d0 0x55933d5cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 4 client.18693 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.341+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933d520bb0 con 0x55933d5c95d0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.342+0000 7fb416d9d640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d5c95d0 0x55933d5cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.342+0000 7fb3faff5640 10 client.18693 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.342+0000 7fb416d9d640 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d5c95d0 0x55933d5cba30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d13d1440bd0eac8 server_cookie=928e42f129630e89 in_seq=0 out_seq=0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb3faff5640 1 -- 172.21.15.45:0/1269376658 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55933d520bb0 con 0x55933d5c95d0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb3faff5640 10 client.18693 handle_command_reply: tid=0 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 client.18693 shutdown 2024-04-25T00:32:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 msgr2=0x55933d5b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c0317e0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 shutdown_connections 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d5c95d0 0x55933d5cba30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5b9570 0x55933d5b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5aeea0 0x55933d5b61e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb3faff5640 0 client.18693 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 --2- 172.21.15.45:0/1269376658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 >> 172.21.15.45:0/1269376658 conn(0x55933d5aa8a0 msgr2=0x55933d5b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 shutdown_connections 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 1 -- 172.21.15.45:0/1269376658 wait complete. 2024-04-25T00:32:23.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.343+0000 7fb41fd65740 20 client.18693 trim_cache size 0 max 16384 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.356+0000 7fb41fd65740 1 Processor -- start 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 -- start start 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5aeea0 0x55933d5aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181aed10 con 0x55933d5b9570 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181b0850 con 0x55933d5ae580 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41fd65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181af3e0 con 0x55933d5aeea0 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52860/0 (socket says 172.21.15.45:52860) 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 -- 172.21.15.45:0/2359851069 learned_addr learned my addr 172.21.15.45:0/2359851069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb416d9d640 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41659c640 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5aeea0 0x55933d5aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 -- 172.21.15.45:0/2359851069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5aeea0 msgr2=0x55933d5aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5aeea0 0x55933d5aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 -- 172.21.15.45:0/2359851069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 msgr2=0x55933d5c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 -- 172.21.15.45:0/2359851069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933d2f17d0 con 0x55933d5b9570 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb41759e640 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb41001d6e0 tx=0x7fb410014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf7364f0040d142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb3fcff9640 1 -- 172.21.15.45:0/2359851069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41001d110 con 0x55933d5b9570 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb3fcff9640 1 -- 172.21.15.45:0/2359851069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41001d2b0 con 0x55933d5b9570 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.357+0000 7fb3fcff9640 1 -- 172.21.15.45:0/2359851069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41001c8c0 con 0x55933d5b9570 2024-04-25T00:32:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- 172.21.15.45:0/2359851069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 msgr2=0x55933d5d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb41001d6e0 tx=0x7fb410014000 comp rx=0 tx=0).stop 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- 172.21.15.45:0/2359851069 shutdown_connections 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d5d1300 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5aeea0 0x55933d5aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 --2- 172.21.15.45:0/2359851069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5ae580 0x55933d5c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- 172.21.15.45:0/2359851069 >> 172.21.15.45:0/2359851069 conn(0x55933d5aa8a0 msgr2=0x55933d5a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- 172.21.15.45:0/2359851069 shutdown_connections 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- 172.21.15.45:0/2359851069 wait complete. 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 -- start start 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.358+0000 7fb41fd65740 1 Processor -- start 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 0x55933d5bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181b0180 con 0x55933d5b9570 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181afab0 con 0x55933d5aeea0 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181acd10 con 0x55933d5ae580 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 10 client.0 _ll_register_callbacks cb 0x7fb40c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 0x55933d5bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52874/0 (socket says 172.21.15.45:52874) 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 -- 172.21.15.45:0/4273403899 learned_addr learned my addr 172.21.15.45:0/4273403899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb416d9d640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 -- 172.21.15.45:0/4273403899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 msgr2=0x55933d5bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 -- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 msgr2=0x55933d5bf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 0x55933d5bf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 -- 172.21.15.45:0/4273403899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933d5c0270 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41659c640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 0x55933d5bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41759e640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb410009910 tx=0x7fb41001d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351b0ad1b05b507a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb3faff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410035060 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fb410023040 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.359+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41001d2b0 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb41003c050 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 10 client.18699 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55933d630190 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41001d2b0 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb41001ca40 con 0x55933d5b9570 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 10 client.18699 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 10 client.18699 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-25T00:32:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 20 client.18699 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d6308b0 0x55933d632cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 4 client.18699 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 --> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55933d51af00 con 0x55933d6308b0 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.360+0000 7fb416d9d640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d6308b0 0x55933d632cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb3faff5640 10 client.18699 ms_handle_connect on v2:172.21.15.71:6836/2635562743 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb416d9d640 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d6308b0 0x55933d632cd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25daff0401cd6d49 server_cookie=b1786c3bec34c0a in_seq=0 out_seq=0 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb3faff5640 1 -- 172.21.15.45:0/4273403899 <== mds.3 v2:172.21.15.71:6836/2635562743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55933d51af00 con 0x55933d6308b0 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb3faff5640 10 client.18699 handle_command_reply: tid=0 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 client.18699 shutdown 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 msgr2=0x55933d62eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb410009910 tx=0x7fb41001d460 comp rx=0 tx=0).stop 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 shutdown_connections 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:6836/2635562743,v1:172.21.15.71:6837/2635562743] conn(0x55933d6308b0 0x55933d632cd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55933d5b9570 0x55933d62eb80 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55933d5aeea0 0x55933d5bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb3faff5640 0 client.18699 ms_handle_reset on v2:172.21.15.71:6836/2635562743 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 --2- 172.21.15.45:0/4273403899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55933d5ae580 0x55933d5bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 >> 172.21.15.45:0/4273403899 conn(0x55933d5aa8a0 msgr2=0x55933d5a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 shutdown_connections 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 1 -- 172.21.15.45:0/4273403899 wait complete. 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.361+0000 7fb41fd65740 20 client.18699 trim_cache size 0 max 16384 2024-04-25T00:32:23.369 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 msgr2=0x7fb4000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 0x7fb4000a51c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb410005fd0 tx=0x7fb410004a40 comp rx=0 tx=0).stop 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 msgr2=0x7fb418198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb40800e9c0 tx=0x7fb40800ee80 comp rx=0 tx=0).stop 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 shutdown_connections 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fb4000a2d00 0x7fb4000a51c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4181066b0 0x7fb4181a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb418100fd0 0x7fb418199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 --2- 172.21.15.45:0/3217134518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb418100610 0x7fb418198df0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 >> 172.21.15.45:0/3217134518 conn(0x7fb418077740 msgr2=0x7fb4180ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 shutdown_connections 2024-04-25T00:32:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.362+0000 7fb41d058640 1 -- 172.21.15.45:0/3217134518 wait complete. 2024-04-25T00:32:23.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:23 smithi071 ceph-mon[32667]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:23.371 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:23 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2744353376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:32:23.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:23 smithi045 ceph-mon[29194]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:23.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:23 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2744353376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:32:23.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:32:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 -- 172.21.15.45:0/136776745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec06a780 msgr2=0x7f20ec10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/136776745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec06a780 0x7f20ec10e5f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009930 tx=0x7f20e402a420 comp rx=0 tx=0).stop 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 -- 172.21.15.45:0/136776745 shutdown_connections 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/136776745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec06a780 0x7f20ec10e5f0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/136776745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec069dc0 0x7f20ec06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/136776745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec107730 0x7f20ec107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.478+0000 7f20f3ea5640 1 -- 172.21.15.45:0/136776745 >> 172.21.15.45:0/136776745 conn(0x7f20ec0776a0 msgr2=0x7f20ec077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- 172.21.15.45:0/136776745 shutdown_connections 2024-04-25T00:32:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- 172.21.15.45:0/136776745 wait complete. 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 Processor -- start 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- start start 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec069dc0 0x7f20ec197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec06a780 0x7f20ec1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec111b40 con 0x7f20ec107730 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1119c0 con 0x7f20ec069dc0 2024-04-25T00:32:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.479+0000 7f20f3ea5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec111cc0 con 0x7f20ec06a780 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52900/0 (socket says 172.21.15.45:52900) 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 -- 172.21.15.45:0/4216159494 learned_addr learned my addr 172.21.15.45:0/4216159494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f1c41640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec069dc0 0x7f20ec197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f1440640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec06a780 0x7f20ec1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 -- 172.21.15.45:0/4216159494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec06a780 msgr2=0x7f20ec1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec06a780 0x7f20ec1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 -- 172.21.15.45:0/4216159494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec069dc0 msgr2=0x7f20ec197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec069dc0 0x7f20ec197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 -- 172.21.15.45:0/4216159494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec19cf90 con 0x7f20ec107730 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f2442640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005ab0 tx=0x7f20e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20e2ffd640 1 -- 172.21.15.45:0/4216159494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e403f030 con 0x7f20ec107730 2024-04-25T00:32:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20e2ffd640 1 -- 172.21.15.45:0/4216159494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4003ce0 con 0x7f20ec107730 2024-04-25T00:32:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec19d220 con 0x7f20ec107730 2024-04-25T00:32:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20e2ffd640 1 -- 172.21.15.45:0/4216159494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4035070 con 0x7f20ec107730 2024-04-25T00:32:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.480+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec1a4b10 con 0x7f20ec107730 2024-04-25T00:32:23.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.482+0000 7f20e2ffd640 1 -- 172.21.15.45:0/4216159494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20e4042030 con 0x7f20ec107730 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 Processor -- start 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 -- start start 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebaad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebaba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ebeba12670 con 0x55ebebaadea0 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ebeb9ba970 con 0x55ebebab8570 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ebeba0e630 con 0x55ebebaad580 2024-04-25T00:32:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f1c41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20e2ffd640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 0x7f20d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f1c41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52910/0 (socket says 172.21.15.45:52910) 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f1c41640 1 -- 172.21.15.45:0/2978580777 learned_addr learned my addr 172.21.15.45:0/2978580777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.483+0000 7f20f2442640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 0x7f20d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1440640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebaad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1440640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebaba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 -- 172.21.15.45:0/2978580777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 msgr2=0x55ebebaad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebaad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20e2ffd640 1 -- 172.21.15.45:0/4216159494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f20e40efc00 con 0x7f20ec107730 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 -- 172.21.15.45:0/2978580777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 msgr2=0x55ebebaba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebaba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 -- 172.21.15.45:0/2978580777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebeb9de400 con 0x55ebebaadea0 2024-04-25T00:32:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1440640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebaba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f1c41640 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f20e8010370 tx=0x7f20e8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e63cffcdc70b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20f2442640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 0x7f20d80a51c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ed040 tx=0x7f20e40b12a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20d7fff640 1 -- 172.21.15.45:0/2978580777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8016070 con 0x55ebebaadea0 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20d7fff640 1 -- 172.21.15.45:0/2978580777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8004d20 con 0x55ebebaadea0 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.484+0000 7f20d7fff640 1 -- 172.21.15.45:0/2978580777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80054e0 con 0x55ebebaadea0 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- 172.21.15.45:0/2978580777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 msgr2=0x55ebebab8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f20e8010370 tx=0x7f20e8010830 comp rx=0 tx=0).stop 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- 172.21.15.45:0/2978580777 shutdown_connections 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebaba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaadea0 0x55ebebab8030 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/2978580777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebaad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- 172.21.15.45:0/2978580777 >> 172.21.15.45:0/2978580777 conn(0x55ebebaa98a0 msgr2=0x55ebebaa9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- 172.21.15.45:0/2978580777 shutdown_connections 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- 172.21.15.45:0/2978580777 wait complete. 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 -- start start 2024-04-25T00:32:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.485+0000 7f20f6bb8740 1 Processor -- start 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebab56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ebebabb360 con 0x55ebebaad580 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ebeb9ce050 con 0x55ebebab8570 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ebebabb4e0 con 0x55ebebaadea0 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f6bb8740 10 client.0 _ll_register_callbacks cb 0x55ebeb725be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52924/0 (socket says 172.21.15.45:52924) 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 -- 172.21.15.45:0/3054154361 learned_addr learned my addr 172.21.15.45:0/3054154361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1c41640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 -- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 msgr2=0x55ebebab51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 -- 172.21.15.45:0/3054154361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 msgr2=0x55ebebab56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebab56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 -- 172.21.15.45:0/3054154361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebeb99fdc0 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20f1440640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00bab0 tx=0x7f20dc00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e004cc2d371123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc005220 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.486+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f20dc0053c0 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e8004d20 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20dc024050 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 10 client.18711 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ebeb7263f0 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8004d20 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20dc012710 con 0x55ebebaad580 2024-04-25T00:32:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 10 client.18711 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 10 client.18711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 20 client.18711 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebac84f0 0x55ebebaca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 4 client.18711 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebeba1fbb0 con 0x55ebebac84f0 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.487+0000 7f20f1c41640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebac84f0 0x55ebebaca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.488+0000 7f20f1c41640 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebac84f0 0x55ebebaca940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd9c40e4c463c779 server_cookie=ae67035ba0818b08 in_seq=0 out_seq=0 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.488+0000 7f20d57fa640 10 client.18711 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.488+0000 7f20d57fa640 1 -- 172.21.15.45:0/3054154361 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebeba1fbb0 con 0x55ebebac84f0 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.488+0000 7f20d57fa640 10 client.18711 handle_command_reply: tid=0 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.488+0000 7f20f6bb8740 1 client.18711 shutdown 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 msgr2=0x55ebebab7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00bab0 tx=0x7f20dc00bf70 comp rx=0 tx=0).stop 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 shutdown_connections 2024-04-25T00:32:23.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebac84f0 0x55ebebaca940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebab8570 0x55ebebab56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3054154361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebaad580 0x55ebebab7eb0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20d57fa640 0 client.18711 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 >> 172.21.15.45:0/3054154361 conn(0x55ebebaa98a0 msgr2=0x55ebebaa8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 shutdown_connections 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3054154361 wait complete. 2024-04-25T00:32:23.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.489+0000 7f20f6bb8740 20 client.18711 trim_cache size 0 max 16384 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 Processor -- start 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 -- start start 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaad580 0x55ebebab0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1aece0 con 0x55ebebab8570 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1b0820 con 0x55ebebaad580 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.502+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1af3b0 con 0x55ebebaadea0 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52926/0 (socket says 172.21.15.45:52926) 2024-04-25T00:32:23.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 -- 172.21.15.45:0/3623950748 learned_addr learned my addr 172.21.15.45:0/3623950748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1c41640 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaad580 0x55ebebab0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 -- 172.21.15.45:0/3623950748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 msgr2=0x55ebebab79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 -- 172.21.15.45:0/3623950748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaad580 msgr2=0x55ebebab0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaad580 0x55ebebab0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 -- 172.21.15.45:0/3623950748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebeb7f07d0 con 0x55ebebab8570 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20f1440640 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0040f0 tx=0x7f20dc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5bb510542c884f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20d77fe640 1 -- 172.21.15.45:0/3623950748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc02f020 con 0x55ebebab8570 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20d77fe640 1 -- 172.21.15.45:0/3623950748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc018030 con 0x55ebebab8570 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.503+0000 7f20d77fe640 1 -- 172.21.15.45:0/3623950748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0050e0 con 0x55ebebab8570 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3623950748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 msgr2=0x55ebebad0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0040f0 tx=0x7f20dc00b7c0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3623950748 shutdown_connections 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebad0210 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaadea0 0x55ebebab79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/3623950748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaad580 0x55ebebab0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3623950748 >> 172.21.15.45:0/3623950748 conn(0x55ebebaa98a0 msgr2=0x55ebebaaf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3623950748 shutdown_connections 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- 172.21.15.45:0/3623950748 wait complete. 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 -- start start 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.504+0000 7f20f6bb8740 1 Processor -- start 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebab1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 0x55ebebab2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1b0150 con 0x55ebebab8570 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1afa80 con 0x55ebebaadea0 2024-04-25T00:32:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1acce0 con 0x55ebebaad580 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 10 client.0 _ll_register_callbacks cb 0x7f20dc004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1440640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52930/0 (socket says 172.21.15.45:52930) 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 0x55ebebab2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1440640 1 -- 172.21.15.45:0/584618950 learned_addr learned my addr 172.21.15.45:0/584618950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1440640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebab1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 -- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 msgr2=0x55ebebab1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebab1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 -- 172.21.15.45:0/584618950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 msgr2=0x55ebebab2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 -- 172.21.15.45:0/584618950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebebb2dab0 con 0x55ebebaadea0 2024-04-25T00:32:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1440640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f1c41640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 0x55ebebab2330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006010 tx=0x7f20e8004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b840ae065be610c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8016040 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20dc018030 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.505+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f20e8003b70 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20e8039050 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 10 client.27767 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ebebb2ed90 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc018030 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20e800f040 con 0x55ebebaadea0 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 10 client.27767 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 10 client.27767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-25T00:32:23.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 20 client.27767 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebb2f280 0x55ebebb316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 4 client.27767 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 --> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebeba19f00 con 0x55ebebb2f280 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.506+0000 7f20f1440640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebb2f280 0x55ebebb316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f1440640 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebb2f280 0x55ebebb316a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=742587ff4772cdbd server_cookie=c8a20b3e6a9b99db in_seq=0 out_seq=0 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20d57fa640 10 client.27767 ms_handle_connect on v2:172.21.15.122:6836/3810708134 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20d57fa640 1 -- 172.21.15.45:0/584618950 <== mds.2 v2:172.21.15.122:6836/3810708134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebeba19f00 con 0x55ebebb2f280 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20d57fa640 10 client.27767 handle_command_reply: tid=0 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 client.27767 shutdown 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 msgr2=0x55ebebab2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 0x55ebebab2330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006010 tx=0x7f20e8004ac0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 shutdown_connections 2024-04-25T00:32:23.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:6836/3810708134,v1:172.21.15.122:6837/3810708134] conn(0x55ebebb2f280 0x55ebebb316a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ebebab8570 0x55ebebab2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ebebaadea0 0x55ebebab2330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 --2- 172.21.15.45:0/584618950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ebebaad580 0x55ebebab1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20d57fa640 0 client.27767 ms_handle_reset on v2:172.21.15.122:6836/3810708134 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 >> 172.21.15.45:0/584618950 conn(0x55ebebaa98a0 msgr2=0x55ebebaa82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 shutdown_connections 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 1 -- 172.21.15.45:0/584618950 wait complete. 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.507+0000 7f20f6bb8740 20 client.27767 trim_cache size 0 max 16384 2024-04-25T00:32:23.515 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:32:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 msgr2=0x7f20d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 0x7f20d80a51c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ed040 tx=0x7f20e40b12a0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 msgr2=0x7f20ec19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005ab0 tx=0x7f20e40042f0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 shutdown_connections 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20d80a2d00 0x7f20d80a51c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ec107730 0x7f20ec19c860 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20ec06a780 0x7f20ec1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 --2- 172.21.15.45:0/4216159494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20ec069dc0 0x7f20ec197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 >> 172.21.15.45:0/4216159494 conn(0x7f20ec0776a0 msgr2=0x7f20ec0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 shutdown_connections 2024-04-25T00:32:23.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.508+0000 7f20f3ea5640 1 -- 172.21.15.45:0/4216159494 wait complete. 2024-04-25T00:32:23.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:32:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:23 smithi122 ceph-mon[32959]: pgmap v323: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T00:32:23.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:23 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2744353376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:32:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- 172.21.15.45:0/4081118714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894069dc0 msgr2=0x7f889406a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 --2- 172.21.15.45:0/4081118714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894069dc0 0x7f889406a240 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8884009930 tx=0x7f888402a420 comp rx=0 tx=0).stop 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- 172.21.15.45:0/4081118714 shutdown_connections 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 --2- 172.21.15.45:0/4081118714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 0x7f889410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 --2- 172.21.15.45:0/4081118714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894069dc0 0x7f889406a240 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 --2- 172.21.15.45:0/4081118714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894107730 0x7f8894107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- 172.21.15.45:0/4081118714 >> 172.21.15.45:0/4081118714 conn(0x7f88940776a0 msgr2=0x7f8894077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- 172.21.15.45:0/4081118714 shutdown_connections 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- 172.21.15.45:0/4081118714 wait complete. 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 Processor -- start 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.623+0000 7f889b4cd640 1 -- start start 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894069dc0 0x7f8894100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 0x7f8894100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8894111b40 con 0x7f8894107730 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88941119c0 con 0x7f889406a780 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8894111cc0 con 0x7f8894069dc0 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52954/0 (socket says 172.21.15.45:52954) 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 -- 172.21.15.45:0/1114355907 learned_addr learned my addr 172.21.15.45:0/1114355907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899269640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894069dc0 0x7f8894100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8898a68640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 0x7f8894100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 -- 172.21.15.45:0/1114355907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894069dc0 msgr2=0x7f8894100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894069dc0 0x7f8894100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 -- 172.21.15.45:0/1114355907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 msgr2=0x7f8894100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 0x7f8894100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 -- 172.21.15.45:0/1114355907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88941a6520 con 0x7f8894107730 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f8899a6a640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f888c00eaa0 tx=0x7f888c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f888a7fc640 1 -- 172.21.15.45:0/1114355907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c00cdb0 con 0x7f8894107730 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f888a7fc640 1 -- 172.21.15.45:0/1114355907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888c005020 con 0x7f8894107730 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88941a6810 con 0x7f8894107730 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.624+0000 7f888a7fc640 1 -- 172.21.15.45:0/1114355907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c0116b0 con 0x7f8894107730 2024-04-25T00:32:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.625+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88941a6d30 con 0x7f8894107730 2024-04-25T00:32:23.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f888a7fc640 1 -- 172.21.15.45:0/1114355907 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f888c005560 con 0x7f8894107730 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f889e1da740 1 Processor -- start 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f889e1da740 1 -- start start 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f889e1da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x56240204c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f889e1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.627+0000 7f889e1da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402059960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f889e1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562401fb1670 con 0x56240204cea0 2024-04-25T00:32:23.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f889e1da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562401f59970 con 0x562402057570 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f889e1da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562401fad630 con 0x56240204c580 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f888a7fc640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 0x7f88800a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52964/0 (socket says 172.21.15.45:52964) 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 -- 172.21.15.45:0/2336269151 learned_addr learned my addr 172.21.15.45:0/2336269151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899a6a640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 0x7f88800a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8898a68640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x56240204c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8898a68640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402059960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f888a7fc640 1 -- 172.21.15.45:0/1114355907 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f888c0cfa30 con 0x7f8894107730 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 -- 172.21.15.45:0/2336269151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 msgr2=0x56240204c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x56240204c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 -- 172.21.15.45:0/2336269151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 msgr2=0x562402059960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402059960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 -- 172.21.15.45:0/2336269151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562401f7d400 con 0x56240204cea0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8898a68640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899a6a640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 0x7f88800a5240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f888c0cd3c0 tx=0x7f888c06db50 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f8899269640 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8890009ca0 tx=0x7f8890011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903d644d1c14c4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f887f7fe640 1 -- 172.21.15.45:0/2336269151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889001e070 con 0x56240204cea0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.628+0000 7f887f7fe640 1 -- 172.21.15.45:0/2336269151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8890019b10 con 0x56240204cea0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f887f7fe640 1 -- 172.21.15.45:0/2336269151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000ea40 con 0x56240204cea0 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- 172.21.15.45:0/2336269151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 msgr2=0x562402057030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8890009ca0 tx=0x7f8890011810 comp rx=0 tx=0).stop 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- 172.21.15.45:0/2336269151 shutdown_connections 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x562402057030 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 --2- 172.21.15.45:0/2336269151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x56240204c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- 172.21.15.45:0/2336269151 >> 172.21.15.45:0/2336269151 conn(0x5624020488a0 msgr2=0x562402048c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- 172.21.15.45:0/2336269151 shutdown_connections 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- 172.21.15.45:0/2336269151 wait complete. 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.629+0000 7f889e1da740 1 -- start start 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 Processor -- start 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402056ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402054730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56240205a360 con 0x56240204cea0 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562401f6d050 con 0x562402057570 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56240205a4e0 con 0x56240204c580 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f889e1da740 10 client.0 _ll_register_callbacks cb 0x562401cc4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52980/0 (socket says 172.21.15.45:52980) 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 -- 172.21.15.45:0/2285912786 learned_addr learned my addr 172.21.15.45:0/2285912786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8898a68640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402056ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8898a68640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402054730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 -- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 msgr2=0x562402056ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402056ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 -- 172.21.15.45:0/2285912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 msgr2=0x562402054730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402054730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8899269640 1 -- 172.21.15.45:0/2285912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562401f3edc0 con 0x56240204cea0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.630+0000 7f8898a68640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402054730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f8899269640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8890009c70 tx=0x7f8890019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb133ec62d90fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889001e050 con 0x56240204cea0 2024-04-25T00:32:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f8890004020 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8890019b10 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8890032050 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 10 client.18729 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562401cc53f0 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8890019b10 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f889000e7c0 con 0x56240204cea0 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 10 client.18729 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 20 client.18729 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x562402067560 0x562402069950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 4 client.18729 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:23.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.631+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562401fbebb0 con 0x562402067560 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.632+0000 7f8898a68640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x562402067560 0x562402069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.632+0000 7f8898a68640 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x562402067560 0x562402069950 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=786f58b34a95422b server_cookie=33e75d9565c54641 in_seq=0 out_seq=0 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.632+0000 7f887cff9640 10 client.18729 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f887cff9640 1 -- 172.21.15.45:0/2285912786 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562401fbebb0 con 0x562402067560 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f887cff9640 10 client.18729 handle_command_reply: tid=0 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 client.18729 shutdown 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 msgr2=0x5624020541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8890009c70 tx=0x7f8890019660 comp rx=0 tx=0).stop 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 shutdown_connections 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x562402067560 0x562402069950 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x562402054730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x5624020541d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f887cff9640 0 client.18729 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 --2- 172.21.15.45:0/2285912786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402056ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 >> 172.21.15.45:0/2285912786 conn(0x5624020488a0 msgr2=0x56240204e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 shutdown_connections 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 1 -- 172.21.15.45:0/2285912786 wait complete. 2024-04-25T00:32:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.633+0000 7f889e1da740 20 client.18729 trim_cache size 0 max 16384 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.646+0000 7f889e1da740 1 Processor -- start 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.646+0000 7f889e1da740 1 -- start start 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402063220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x56240206f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88941b0e50 con 0x56240204cea0 2024-04-25T00:32:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88941b2990 con 0x562402057570 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f889e1da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88941b1520 con 0x56240204c580 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52986/0 (socket says 172.21.15.45:52986) 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 -- 172.21.15.45:0/4098386806 learned_addr learned my addr 172.21.15.45:0/4098386806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 -- 172.21.15.45:0/4098386806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 msgr2=0x562402063220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402063220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 -- 172.21.15.45:0/4098386806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 msgr2=0x56240206f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x56240206f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 -- 172.21.15.45:0/4098386806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562401d8f7d0 con 0x56240204cea0 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f8899269640 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f889000eb60 tx=0x7f8890011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aced3e7363a2b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f887effd640 1 -- 172.21.15.45:0/4098386806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890015020 con 0x56240204cea0 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f887effd640 1 -- 172.21.15.45:0/4098386806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889001f040 con 0x56240204cea0 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.647+0000 7f887effd640 1 -- 172.21.15.45:0/4098386806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889001e070 con 0x56240204cea0 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- 172.21.15.45:0/4098386806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 msgr2=0x56240206f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f889000eb60 tx=0x7f8890011e40 comp rx=0 tx=0).stop 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- 172.21.15.45:0/4098386806 shutdown_connections 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562402057570 0x56240206f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56240204cea0 0x56240206f110 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 --2- 172.21.15.45:0/4098386806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204c580 0x562402063220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- 172.21.15.45:0/4098386806 >> 172.21.15.45:0/4098386806 conn(0x5624020488a0 msgr2=0x562402048c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- 172.21.15.45:0/4098386806 shutdown_connections 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- 172.21.15.45:0/4098386806 wait complete. 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 -- start start 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.648+0000 7f889e1da740 1 Processor -- start 2024-04-25T00:32:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56240204c580 0x56240205d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204cea0 0x56240205d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88941b22c0 con 0x562402057570 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88941b1bf0 con 0x56240204c580 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88941aee50 con 0x56240204cea0 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f889e1da740 10 client.0 _ll_register_callbacks cb 0x7f889001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53002/0 (socket says 172.21.15.45:53002) 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 -- 172.21.15.45:0/22391116 learned_addr learned my addr 172.21.15.45:0/22391116 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8899269640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204cea0 0x56240205d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56240204c580 0x56240205d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 -- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204cea0 msgr2=0x56240205d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204cea0 0x56240205d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 -- 172.21.15.45:0/22391116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56240204c580 msgr2=0x56240205d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56240204c580 0x56240205d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 -- 172.21.15.45:0/22391116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624020ccc30 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f8898a68640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2b18a650141e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.649+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884035040 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f888404a760 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889001f040 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8884058050 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 10 client.18735 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624020cdf30 con 0x562402057570 2024-04-25T00:32:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f889001f040 con 0x562402057570 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f888403a040 con 0x562402057570 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 10 client.18735 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 10 client.18735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 20 client.18735 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5624020ce6e0 0x5624020d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 4 client.18735 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 --> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562401fb8f00 con 0x5624020ce6e0 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.650+0000 7f8899269640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5624020ce6e0 0x5624020d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f887cff9640 10 client.18735 ms_handle_connect on v2:172.21.15.122:6838/2298542043 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f8899269640 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5624020ce6e0 0x5624020d0b00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a51c63efc15b2848 server_cookie=efccf39b31ea83ea in_seq=0 out_seq=0 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f887cff9640 1 -- 172.21.15.45:0/22391116 <== mds.1 v2:172.21.15.122:6838/2298542043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562401fb8f00 con 0x5624020ce6e0 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f887cff9640 10 client.18735 handle_command_reply: tid=0 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 client.18735 shutdown 2024-04-25T00:32:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 msgr2=0x56240205de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884042040 comp rx=0 tx=0).stop 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 shutdown_connections 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:6838/2298542043,v1:172.21.15.122:6839/2298542043] conn(0x5624020ce6e0 0x5624020d0b00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562402057570 0x56240205de90 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56240204cea0 0x56240205d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f887cff9640 0 client.18735 ms_handle_reset on v2:172.21.15.122:6838/2298542043 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 --2- 172.21.15.45:0/22391116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56240204c580 0x56240205d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 >> 172.21.15.45:0/22391116 conn(0x5624020488a0 msgr2=0x562402062210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 shutdown_connections 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 1 -- 172.21.15.45:0/22391116 wait complete. 2024-04-25T00:32:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.651+0000 7f889e1da740 20 client.18735 trim_cache size 0 max 16384 2024-04-25T00:32:23.660 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:32:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 msgr2=0x7f88800a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 0x7f88800a5240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f888c0cd3c0 tx=0x7f888c06db50 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 msgr2=0x7f88941010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f888c00eaa0 tx=0x7f888c00ef60 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 shutdown_connections 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88800a2d80 0x7f88800a5240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8894107730 0x7f88941010f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889406a780 0x7f8894100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 --2- 172.21.15.45:0/1114355907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8894069dc0 0x7f8894100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 >> 172.21.15.45:0/1114355907 conn(0x7f88940776a0 msgr2=0x7f88940fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 shutdown_connections 2024-04-25T00:32:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.652+0000 7f889b4cd640 1 -- 172.21.15.45:0/1114355907 wait complete. 2024-04-25T00:32:23.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:32:23.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- 172.21.15.45:0/1050426040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 msgr2=0x7feff806a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 --2- 172.21.15.45:0/1050426040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff806a240 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009930 tx=0x7fefe802a420 comp rx=0 tx=0).stop 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- 172.21.15.45:0/1050426040 shutdown_connections 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 --2- 172.21.15.45:0/1050426040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 0x7feff810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 --2- 172.21.15.45:0/1050426040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff806a240 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 --2- 172.21.15.45:0/1050426040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 0x7feff8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- 172.21.15.45:0/1050426040 >> 172.21.15.45:0/1050426040 conn(0x7feff80776a0 msgr2=0x7feff8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- 172.21.15.45:0/1050426040 shutdown_connections 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- 172.21.15.45:0/1050426040 wait complete. 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 Processor -- start 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.767+0000 7feffeea5640 1 -- start start 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 0x7feff81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 0x7feff819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feff8111b40 con 0x7feff8069dc0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feff81119c0 con 0x7feff8107730 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff8111cc0 con 0x7feff806a780 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57488/0 (socket says 172.21.15.45:57488) 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 -- 172.21.15.45:0/229814275 learned_addr learned my addr 172.21.15.45:0/229814275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feff7fff640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 0x7feff81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffd442640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 0x7feff819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 -- 172.21.15.45:0/229814275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 msgr2=0x7feff81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 0x7feff81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 -- 172.21.15.45:0/229814275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 msgr2=0x7feff819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 0x7feff819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 -- 172.21.15.45:0/229814275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff819d000 con 0x7feff8069dc0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffcc41640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fefe000eab0 tx=0x7fefe000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feff5ffb640 1 -- 172.21.15.45:0/229814275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe000ce40 con 0x7feff8069dc0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feff5ffb640 1 -- 172.21.15.45:0/229814275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe0011950 con 0x7feff8069dc0 2024-04-25T00:32:23.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.768+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff819d2f0 con 0x7feff8069dc0 2024-04-25T00:32:23.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.769+0000 7feff5ffb640 1 -- 172.21.15.45:0/229814275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0010ae0 con 0x7feff8069dc0 2024-04-25T00:32:23.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.769+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a4b80 con 0x7feff8069dc0 2024-04-25T00:32:23.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7feff5ffb640 1 -- 172.21.15.45:0/229814275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fefe0011b00 con 0x7feff8069dc0 2024-04-25T00:32:23.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 Processor -- start 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 -- start start 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7feff5ffb640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 0x7fefe40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21d80 0x560001a30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a30980 0x560001a32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560001986670 con 0x560001a21460 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56000192e970 con 0x560001a30980 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.771+0000 7ff001bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560001982630 con 0x560001a21d80 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffd442640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 0x7fefe40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feff5ffb640 1 -- 172.21.15.45:0/229814275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fefe00cfde0 con 0x7feff8069dc0 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57490/0 (socket says 172.21.15.45:57490) 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 -- 172.21.15.45:0/2653498193 learned_addr learned my addr 172.21.15.45:0/2653498193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffd442640 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a30980 0x560001a32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feff7fff640 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21d80 0x560001a30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 -- 172.21.15.45:0/2653498193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21d80 msgr2=0x560001a30440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21d80 0x560001a30440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 -- 172.21.15.45:0/2653498193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a30980 msgr2=0x560001a32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a30980 0x560001a32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 -- 172.21.15.45:0/2653498193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560001952400 con 0x560001a21460 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffd442640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 0x7fefe40a51c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feff0009710 tx=0x7feff0009a70 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7feffcc41640 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fefe00cd1d0 tx=0x7fefe009bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b230040193f0b0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7fefeeffd640 1 -- 172.21.15.45:0/2653498193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe009c640 con 0x560001a21460 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7fefeeffd640 1 -- 172.21.15.45:0/2653498193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe009bf40 con 0x560001a21460 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.772+0000 7fefeeffd640 1 -- 172.21.15.45:0/2653498193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00985b0 con 0x560001a21460 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- 172.21.15.45:0/2653498193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 msgr2=0x560001a21840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fefe00cd1d0 tx=0x7fefe009bb40 comp rx=0 tx=0).stop 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- 172.21.15.45:0/2653498193 shutdown_connections 2024-04-25T00:32:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a30980 0x560001a32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21d80 0x560001a30440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2653498193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a21840 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- 172.21.15.45:0/2653498193 >> 172.21.15.45:0/2653498193 conn(0x560001a1d8a0 msgr2=0x560001a1dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- 172.21.15.45:0/2653498193 shutdown_connections 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- 172.21.15.45:0/2653498193 wait complete. 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 -- start start 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.773+0000 7ff001bb8740 1 Processor -- start 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21460 0x560001a30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a2d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560001a337c0 con 0x560001a30980 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560001942050 con 0x560001a21d80 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560001a33940 con 0x560001a21460 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7ff001bb8740 10 client.0 _ll_register_callbacks cb 0x560001699be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feff7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a2d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57502/0 (socket says 172.21.15.45:57502) 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feff7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a2d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.45:40252/0 (socket says 172.21.15.45:40252) 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 -- 172.21.15.45:0/1497351333 learned_addr learned my addr 172.21.15.45:0/1497351333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffcc41640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21460 0x560001a30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 -- 172.21.15.45:0/1497351333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21460 msgr2=0x560001a30260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21460 0x560001a30260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 -- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 msgr2=0x560001a2d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a2d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.774+0000 7feffd442640 1 -- 172.21.15.45:0/1497351333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560001913dc0 con 0x560001a30980 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7feffd442640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feff0004290 tx=0x7feff001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b78fb84d9ebee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00042c0 con 0x560001a30980 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7feff0004460 con 0x560001a30980 2024-04-25T00:32:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe009bf40 con 0x560001a30980 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feff0032050 con 0x560001a30980 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 10 client.18747 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56000169a3f0 con 0x560001a30980 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe009bf40 con 0x560001a30980 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feff0020050 con 0x560001a30980 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 10 client.18747 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 20 client.18747 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001a409d0 0x560001a42de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 4 client.18747 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.775+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560001993bb0 con 0x560001a409d0 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.776+0000 7feffcc41640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001a409d0 0x560001a42de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.776+0000 7feffcc41640 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001a409d0 0x560001a42de0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f214def6a8246134 server_cookie=26c7bda16bc72043 in_seq=0 out_seq=0 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.776+0000 7fefd7fff640 10 client.18747 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7fefd7fff640 1 -- 172.21.15.45:0/1497351333 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560001993bb0 con 0x560001a409d0 2024-04-25T00:32:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7fefd7fff640 10 client.18747 handle_command_reply: tid=0 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 client.18747 shutdown 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 msgr2=0x560001a2da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7feff0004290 tx=0x7feff001d000 comp rx=0 tx=0).stop 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 shutdown_connections 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001a409d0 0x560001a42de0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a30980 0x560001a2da90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a2d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7fefd7fff640 0 client.18747 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 --2- 172.21.15.45:0/1497351333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a21460 0x560001a30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 >> 172.21.15.45:0/1497351333 conn(0x560001a1d8a0 msgr2=0x560001a23630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 shutdown_connections 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 1 -- 172.21.15.45:0/1497351333 wait complete. 2024-04-25T00:32:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.777+0000 7ff001bb8740 20 client.18747 trim_cache size 0 max 16384 2024-04-25T00:32:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 Processor -- start 2024-04-25T00:32:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 -- start start 2024-04-25T00:32:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feff81aca70 con 0x560001a21460 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feff81ae5b0 con 0x560001a21d80 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7ff001bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff81ad140 con 0x560001a30980 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57504/0 (socket says 172.21.15.45:57504) 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 -- 172.21.15.45:0/2456316541 learned_addr learned my addr 172.21.15.45:0/2456316541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feff7fff640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffd442640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 -- 172.21.15.45:0/2456316541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 msgr2=0x560001a48bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a48bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 -- 172.21.15.45:0/2456316541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 msgr2=0x560001a486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 -- 172.21.15.45:0/2456316541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600017647d0 con 0x560001a21460 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffd442640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a48bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.791+0000 7feffcc41640 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fefe009bed0 tx=0x7fefe009bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04f4c120adbf53e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7fefee7fc640 1 -- 172.21.15.45:0/2456316541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe009a240 con 0x560001a21460 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7fefee7fc640 1 -- 172.21.15.45:0/2456316541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe00cd3f0 con 0x560001a21460 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7fefee7fc640 1 -- 172.21.15.45:0/2456316541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00a4720 con 0x560001a21460 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 -- 172.21.15.45:0/2456316541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 msgr2=0x560001a2fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fefe009bed0 tx=0x7fefe009bbb0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 -- 172.21.15.45:0/2456316541 shutdown_connections 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a48bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21d80 0x560001a486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 --2- 172.21.15.45:0/2456316541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21460 0x560001a2fd90 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 -- 172.21.15.45:0/2456316541 >> 172.21.15.45:0/2456316541 conn(0x560001a1d8a0 msgr2=0x560001a22d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 -- 172.21.15.45:0/2456316541 shutdown_connections 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.792+0000 7ff001bb8740 1 -- 172.21.15.45:0/2456316541 wait complete. 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 -- start start 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 Processor -- start 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21460 0x560001a2a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a2acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feff81adee0 con 0x560001a21d80 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feff81ad810 con 0x560001a21460 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff81aaa70 con 0x560001a30980 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7ff001bb8740 10 client.0 _ll_register_callbacks cb 0x7feff0009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57520/0 (socket says 172.21.15.45:57520) 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 -- 172.21.15.45:0/4003089703 learned_addr learned my addr 172.21.15.45:0/4003089703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feffcc41640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21460 0x560001a2a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 -- 172.21.15.45:0/4003089703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 msgr2=0x560001a2acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a2acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 -- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21460 msgr2=0x560001a2a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21460 0x560001a2a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.793+0000 7feff7fff640 1 -- 172.21.15.45:0/4003089703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560001aa5f70 con 0x560001a21d80 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7feff7fff640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009930 tx=0x7fefe8031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aa3d42f4905ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe804b060 con 0x560001a21d80 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fefe8004460 con 0x560001a21d80 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe00cd3f0 con 0x560001a21d80 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fefe8052050 con 0x560001a21d80 2024-04-25T00:32:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 10 client.18753 fetch_fsmap learned FSMap version 28 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560001aa7210 con 0x560001a21d80 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe00cd3f0 con 0x560001a21d80 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fefe803c7b0 con 0x560001a21d80 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 10 client.18753 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 20 client.18753 populate_metadata read hostname 'smithi045' 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001aa7a10 0x560001aa9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 4 client.18753 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.794+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 --> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56000198df00 con 0x560001aa7a10 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7feffcc41640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001aa7a10 0x560001aa9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7fefd7fff640 10 client.18753 ms_handle_connect on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7feffcc41640 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001aa7a10 0x560001aa9e30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b9069401da4ab9 server_cookie=c4c122ccb4a60eec in_seq=0 out_seq=0 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7fefd7fff640 1 -- 172.21.15.45:0/4003089703 <== mds.0 v2:172.21.15.71:6838/102345855 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56000198df00 con 0x560001aa7a10 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7fefd7fff640 10 client.18753 handle_command_reply: tid=0 2024-04-25T00:32:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 client.18753 shutdown 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 msgr2=0x560001a2a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009930 tx=0x7fefe8031920 comp rx=0 tx=0).stop 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 shutdown_connections 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560001a30980 0x560001a2acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:6838/102345855,v1:172.21.15.71:6839/102345855] conn(0x560001aa7a10 0x560001aa9e30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560001a21d80 0x560001a2a7b0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 --2- 172.21.15.45:0/4003089703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560001a21460 0x560001a2a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 >> 172.21.15.45:0/4003089703 conn(0x560001a1d8a0 msgr2=0x560001a22d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.795+0000 7fefd7fff640 0 client.18753 ms_handle_reset on v2:172.21.15.71:6838/102345855 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 shutdown_connections 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7ff001bb8740 1 -- 172.21.15.45:0/4003089703 wait complete. 2024-04-25T00:32:23.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7ff001bb8740 20 client.18753 trim_cache size 0 max 16384 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 msgr2=0x7fefe40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 0x7fefe40a51c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feff0009710 tx=0x7feff0009a70 comp rx=0 tx=0).stop 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 msgr2=0x7feff8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:23.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fefe000eab0 tx=0x7fefe000ef70 comp rx=0 tx=0).stop 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 shutdown_connections 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fefe40a2d00 0x7fefe40a51c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feff8107730 0x7feff819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff806a780 0x7feff81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 --2- 172.21.15.45:0/229814275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feff8069dc0 0x7feff8198060 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 >> 172.21.15.45:0/229814275 conn(0x7feff80776a0 msgr2=0x7feff80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 shutdown_connections 2024-04-25T00:32:23.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:23.796+0000 7feffeea5640 1 -- 172.21.15.45:0/229814275 wait complete. 2024-04-25T00:32:24.808 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T00:32:24.808 INFO:tasks.fwd_scrub:done joining 2024-04-25T00:32:24.808 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T00:32:24.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 -- 172.21.15.45:0/128995319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 msgr2=0x7fd80c110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 --2- 172.21.15.45:0/128995319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c110540 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd804009ca0 tx=0x7fd80402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 -- 172.21.15.45:0/128995319 shutdown_connections 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 --2- 172.21.15.45:0/128995319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c110540 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 --2- 172.21.15.45:0/128995319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 0x7fd80c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 --2- 172.21.15.45:0/128995319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 0x7fd80c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.934+0000 7fd81356a640 1 -- 172.21.15.45:0/128995319 >> 172.21.15.45:0/128995319 conn(0x7fd80c0777a0 msgr2=0x7fd80c077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- 172.21.15.45:0/128995319 shutdown_connections 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- 172.21.15.45:0/128995319 wait complete. 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 Processor -- start 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- start start 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 0x7fd80c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 0x7fd80c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c113d60 con 0x7fd80c10e150 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c113be0 con 0x7fd80c1005a0 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.935+0000 7fd81356a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c113ee0 con 0x7fd80c100ec0 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57530/0 (socket says 172.21.15.45:57530) 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 -- 172.21.15.45:0/2283710235 learned_addr learned my addr 172.21.15.45:0/2283710235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811306640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 0x7fd80c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd810b05640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 0x7fd80c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 -- 172.21.15.45:0/2283710235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 msgr2=0x7fd80c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 0x7fd80c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 -- 172.21.15.45:0/2283710235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 msgr2=0x7fd80c19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 0x7fd80c19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 -- 172.21.15.45:0/2283710235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c19f150 con 0x7fd80c10e150 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd811b07640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd80402ade0 tx=0x7fd8040319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804003c30 con 0x7fd80c10e150 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c19f3e0 con 0x7fd80c10e150 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd804003dd0 con 0x7fd80c10e150 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80402fc40 con 0x7fd80c10e150 2024-04-25T00:32:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.936+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1a6cd0 con 0x7fd80c10e150 2024-04-25T00:32:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.937+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c0689b0 con 0x7fd80c10e150 2024-04-25T00:32:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.939+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd804004b40 con 0x7fd80c10e150 2024-04-25T00:32:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.939+0000 7fd8027fc640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 0x7fd7f80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.940+0000 7fd811306640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 0x7fd7f80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.940+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fd8040f0a00 con 0x7fd80c10e150 2024-04-25T00:32:24.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.940+0000 7fd811306640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 0x7fd7f80a51c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd808005d00 tx=0x7fd80800b410 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:24.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:24.943+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd8040bd310 con 0x7fd80c10e150 2024-04-25T00:32:25.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.130+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd80c1a6f00 con 0x7fd80c10e150 2024-04-25T00:32:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.131+0000 7fd8027fc640 1 -- 172.21.15.45:0/2283710235 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+25272 (secure 0 0 0) 0x7fd8040bcc10 con 0x7fd80c10e150 2024-04-25T00:32:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:32:25.133 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":102,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:32:12.003289+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:32:25.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 msgr2=0x7fd7f80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 0x7fd7f80a51c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd808005d00 tx=0x7fd80800b410 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 msgr2=0x7fd80c19ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd80402ade0 tx=0x7fd8040319f0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 shutdown_connections 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fd7f80a2d00 0x7fd7f80a51c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd80c10e150 0x7fd80c19ea20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd80c100ec0 0x7fd80c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 --2- 172.21.15.45:0/2283710235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd80c1005a0 0x7fd80c19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 >> 172.21.15.45:0/2283710235 conn(0x7fd80c0777a0 msgr2=0x7fd80c10f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 shutdown_connections 2024-04-25T00:32:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.134+0000 7fd81356a640 1 -- 172.21.15.45:0/2283710235 wait complete. 2024-04-25T00:32:25.145 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:32:25.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:32:25.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.250+0000 7f102f438640 1 -- 172.21.15.45:0/1906375216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100610 msgr2=0x7f1028100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.250+0000 7f102f438640 1 --2- 172.21.15.45:0/1906375216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100610 0x7f1028100a90 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1018009930 tx=0x7f101802a420 comp rx=0 tx=0).stop 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 -- 172.21.15.45:0/1906375216 shutdown_connections 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 --2- 172.21.15.45:0/1906375216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100fd0 0x7f102810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 --2- 172.21.15.45:0/1906375216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100610 0x7f1028100a90 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 --2- 172.21.15.45:0/1906375216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 -- 172.21.15.45:0/1906375216 >> 172.21.15.45:0/1906375216 conn(0x7f1028077740 msgr2=0x7f1028077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 -- 172.21.15.45:0/1906375216 shutdown_connections 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 -- 172.21.15.45:0/1906375216 wait complete. 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 Processor -- start 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.251+0000 7f102f438640 1 -- start start 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100610 0x7f10281022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1028110a30 con 0x7f1028100fd0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f10281108b0 con 0x7f10281066b0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1028110bb0 con 0x7f1028100610 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57556/0 (socket says 172.21.15.45:57556) 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 -- 172.21.15.45:0/2096429879 learned_addr learned my addr 172.21.15.45:0/2096429879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102d1d4640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100610 0x7f10281022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102d9d5640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 -- 172.21.15.45:0/2096429879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100610 msgr2=0x7f10281022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100610 0x7f10281022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 -- 172.21.15.45:0/2096429879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 msgr2=0x7f1028102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 -- 172.21.15.45:0/2096429879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10281a64b0 con 0x7f1028100fd0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102d9d5640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102c9d3640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1018009230 tx=0x7f1018004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101803f030 con 0x7f1028100fd0 2024-04-25T00:32:25.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281a6740 con 0x7f1028100fd0 2024-04-25T00:32:25.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1018003ad0 con 0x7f1028100fd0 2024-04-25T00:32:25.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.252+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018035070 con 0x7f1028100fd0 2024-04-25T00:32:25.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.253+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281a6c30 con 0x7f1028100fd0 2024-04-25T00:32:25.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.253+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1028068560 con 0x7f1028100fd0 2024-04-25T00:32:25.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.255+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1018042030 con 0x7f1028100fd0 2024-04-25T00:32:25.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.256+0000 7f101e7fc640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 0x7f10140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.256+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f10180f01a0 con 0x7f1028100fd0 2024-04-25T00:32:25.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.256+0000 7f102d1d4640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 0x7f10140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.256+0000 7f102d1d4640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 0x7f10140a51c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1024005fd0 tx=0x7f102400b410 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.259+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f10180bc9a0 con 0x7f1028100fd0 2024-04-25T00:32:25.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:25 smithi045 ceph-mon[29194]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:32:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.444+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10281a6e90 con 0x7f1028100fd0 2024-04-25T00:32:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.444+0000 7f101e7fc640 1 -- 172.21.15.45:0/2096429879 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f10180bc2a0 con 0x7f1028100fd0 2024-04-25T00:32:25.445 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.446+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 msgr2=0x7f10140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.446+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 0x7f10140a51c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1024005fd0 tx=0x7f102400b410 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.446+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 msgr2=0x7f10281027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.446+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1018009230 tx=0x7f1018004270 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 shutdown_connections 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f10140a2d00 0x7f10140a51c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10281066b0 0x7f1028102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1028100fd0 0x7f10281027f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 --2- 172.21.15.45:0/2096429879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1028100610 0x7f10281022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 >> 172.21.15.45:0/2096429879 conn(0x7f1028077740 msgr2=0x7f10280ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 shutdown_connections 2024-04-25T00:32:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.447+0000 7f102f438640 1 -- 172.21.15.45:0/2096429879 wait complete. 2024-04-25T00:32:25.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:32:25.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:25 smithi122 ceph-mon[32959]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:32:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 -- 172.21.15.45:0/537659467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 msgr2=0x7fc26c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 --2- 172.21.15.45:0/537659467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c10e1a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc264009ca0 tx=0x7fc26402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 -- 172.21.15.45:0/537659467 shutdown_connections 2024-04-25T00:32:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 --2- 172.21.15.45:0/537659467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c10e1a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 --2- 172.21.15.45:0/537659467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1032e0 0x7fc26c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 --2- 172.21.15.45:0/537659467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1029c0 0x7fc26c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 -- 172.21.15.45:0/537659467 >> 172.21.15.45:0/537659467 conn(0x7fc26c077740 msgr2=0x7fc26c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 -- 172.21.15.45:0/537659467 shutdown_connections 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 -- 172.21.15.45:0/537659467 wait complete. 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.563+0000 7fc2741ce640 1 Processor -- start 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 -- start start 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1029c0 0x7fc26c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1032e0 0x7fc26c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c111b00 con 0x7fc26c10bd00 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c111980 con 0x7fc26c1029c0 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc2741ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c111c80 con 0x7fc26c1032e0 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57578/0 (socket says 172.21.15.45:57578) 2024-04-25T00:32:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 -- 172.21.15.45:0/4232423237 learned_addr learned my addr 172.21.15.45:0/4232423237 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc271f6a640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1029c0 0x7fc26c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc271769640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1032e0 0x7fc26c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 -- 172.21.15.45:0/4232423237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1032e0 msgr2=0x7fc26c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1032e0 0x7fc26c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 -- 172.21.15.45:0/4232423237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1029c0 msgr2=0x7fc26c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1029c0 0x7fc26c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 -- 172.21.15.45:0/4232423237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c19cfa0 con 0x7fc26c10bd00 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.564+0000 7fc27276b640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc264006010 tx=0x7fc264003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.565+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264047070 con 0x7fc26c10bd00 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.565+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c19d230 con 0x7fc26c10bd00 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.565+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26402fb20 con 0x7fc26c10bd00 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.565+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264033050 con 0x7fc26c10bd00 2024-04-25T00:32:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.565+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1a4b20 con 0x7fc26c10bd00 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.566+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c068560 con 0x7fc26c10bd00 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.567+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc26404a070 con 0x7fc26c10bd00 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.567+0000 7fc262ffd640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 0x7fc2580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.567+0000 7fc271f6a640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 0x7fc2580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.567+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fc2640f03f0 con 0x7fc26c10bd00 2024-04-25T00:32:25.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.568+0000 7fc271f6a640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 0x7fc2580a51c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc268005fd0 tx=0x7fc26800b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.572+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc26407e740 con 0x7fc26c10bd00 2024-04-25T00:32:25.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:25 smithi071 ceph-mon[32667]: pgmap v324: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 88 B/s wr, 12 op/s 2024-04-25T00:32:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.759+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc26c106eb0 con 0x7fc26c10bd00 2024-04-25T00:32:25.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.759+0000 7fc262ffd640 1 -- 172.21.15.45:0/4232423237 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7fc2640bcc50 con 0x7fc26c10bd00 2024-04-25T00:32:25.762 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:32:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 msgr2=0x7fc2580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 0x7fc2580a51c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc268005fd0 tx=0x7fc26800b390 comp rx=0 tx=0).stop 2024-04-25T00:32:25.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 msgr2=0x7fc26c19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fc264006010 tx=0x7fc264003ba0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 shutdown_connections 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc2580a2d00 0x7fc2580a51c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc26c10bd00 0x7fc26c19c870 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc26c1032e0 0x7fc26c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 --2- 172.21.15.45:0/4232423237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c1029c0 0x7fc26c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.761+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 >> 172.21.15.45:0/4232423237 conn(0x7fc26c077740 msgr2=0x7fc26c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.762+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 shutdown_connections 2024-04-25T00:32:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.762+0000 7fc2741ce640 1 -- 172.21.15.45:0/4232423237 wait complete. 2024-04-25T00:32:25.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.877+0000 7f8903c7b640 1 -- 172.21.15.45:0/2753595841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 msgr2=0x7f88fc06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.877+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2753595841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc06a240 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009930 tx=0x7f88ec02a420 comp rx=0 tx=0).stop 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- 172.21.15.45:0/2753595841 shutdown_connections 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2753595841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc06a780 0x7f88fc10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2753595841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc06a240 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2753595841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc107730 0x7f88fc107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- 172.21.15.45:0/2753595841 >> 172.21.15.45:0/2753595841 conn(0x7f88fc0776a0 msgr2=0x7f88fc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- 172.21.15.45:0/2753595841 shutdown_connections 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- 172.21.15.45:0/2753595841 wait complete. 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 Processor -- start 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- start start 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc06a780 0x7f88fc198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 0x7f88fc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc111b90 con 0x7f88fc069dc0 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc111a10 con 0x7f88fc06a780 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8903c7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc111d10 con 0x7f88fc107730 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.878+0000 7f8901a17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57584/0 (socket says 172.21.15.45:57584) 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 -- 172.21.15.45:0/2507617111 learned_addr learned my addr 172.21.15.45:0/2507617111 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8902218640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 0x7f88fc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901216640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc06a780 0x7f88fc198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 -- 172.21.15.45:0/2507617111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 msgr2=0x7f88fc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 0x7f88fc19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 -- 172.21.15.45:0/2507617111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc06a780 msgr2=0x7f88fc198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc06a780 0x7f88fc198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 -- 172.21.15.45:0/2507617111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc19d100 con 0x7f88fc069dc0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8902218640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 0x7f88fc19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8901a17640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f88f800d950 tx=0x7f88f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8017070 con 0x7f88fc069dc0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc19d3f0 con 0x7f88fc069dc0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f8009c30 con 0x7f88fc069dc0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f800f6b0 con 0x7f88fc069dc0 2024-04-25T00:32:25.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.879+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc1a4c80 con 0x7f88fc069dc0 2024-04-25T00:32:25.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.880+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc0689b0 con 0x7f88fc069dc0 2024-04-25T00:32:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.881+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f88f80109d0 con 0x7f88fc069dc0 2024-04-25T00:32:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.882+0000 7f88f2ffd640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 0x7f88e80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.882+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f88f80cf450 con 0x7f88fc069dc0 2024-04-25T00:32:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.882+0000 7f8901216640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 0x7f88e80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:25.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.882+0000 7f8901216640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 0x7f88e80a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88ec006010 tx=0x7f88ec005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:25.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:25.887+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f88f8009de0 con 0x7f88fc069dc0 2024-04-25T00:32:26.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.073+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88fc109ce0 con 0x7f88fc069dc0 2024-04-25T00:32:26.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.074+0000 7f88f2ffd640 1 -- 172.21.15.45:0/2507617111 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f88f809bcb0 con 0x7f88fc069dc0 2024-04-25T00:32:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:32:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 msgr2=0x7f88e80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 0x7f88e80a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88ec006010 tx=0x7f88ec005a20 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 msgr2=0x7f88fc198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f88f800d950 tx=0x7f88f800de10 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 shutdown_connections 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f88e80a2d00 0x7f88e80a51c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88fc107730 0x7f88fc19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc06a780 0x7f88fc198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 --2- 172.21.15.45:0/2507617111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88fc069dc0 0x7f88fc198110 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 >> 172.21.15.45:0/2507617111 conn(0x7f88fc0776a0 msgr2=0x7f88fc0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 shutdown_connections 2024-04-25T00:32:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.076+0000 7f8903c7b640 1 -- 172.21.15.45:0/2507617111 wait complete. 2024-04-25T00:32:26.087 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:32:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3892853025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 msgr2=0x7fdb3010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3892853025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb3010e5e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009cd0 tx=0x7fdb2802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3892853025 shutdown_connections 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3892853025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb3010e5e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3892853025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 0x7fdb30101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3892853025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 0x7fdb30107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3892853025 >> 172.21.15.45:0/3892853025 conn(0x7fdb300fd180 msgr2=0x7fdb300ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3892853025 shutdown_connections 2024-04-25T00:32:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.193+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3892853025 wait complete. 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 Processor -- start 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 -- start start 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 0x7fdb30198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 0x7fdb3019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30111b30 con 0x7fdb301020d0 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301119b0 con 0x7fdb30107720 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb36d9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30111cb0 con 0x7fdb30101710 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb2ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57600/0 (socket says 172.21.15.45:57600) 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb2ffff640 1 -- 172.21.15.45:0/3120688342 learned_addr learned my addr 172.21.15.45:0/3120688342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb34b3a640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 0x7fdb30198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.194+0000 7fdb3533b640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 0x7fdb3019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 -- 172.21.15.45:0/3120688342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 msgr2=0x7fdb30198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 0x7fdb30198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 -- 172.21.15.45:0/3120688342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 msgr2=0x7fdb3019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 0x7fdb3019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 -- 172.21.15.45:0/3120688342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3019d020 con 0x7fdb301020d0 2024-04-25T00:32:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2ffff640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d770 tx=0x7fdb1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c017070 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3019d310 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c005500 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c004e00 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.195+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301a4ba0 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.196+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb300689b0 con 0x7fdb301020d0 2024-04-25T00:32:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.197+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb1c004fa0 con 0x7fdb301020d0 2024-04-25T00:32:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.198+0000 7fdb2dffb640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 0x7fdb200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.198+0000 7fdb34b3a640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 0x7fdb200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.198+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fdb1c0cf2f0 con 0x7fdb301020d0 2024-04-25T00:32:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.198+0000 7fdb34b3a640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 0x7fdb200a51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005fd0 tx=0x7fdb24005850 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.202+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdb1c09bb50 con 0x7fdb301020d0 2024-04-25T00:32:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.385+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdb30109710 con 0x7fdb301020d0 2024-04-25T00:32:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.385+0000 7fdb2dffb640 1 -- 172.21.15.45:0/3120688342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) ==== 103+0+11 (secure 0 0 0) 0x7fdb1c09b450 con 0x7fdb301020d0 2024-04-25T00:32:26.386 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 msgr2=0x7fdb200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 0x7fdb200a51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb24005fd0 tx=0x7fdb24005850 comp rx=0 tx=0).stop 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 msgr2=0x7fdb30198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d770 tx=0x7fdb1c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 shutdown_connections 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fdb200a2d00 0x7fdb200a51c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb30107720 0x7fdb3019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301020d0 0x7fdb30198570 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 --2- 172.21.15.45:0/3120688342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb30101710 0x7fdb30198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 >> 172.21.15.45:0/3120688342 conn(0x7fdb300fd180 msgr2=0x7fdb300fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 shutdown_connections 2024-04-25T00:32:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.387+0000 7fdb36d9e640 1 -- 172.21.15.45:0/3120688342 wait complete. 2024-04-25T00:32:26.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:26 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2283710235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:32:26.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:26 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2096429879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:26 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4232423237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:26 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2507617111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.399 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:32:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.500+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/1558321907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8100690 msgr2=0x7fe2c8100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.500+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/1558321907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8100690 0x7fe2c8100b10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009930 tx=0x7fe2bc02a420 comp rx=0 tx=0).stop 2024-04-25T00:32:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.500+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/1558321907 shutdown_connections 2024-04-25T00:32:26.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.500+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/1558321907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8101050 0x7fe2c810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.500+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/1558321907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8100690 0x7fe2c8100b10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/1558321907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 0x7fe2c8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/1558321907 >> 172.21.15.45:0/1558321907 conn(0x7fe2c80777a0 msgr2=0x7fe2c8077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/1558321907 shutdown_connections 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/1558321907 wait complete. 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 Processor -- start 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- start start 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8100690 0x7fe2c8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 0x7fe2c81a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8110a60 con 0x7fe2c8101050 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81108e0 con 0x7fe2c8100690 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.501+0000 7fe2ce1b0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8110be0 con 0x7fe2c8106650 2024-04-25T00:32:26.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57618/0 (socket says 172.21.15.45:57618) 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 -- 172.21.15.45:0/999179475 learned_addr learned my addr 172.21.15.45:0/999179475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c7fff640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 0x7fe2c81a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c77fe640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8100690 0x7fe2c8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 -- 172.21.15.45:0/999179475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 msgr2=0x7fe2c81a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 0x7fe2c81a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 -- 172.21.15.45:0/999179475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8100690 msgr2=0x7fe2c8198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8100690 0x7fe2c8198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 -- 172.21.15.45:0/999179475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c81a4560 con 0x7fe2c8101050 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c6ffd640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009230 tx=0x7fe2bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc03f030 con 0x7fe2c8101050 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c81a4730 con 0x7fe2c8101050 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2bc003ad0 con 0x7fe2c8101050 2024-04-25T00:32:26.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc035070 con 0x7fe2c8101050 2024-04-25T00:32:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.502+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81a4bf0 con 0x7fe2c8101050 2024-04-25T00:32:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.503+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c80689b0 con 0x7fe2c8101050 2024-04-25T00:32:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.505+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe2bc042030 con 0x7fe2c8101050 2024-04-25T00:32:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.505+0000 7fe2c4ff9640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 0x7fe2b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.505+0000 7fe2c77fe640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 0x7fe2b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.505+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fe2bc0efc80 con 0x7fe2c8101050 2024-04-25T00:32:26.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.506+0000 7fe2c77fe640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 0x7fe2b00a51c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005fd0 tx=0x7fe2b800b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:26.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.510+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe2bc0bbe90 con 0x7fe2c8101050 2024-04-25T00:32:26.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:26 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2283710235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:32:26.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:26 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2096429879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:26 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4232423237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:26 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2507617111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:26 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2283710235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:32:26.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:26 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2096429879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:26 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4232423237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:26 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2507617111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:32:26.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.749+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe2c8108bd0 con 0x7fe2c8101050 2024-04-25T00:32:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.758+0000 7fe2c4ff9640 1 -- 172.21.15.45:0/999179475 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740901 (secure 0 0 0) 0x7fe2bc0c0190 con 0x7fe2c8101050 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 25, 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 25163, 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1314659631 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1314659631 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/1314659631", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-04-25T00:21:57.361963+0000", 2024-04-25T00:32:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 15540, 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T00:32:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:32:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:32:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:32:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T00:32:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:32:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:32:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:32:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:32:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T00:32:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T00:32:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T00:32:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:32:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:32:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:32:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:32:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:32:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:32:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:32:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T00:32:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T00:32:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:32:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T00:32:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T00:32:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T00:32:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T00:32:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:32:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:32:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T00:32:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T00:32:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T00:32:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T00:32:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T00:32:26.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T00:32:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T00:32:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:32:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T00:32:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T00:32:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T00:32:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:32:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T00:32:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T00:32:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T00:32:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T00:32:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T00:32:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T00:32:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25141, 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T00:32:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T00:32:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T00:32:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:32:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T00:32:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T00:32:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T00:32:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T00:32:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T00:32:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T00:32:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T00:32:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:32:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T00:32:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T00:32:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:32:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T00:32:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:32:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T00:32:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:32:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T00:32:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:32:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:32:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:32:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T00:32:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:32:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:32:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:32:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:32:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:32:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T00:32:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T00:32:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T00:32:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T00:32:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T00:32:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T00:32:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T00:32:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T00:32:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T00:32:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T00:32:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T00:32:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T00:32:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T00:32:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T00:32:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T00:32:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T00:32:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:32:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T00:32:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T00:32:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T00:32:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T00:32:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T00:32:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T00:32:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T00:32:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T00:32:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T00:32:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T00:32:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T00:32:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T00:32:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T00:32:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T00:32:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T00:32:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T00:32:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T00:32:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.610 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.611 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.614 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:32:27.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-04-25T00:32:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.622 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.625 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.627 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T00:32:27.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T00:32:27.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T00:32:27.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T00:32:27.634 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.637 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.638 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.640 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T00:32:27.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T00:32:27.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T00:32:27.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:27.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:32:27.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:32:27.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:32:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T00:32:27.660 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:32:27.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:32:27.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:32:27.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.691 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.695 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:32:27.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:32:27.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T00:32:27.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T00:32:27.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.705 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T00:32:27.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T00:32:27.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:32:27.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T00:32:27.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.710 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T00:32:27.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T00:32:27.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:32:27.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:32:27.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T00:32:27.716 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.720 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T00:32:27.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T00:32:27.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T00:32:27.723 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.729 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.730 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.731 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.732 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.733 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.734 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.735 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:32:27.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.746 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.768 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:27.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.773 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.788 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:32:27.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:32:27.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T00:32:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T00:32:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:32:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.880 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T00:32:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.894 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T00:32:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T00:32:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T00:32:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:32:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T00:32:27.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T00:32:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T00:32:27.913 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.914 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:32:27.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T00:32:27.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T00:32:27.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.926 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.936 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.937 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.939 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T00:32:27.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.942 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.943 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.947 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.948 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T00:32:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T00:32:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.964 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.966 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.967 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.972 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T00:32:27.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:27.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:27.984 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T00:32:27.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T00:32:27.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:27.997 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:27.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:27.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:28.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T00:32:28.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T00:32:28.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T00:32:28.004 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:28.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:28.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.010 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T00:32:28.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T00:32:28.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T00:32:28.032 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T00:32:28.033 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T00:32:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T00:32:28.035 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.036 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 94, 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3320379688 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 912617528 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1775794839 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3876873433 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 862542721 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2719647839 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.040 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:28.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T00:32:28.041 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:27 smithi071 ceph-mon[32667]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3120688342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:27 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/999179475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:27 smithi122 ceph-mon[32959]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3120688342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:32:28.042 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:27 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/999179475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:32:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 msgr2=0x7fe2b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 0x7fe2b00a51c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005fd0 tx=0x7fe2b800b390 comp rx=0 tx=0).stop 2024-04-25T00:32:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 msgr2=0x7fe2c81994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009230 tx=0x7fe2bc004270 comp rx=0 tx=0).stop 2024-04-25T00:32:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 shutdown_connections 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe2b00a2d00 0x7fe2b00a51c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2c8106650 0x7fe2c81a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2c8101050 0x7fe2c81994e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 --2- 172.21.15.45:0/999179475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c8100690 0x7fe2c8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 >> 172.21.15.45:0/999179475 conn(0x7fe2c80777a0 msgr2=0x7fe2c8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 shutdown_connections 2024-04-25T00:32:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:32:26.763+0000 7fe2ce1b0640 1 -- 172.21.15.45:0/999179475 wait complete. 2024-04-25T00:32:28.049 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T00:32:28.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T00:32:28.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:27 smithi045 ceph-mon[29194]: pgmap v325: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:28.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3120688342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:32:28.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:27 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/999179475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:32:28.311 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:32:29.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:29 smithi045 ceph-mon[29194]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:29.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:29 smithi122 ceph-mon[32959]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:29.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:29 smithi071 ceph-mon[32667]: pgmap v326: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1498, 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1500, 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 448419, 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 476365, 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T00:32:30.060 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.135115884, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.046575016, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.033976067, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.041098311, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1498, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.051288454, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000034237 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 5, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 165, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1710, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 444158, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 7, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.013151366, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008464961, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002503677, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000069728, 2024-04-25T00:32:30.062 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 162, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017356330, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000107137 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 443136 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:30.063 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 246, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 233, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1479308, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 133870, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.047131366, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.014895424, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.037024283, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.004240111, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 233, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.012897362, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000055353 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.064 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1479040, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 133552 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T00:32:30.065 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2997, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.081759110, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000027280 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.066 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T00:32:30.067 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 1, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:30.068 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:30.069 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:30.070 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:30.071 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 2, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 2, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T00:32:30.073 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 4194892, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 4194352, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 2 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 3, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 3, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 0, 2024-04-25T00:32:30.074 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 1, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 1, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 52768, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215420 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:30.075 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:30.076 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.077 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.078 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T00:32:30.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T00:32:30.080 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.082 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:30.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:30.084 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.085 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.086 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 0, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 0, 2024-04-25T00:32:30.087 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:30.088 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:30.089 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:30.090 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 189473, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 53, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:30.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 181, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 8464, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 13, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:30.092 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 1502, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 21, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 3, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 1502, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1502, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.806987421, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000537275 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.093 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1502, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1503 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 1502, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1502, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 748, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 752, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2024-04-25T00:32:30.094 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T00:32:30.095 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T00:32:30.096 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 102, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 13, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 5, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 5, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T00:32:30.097 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T00:32:30.098 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 1745, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1757062, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:30.099 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 1745, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1745, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1757062, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:30.100 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1502, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 593, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1502, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 593, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.101 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 1502, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 1502, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1502, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1502, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-25T00:32:30.103 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x557965b2a6e0": { 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:30.104 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:30.105 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:30.855 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T00:32:30.855 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T00:32:30.855 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-25T00:32:30.855 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:30.855 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:30.855 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T00:32:30.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T00:32:31.113 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:32:31.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:31 smithi045 ceph-mon[29194]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:31.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:31 smithi122 ceph-mon[32959]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:31.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:31 smithi071 ceph-mon[32667]: pgmap v327: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:32.904 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2992, 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-25T00:32:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001750124, 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000039417, 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000612025, 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000708111, 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000236037 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 228, 2024-04-25T00:32:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 218, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1444736, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 133824, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.043283752, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.014488882, 2024-04-25T00:32:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.032362382, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.004314310, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 217, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.010500458, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000048389 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1444736, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 133824 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 165, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:32:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 184320, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.014300970, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010036936, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001726974, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 162, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.014443671, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000089158 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 184320 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T00:32:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148994, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 230, 2024-04-25T00:32:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1412580, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 230, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 230, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1412580, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:32.919 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:33.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:33 smithi045 ceph-mon[29194]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:33.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:33 smithi122 ceph-mon[32959]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:33.580 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T00:32:33.580 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T00:32:33.580 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T00:32:33.580 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T00:32:33.580 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T00:32:33.580 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T00:32:33.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T00:32:33.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:33 smithi071 ceph-mon[32667]: pgmap v328: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:33.834 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:32:35.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:35 smithi045 ceph-mon[29194]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 997, 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 985, 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1730026, 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 373838, 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T00:32:35.519 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.131199347, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.051403011, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.053076151, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.026142465, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 984, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.050023068, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000050836 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.520 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.521 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.521 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1476592, 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 134384 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1388, 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1549, 2024-04-25T00:32:35.522 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 354284, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 885733, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 7, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.106838705, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.035092606, 2024-04-25T00:32:35.523 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.026739134, 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.035715436, 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1546, 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.057325784, 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000037080 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.524 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:32:35.525 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 444928 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 8, 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 8, 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 4621, 2024-04-25T00:32:35.526 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 1732, 2024-04-25T00:32:35.527 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T00:32:35.527 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T00:32:35.527 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001851234, 2024-04-25T00:32:35.527 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000140172, 2024-04-25T00:32:35.527 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001537440, 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000109669, 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7, 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003153201, 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000450457 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.528 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T00:32:35.529 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:32:35.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.530 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T00:32:35.530 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T00:32:35.530 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T00:32:35.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3637, 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.090946698, 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000025005 2024-04-25T00:32:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.532 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:35.532 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T00:32:35.532 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T00:32:35.532 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2024-04-25T00:32:35.533 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.534 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T00:32:35.535 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2024-04-25T00:32:35.536 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 1, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T00:32:35.537 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:35.538 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:35.540 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-25T00:32:35.541 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:35.542 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 3, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T00:32:35.543 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 2, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 4195200, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 4194892, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 3 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 3, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 3, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 1, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 1, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2024-04-25T00:32:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 53064, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215420 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T00:32:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T00:32:35.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T00:32:35.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:35.549 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T00:32:35.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.551 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T00:32:35.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:35.553 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:35.554 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.555 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T00:32:35.556 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.557 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 1780, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.559 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:35.560 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:35.561 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 197473, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 55, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T00:32:35.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 181, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 8464, 2024-04-25T00:32:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 13, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 2143, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 21, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 3, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 2143, 2024-04-25T00:32:35.564 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2143, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.492976670, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000696676 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2143, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 2144 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 2143, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 2143, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 1383, 2024-04-25T00:32:35.565 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 758, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1, 2024-04-25T00:32:35.566 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T00:32:35.567 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 102, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 13, 2024-04-25T00:32:35.568 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 6, 2024-04-25T00:32:35.569 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 6, 2024-04-25T00:32:35.569 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T00:32:35.569 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T00:32:35.569 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2024-04-25T00:32:35.569 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:35.570 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:35.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 2389, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1858492, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 2389, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:35.572 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2389, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1858492, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:35.573 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:35.574 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 2143, 2024-04-25T00:32:35.574 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 901, 2024-04-25T00:32:35.574 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2143, 2024-04-25T00:32:35.574 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 901, 2024-04-25T00:32:35.574 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T00:32:35.575 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 2143, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 2143, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2143, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2143, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.576 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-25T00:32:35.577 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x5596cfd266e0": { 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T00:32:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:35.579 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:35.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:35 smithi122 ceph-mon[32959]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:32:35.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:35 smithi071 ceph-mon[32667]: pgmap v329: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T00:32:36.295 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T00:32:36.295 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T00:32:36.295 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T00:32:36.295 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:36.295 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:36.295 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T00:32:36.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T00:32:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:32:37.322 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:37 smithi045 ceph-mon[29194]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:37.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:37 smithi122 ceph-mon[32959]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:37.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:37 smithi071 ceph-mon[32667]: pgmap v330: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:38.210 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001483713, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000456949, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:38.211 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 234, 2024-04-25T00:32:38.212 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 222, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1448272, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 135552, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.044689300, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.014908862, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.032284506, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.004688817, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 222, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.012516033, 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000056378 2024-04-25T00:32:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1448272, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 135552 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 167, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 184672, 2024-04-25T00:32:38.214 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.015440098, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010672401, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002048391, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 164, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.012161480, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000074155 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.215 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 184672 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:38.216 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:38.217 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:38.218 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148977, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:38.219 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 232, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1412840, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 232, 2024-04-25T00:32:38.220 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 232, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1412840, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T00:32:38.221 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T00:32:38.909 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T00:32:38.909 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T00:32:38.909 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T00:32:38.909 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T00:32:38.909 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T00:32:38.909 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T00:32:38.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T00:32:39.162 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:32:39.283 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:39 smithi071 ceph-mon[32667]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:39.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:39 smithi045 ceph-mon[29194]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:39.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:39 smithi122 ceph-mon[32959]: pgmap v331: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:40.990 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-25T00:32:40.990 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:40.990 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2309, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1452, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 861816, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 1555831, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 24, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.253078852, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.063415350, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.089190682, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.084087698, 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:40.991 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1450, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.675851723, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000466104 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2696, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000512789, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000190 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 304000, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 85600 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.992 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1573, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 904, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2003784, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 524696, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 23, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.193638154, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.049070496, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.093821817, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.059482985, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 903, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.344914711, 2024-04-25T00:32:40.993 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000381965 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1139, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000200774, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000176 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1916304, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 270080 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1652, 2024-04-25T00:32:40.994 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1426, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 469149, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 2439944, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 152, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 9, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.204781727, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.060843593, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.096618771, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.038780152, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1424, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.194641398, 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000136686 2024-04-25T00:32:40.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2184, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000455565, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000208 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 45536, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 459008 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.996 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 592, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.166136532, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000280636 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 87, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002750194, 2024-04-25T00:32:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000031611 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 170, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 170, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 170, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.163239048, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000960229 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:40.998 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 12, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 39, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 2, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 37, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 36, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 3, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 3, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 2, 2024-04-25T00:32:40.999 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 236, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 227, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 234, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-04-25T00:32:41.000 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 234, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 146, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 20, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 126, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 88, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 1, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 16, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 36, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:41.001 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 104, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 84, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 2, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 27 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 8, 2024-04-25T00:32:41.002 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 2, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 6, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 4, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 20, 2024-04-25T00:32:41.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 225, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 225, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 2, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-04-25T00:32:41.004 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 5132601, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 188, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.367991333, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001957400 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 29, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 53, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 24, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 20, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 20, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 0, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 39, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 43, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 4, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 3, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 21, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 18, 2024-04-25T00:32:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 55144, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 231804 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 218, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 1, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 170, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 32, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 2, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-04-25T00:32:41.007 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.006774530, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000846816 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 108, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.021482563, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000198912 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.008 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-04-25T00:32:41.009 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000194352, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000194352 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.010 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 10, 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001527364, 2024-04-25T00:32:41.011 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000152736 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.015400485, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001925060 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-04-25T00:32:41.012 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 10, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.049965891, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004996589 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000206228, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000206228 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003999921, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001999960 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-04-25T00:32:41.013 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002819096, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001409548 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.012179604, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006089802 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.018377774, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004594443 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001634614, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000817307 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-04-25T00:32:41.016 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.021260376, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002657547 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007416250, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001854062 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-04-25T00:32:41.018 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 5, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 3, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 27, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 13, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 609, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:41.019 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:41.020 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 367261, 2024-04-25T00:32:41.021 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 156, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 704, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 8, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 181, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-25T00:32:41.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 133797, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 1189, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 1119, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 956683, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 29, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 1119, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1119, 2024-04-25T00:32:41.023 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 21.608617089, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.019310649 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1119, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1158 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 1119, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 276, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 843, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 11, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 22, 2024-04-25T00:32:41.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 190, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 30, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 583, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 254, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 22, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-04-25T00:32:41.025 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-04-25T00:32:41.026 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 102, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 18, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-04-25T00:32:41.027 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 38, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 8 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 65, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-04-25T00:32:41.028 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 3, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 71, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 4, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 5530, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 2740749, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 5530, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5530, 2024-04-25T00:32:41.029 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2740749, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:41.030 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 1119, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 957151, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1119, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 957151, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:41.031 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 1119, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 1119, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1119, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1119, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.032 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 4, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 468, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 4, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 468, 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:41.033 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x563dde5ba500": { 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 225, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 938297, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 225, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:41.034 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 188, 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 938297, 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:41.035 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-25T00:32:41.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:41 smithi045 ceph-mon[29194]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:41.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:41 smithi071 ceph-mon[32667]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:41.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:41 smithi122 ceph-mon[32959]: pgmap v332: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:41.681 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T00:32:41.681 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T00:32:41.681 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T00:32:41.681 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T00:32:41.681 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T00:32:41.681 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T00:32:41.681 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T00:32:41.920 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:43 smithi071 ceph-mon[32667]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:43.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:43 smithi045 ceph-mon[29194]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:43.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:43 smithi122 ceph-mon[32959]: pgmap v333: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1635, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1637, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1421794, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 519940, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.149603835, 2024-04-25T00:32:43.617 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.057413696, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.034664145, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.043812547, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1635, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.056224686, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000034388 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 10, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 174, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 4968, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 450826, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 7, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.011851943, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.006966224, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.002375177, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000072410, 2024-04-25T00:32:43.619 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 172, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.018612185, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000108210 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 449488 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.620 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1011, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1000, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1665635, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 380061, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 3, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.110427158, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.033425718, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.047728542, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.031355359, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 999, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.034179743, 2024-04-25T00:32:43.621 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000034213 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1478112, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 137744 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.622 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4025, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.101837050, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000025301 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:43.624 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 31, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 31, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 31, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 2, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-04-25T00:32:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-04-25T00:32:43.626 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-04-25T00:32:43.627 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-04-25T00:32:43.628 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:43.629 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 225, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-04-25T00:32:43.630 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 5132601, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 5132465, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 225 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 20, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 24, 2024-04-25T00:32:43.631 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 4, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 11, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 11, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 31, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 31, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 53680, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 215420 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:43.632 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.633 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:43.634 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:43.635 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-04-25T00:32:43.636 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.637 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.639 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.641 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-04-25T00:32:43.643 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 5, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 3, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 3, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 2353, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:43.644 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:43.645 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:43.646 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 189726, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 60, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 181, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:43.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 92229, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 760, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2401, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 26, 2024-04-25T00:32:43.648 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 4, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2401, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2401, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.925983501, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000385665 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2401, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2402 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2401, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2401, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-04-25T00:32:43.649 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 1522, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 876, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2, 2024-04-25T00:32:43.650 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-04-25T00:32:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 102, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 13, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 5, 2024-04-25T00:32:43.652 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 5, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.653 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2652, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 2837982, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:43.654 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2652, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2652, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2837982, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:43.655 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2401, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 938307, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2401, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 938307, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:43.656 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2401, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2401, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2401, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2401, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x562498f406e0": { 2024-04-25T00:32:43.658 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:43.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.660 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:43.660 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-25T00:32:44.527 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T00:32:44.527 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T00:32:44.527 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-25T00:32:44.527 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:44.527 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:44.527 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T00:32:44.528 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T00:32:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:32:45.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:45 smithi122 ceph-mon[32959]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:45.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:45 smithi071 ceph-mon[32667]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:45.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:45 smithi045 ceph-mon[29194]: pgmap v334: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 112, 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 114, 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 32096, 2024-04-25T00:32:46.492 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 546011, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 53, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.023050016, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.002962511, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.015685839, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000276625, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 112, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.017449874, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000155802 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 136, 2024-04-25T00:32:46.493 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000022789, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000167 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 343, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 329, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1517122, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 168510, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 7, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.052629623, 2024-04-25T00:32:46.494 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.016858284, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.041535317, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.005833139, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 328, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.028747696, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000087645 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 88, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000022863, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000259 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.495 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1492736, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 141744 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 444, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 767, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 130469, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 566096, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 10, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.072711252, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.035822254, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.029782014, 2024-04-25T00:32:46.496 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.004256462, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 765, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.048023386, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000062775 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1008, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000254651, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000252 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 449136 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-25T00:32:46.497 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 43, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000850679, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000019783 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:46.498 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000010461, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000005230 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:46.499 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 11, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 10, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 10, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 10, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 1, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-04-25T00:32:46.500 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-04-25T00:32:46.501 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:46.502 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 1, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-04-25T00:32:46.503 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:46.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 2, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 2, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 2, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-04-25T00:32:46.505 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 4194892, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001402724, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001402724 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 13, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 13, 2024-04-25T00:32:46.506 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 11, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 12, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 1, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 10, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 10, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 55104, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223612 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-04-25T00:32:46.507 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-04-25T00:32:46.509 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-04-25T00:32:46.511 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-04-25T00:32:46.512 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-04-25T00:32:46.513 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.514 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.515 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.517 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-04-25T00:32:46.518 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:46.520 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:46.521 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 221879, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 71, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 181, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:46.522 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 50664, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 198, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 35, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 10710, 2024-04-25T00:32:46.523 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 33, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 35, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 35, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.200996416, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005742754 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 35, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 57 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 35, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 0, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 35, 2024-04-25T00:32:46.524 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 1, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 5, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 6, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:46.525 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-04-25T00:32:46.526 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-04-25T00:32:46.527 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 102, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 13, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 8, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 8, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 14, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 1 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-04-25T00:32:46.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:46.529 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 895, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1581992, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 895, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 895, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1581992, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.530 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 35, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 10710, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 35, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 10710, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.531 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 35, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 35, 2024-04-25T00:32:46.532 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 35, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 35, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:46.533 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x56216193a500": { 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 588, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 588, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:46.535 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-25T00:32:47.232 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:47 smithi071 ceph-mon[32667]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:47.258 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T00:32:47.258 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T00:32:47.258 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T00:32:47.258 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:47.258 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:47.258 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T00:32:47.259 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T00:32:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:32:47.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:47 smithi122 ceph-mon[32959]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:47.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:47 smithi045 ceph-mon[29194]: pgmap v335: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 779, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 779, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 208644, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 246962, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:49.154 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.070674769, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.027562263, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.017181238, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.018825932, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 778, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.027020316, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000034730 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T00:32:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 937, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 926, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1647572, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 357979, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.108023796, 2024-04-25T00:32:49.156 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.033033970, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.048443493, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.029211210, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 925, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.034526872, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000037326 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1479040, 2024-04-25T00:32:49.157 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 140144 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 769, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 938, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 258032, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 695090, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.098288328, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.035486126, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.023472155, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.028477308, 2024-04-25T00:32:49.158 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 935, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.055885820, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000059770 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 451184 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-04-25T00:32:49.159 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3670, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.105587646, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000028770 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-25T00:32:49.160 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:49.161 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 1, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-04-25T00:32:49.162 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-04-25T00:32:49.163 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:49.164 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:49.165 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-04-25T00:32:49.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 7, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 2, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 2, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 4196347, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 4196206, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 7 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 3, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 3, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 1, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 1, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-04-25T00:32:49.168 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 51400, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 215420 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.169 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.171 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-04-25T00:32:49.172 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.173 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.174 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.176 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.177 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.178 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 3, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 2, 2024-04-25T00:32:49.179 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 2929, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:49.180 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:49.181 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 189410, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 52, 2024-04-25T00:32:49.182 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 181, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 8464, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 13, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:49.183 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2227, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 3, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2227, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2227, 2024-04-25T00:32:49.184 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.991982299, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000445434 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2227, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2228 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2227, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2227, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 1450, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 775, 2024-04-25T00:32:49.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-04-25T00:32:49.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-04-25T00:32:49.187 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 102, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 13, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 5, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 5, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-04-25T00:32:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:49.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2481, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1875061, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2481, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:49.190 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2481, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1875061, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:49.191 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2227, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2048, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2227, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2048, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:49.192 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2227, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2227, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2227, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2227, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:49.193 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-25T00:32:49.194 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x56090001e6e0": { 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-25T00:32:49.196 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-25T00:32:49.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:49 smithi122 ceph-mon[32959]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:49.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:49 smithi071 ceph-mon[32667]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:49.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:49 smithi045 ceph-mon[29194]: pgmap v336: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:50.045 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T00:32:50.046 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T00:32:50.046 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T00:32:50.046 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:50.046 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:50.046 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T00:32:50.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T00:32:50.295 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:32:50.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:50 smithi071 ceph-mon[32667]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:50.413 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:50 smithi122 ceph-mon[32959]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:50.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:50 smithi045 ceph-mon[29194]: pgmap v337: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 443, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 725, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 123588, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 563126, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 10, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.071158999, 2024-04-25T00:32:52.035 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.028697235, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.030682966, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005575578, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 727, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.067315345, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000092593 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 974, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000293887, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000301 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:32:52.036 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 322112 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 371, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 383, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1528588, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 236291, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 12, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.057797769, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.018432408, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.044797553, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.006182059, 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:52.037 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 382, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.042448666, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000111122 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 149, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000042121, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000282 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1502256, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 154720 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:52.038 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 167, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 206, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 56517, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 647800, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 52, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 7, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.031346279, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.006811400, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.023664296, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000162920, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 200, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.021234269, 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000106171 2024-04-25T00:32:52.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 262, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000052504, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000200 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 6576, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 133312 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-25T00:32:52.040 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 45, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.002260635, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000050236 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000008143, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000004071 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-25T00:32:52.041 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 11, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 7, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 7, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.003188054, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000455436 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 4, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 11, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-25T00:32:52.042 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 11, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 11, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 10, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 1, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 1, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 1, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 2, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 18, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 12, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 2, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:52.043 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 1000, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 1, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:52.044 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:52.045 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 1 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 3, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:52.046 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 7, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 7, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 2, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-25T00:32:52.048 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 4196347, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 0, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 6, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011246290, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001874381 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 0 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 14, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 14, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 0, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 13, 2024-04-25T00:32:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 14, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 1, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 11, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 12, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 1, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 1, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 4, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 3, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 53564, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 223612 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 16, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 11, 2024-04-25T00:32:52.050 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 16, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 1, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 5, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.002477402, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000495480 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.051 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000710652, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000355326 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.052 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.053 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.054 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.055 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.058 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:52.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-25T00:32:52.060 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 3, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 2, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 1, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 4, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 2, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 587, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-25T00:32:52.062 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:52.063 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:52.064 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 265047, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 106, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 176, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 2, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:52.065 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 181, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 56704, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 228, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.066 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 46, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 12670, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 33, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 46, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 46, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.278993800, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.006065082 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 46, 2024-04-25T00:32:52.067 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 70 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 46, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 46, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 10, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 6, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 9, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-25T00:32:52.068 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 6, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 20, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:52.069 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-25T00:32:52.070 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 102, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 13, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 8, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 8, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 17, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 0, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 2 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 1, 2024-04-25T00:32:52.071 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:52.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 977, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1602184, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 977, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 977, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1602184, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.073 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 46, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 12670, 2024-04-25T00:32:52.074 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 46, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 12670, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:52.075 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 46, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 46, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 46, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 46, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:52.076 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x5594ffd98500": { 2024-04-25T00:32:52.077 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 7, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 2043, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 7, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 6, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 2043, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:52.078 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:52.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:52.079 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-25T00:32:52.825 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T00:32:52.825 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T00:32:52.825 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T00:32:52.825 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:52.825 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:52.825 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T00:32:52.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T00:32:52.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:52 smithi045 ceph-mon[29194]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:53.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:52 smithi122 ceph-mon[32959]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:53.071 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:32:53.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:52 smithi071 ceph-mon[32667]: pgmap v338: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:54.819 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:54 smithi122 ceph-mon[32959]: pgmap v339: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:54.819 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-25T00:32:54.819 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:54.819 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 447, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 604, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 128423, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 223206, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 9, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.066599515, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.025788640, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.030513388, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005494681, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 609, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.044369720, 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000072856 2024-04-25T00:32:54.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 995, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000284470, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000285 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 65, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 268, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 11085, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 986022, 2024-04-25T00:32:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 57, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 8, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.036406327, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.010977921, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.021169748, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000271345, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 257, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.031391748, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000122146 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 88, 2024-04-25T00:32:54.822 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000016511, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000187 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 2224, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 459792 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 401, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 387, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1534239, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 194435, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 7, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:54.823 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.058902106, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.019105144, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.043591925, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.006701974, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 388, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.042162542, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000108666 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 180, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000041097, 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000228 2024-04-25T00:32:54.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1487392, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 144304 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 39, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.001598295, 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000040981 2024-04-25T00:32:54.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000009105, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000004552 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 1, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 1, 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-25T00:32:54.826 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000133450, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000133450 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 11, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 10, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-04-25T00:32:54.827 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 10, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 10, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 1, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 1, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 1, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-04-25T00:32:54.828 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:54.829 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-04-25T00:32:54.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 1, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:54.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-25T00:32:54.832 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 3, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 3, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 2, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-04-25T00:32:54.833 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 4195200, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 0, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.003266398, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001633199 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 0 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 13, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 13, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 0, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 11, 2024-04-25T00:32:54.834 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 12, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 1, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 10, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 10, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 0, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 1, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 1, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 51660, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 223612 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 1, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 1, 2024-04-25T00:32:54.835 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 9, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.836 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.837 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000133450, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000133450 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.838 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-25T00:32:54.839 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.840 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.840 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.840 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-25T00:32:54.840 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-25T00:32:54.841 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.842 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.844 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.845 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:54.846 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 1, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 0, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 1, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 1, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 1, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 645, 2024-04-25T00:32:54.847 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:54.848 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:54.849 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 254391, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 84, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:54.850 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 181, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 51032, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 200, 2024-04-25T00:32:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 37, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 11108, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 33, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 37, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 37, 2024-04-25T00:32:54.852 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.209995350, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.005675550 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 37, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 59 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 37, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 0, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 37, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 0, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 10, 2024-04-25T00:32:54.853 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 2, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 6, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 6, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 20, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:54.854 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-25T00:32:54.855 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 102, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 13, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 9, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 9, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-25T00:32:54.856 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 14, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 0, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 1 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 1, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:54.857 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 909, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1574247, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 909, 2024-04-25T00:32:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 909, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1574247, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:54.859 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 37, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 11108, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 37, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 11108, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:54.860 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 37, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 37, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 37, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 37, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-25T00:32:54.862 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x557dea424500": { 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 3, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 896, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 3, 2024-04-25T00:32:54.863 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 896, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:54.864 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-25T00:32:54.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:54 smithi071 ceph-mon[32667]: pgmap v339: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:54.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:54 smithi045 ceph-mon[29194]: pgmap v339: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T00:32:55.618 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T00:32:55.618 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T00:32:55.618 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-25T00:32:55.618 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:55.618 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:55.618 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T00:32:55.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T00:32:55.857 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:32:56.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:56 smithi045 ceph-mon[29194]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:32:57.000 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:56 smithi122 ceph-mon[32959]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:32:57.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:56 smithi071 ceph-mon[32667]: pgmap v340: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 139, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 146, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 51051, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 126965, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 19, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:57.575 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.019951854, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.005071299, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.010043358, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000596759, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 139, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.021396949, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000153934 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 205, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000080969, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000394 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.576 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 736 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 513, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 716, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 138340, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 813303, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 46, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 6, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.086678091, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.031140783, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.048145687, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005169931, 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:57.577 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 718, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.044292961, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000061689 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1119, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000338990, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000302 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 5360, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 144160 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 352, 2024-04-25T00:32:57.578 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 464, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1529896, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 522815, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 10, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.061423276, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.021713276, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.041081317, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.006258303, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 464, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.041109084, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000088597 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:32:57.579 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 105, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000038165, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000363 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1504832, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 466576 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:57.580 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 43, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.001594341, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000037077 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000006115, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000003057 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-25T00:32:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 6, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 4, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 4, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.001743040, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000435760 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 2, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 11, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-25T00:32:57.582 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 14, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 14, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 14, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 1, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 1, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 1, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 2, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 11, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 5, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 1, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 2, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-25T00:32:57.583 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 1000, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 1, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:32:57.584 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 2, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 3, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 2, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 1 2024-04-25T00:32:57.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 3, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-25T00:32:57.586 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-25T00:32:57.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 7, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 7, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 2, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-25T00:32:57.588 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 4196264, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 0, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 6, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.010186453, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001697742 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 0 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 17, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 17, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 0, 2024-04-25T00:32:57.589 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 15, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 16, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 1, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 14, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 15, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 1, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 1, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 3, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 2, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 55112, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 223612 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 8, 2024-04-25T00:32:57.590 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 6, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 17, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.001203479, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000601739 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.591 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000539561, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000269780 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.592 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.593 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.596 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.597 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-25T00:32:57.598 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.599 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.601 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 3, 2024-04-25T00:32:57.602 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 2, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 1, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 3, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 1, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 659, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:32:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:32:57.604 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:32:57.605 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 279975, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 103, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9424, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 181, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:32:57.606 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 67829, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 296, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 47, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 12941, 2024-04-25T00:32:57.607 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 28, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 47, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 47, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.225994989, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004808404 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 47, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 73 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 47, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 0, 2024-04-25T00:32:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 47, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 10, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 6, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 8, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 6, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-25T00:32:57.609 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 20, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-25T00:32:57.610 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-25T00:32:57.611 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 102, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 13, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 9, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 9, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 22, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 1 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 1, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-25T00:32:57.612 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:32:57.613 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 1000, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1610986, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 1000, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 1000, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1610986, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.614 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 47, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 12941, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 47, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 12941, 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 47, 2024-04-25T00:32:57.616 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 47, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 47, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 47, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:32:57.617 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x5604872f2500": { 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:32:57.618 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 7, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1960, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 7, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 6, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1960, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:32:57.620 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-25T00:32:58.406 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T00:32:58.406 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T00:32:58.406 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T00:32:58.407 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:32:58.407 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:32:58.407 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T00:32:58.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T00:32:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:32:58.898 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:32:58 smithi122 ceph-mon[32959]: pgmap v341: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:58.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:32:58 smithi045 ceph-mon[29194]: pgmap v341: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:32:59.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:32:58 smithi071 ceph-mon[32667]: pgmap v341: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:33:00.412 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-25T00:33:00.412 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:33:00.412 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2261, 2024-04-25T00:33:00.412 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2261, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 643265, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 718181, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.198513256, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.066598806, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.047359433, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.068924488, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2260, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.087892466, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000038890 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.413 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 258, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 247, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1480684, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 143402, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 3, 2024-04-25T00:33:00.414 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.046631654, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.014701027, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.034162138, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.005684629, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 246, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.013029750, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000052966 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.415 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1480128, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 142704 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 3, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 174, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 588, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 453470, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.010386456, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.006836373, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001463552, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000022786, 2024-04-25T00:33:00.416 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 171, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.013527214, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000079106 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 453152 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-25T00:33:00.417 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 3819, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.110194114, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000028854 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.418 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-25T00:33:00.419 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-04-25T00:33:00.420 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 1, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:33:00.421 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:33:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:33:00.423 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-25T00:33:00.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 7, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-25T00:33:00.425 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 4196264, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 4196123, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 7 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 3, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 3, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 0, 2024-04-25T00:33:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 1, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 1, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 50260, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215420 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-04-25T00:33:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.428 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-25T00:33:00.429 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-25T00:33:00.430 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-25T00:33:00.431 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.432 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.433 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.434 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.436 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.437 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-25T00:33:00.438 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 3, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 2, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 2, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 2934, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:33:00.439 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:33:00.440 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:33:00.441 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 182071, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 47, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9424, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 181, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 8464, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 13, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 2260, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 21, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 3, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 2260, 2024-04-25T00:33:00.443 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2260, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.986978039, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000436715 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2260, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 2261 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 2260, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 2260, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 1472, 2024-04-25T00:33:00.444 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 786, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 1, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-25T00:33:00.445 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-25T00:33:00.446 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 102, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 13, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 4, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 4, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-25T00:33:00.447 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-25T00:33:00.448 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 2518, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1881812, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 2518, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:33:00.449 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2518, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1881812, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:33:00.450 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2260, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 1965, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2260, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1965, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:33:00.451 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 2260, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 2260, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 2260, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 2260, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:33:00.452 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x5603e79ee6e0": { 2024-04-25T00:33:00.453 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-25T00:33:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-25T00:33:00.455 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-25T00:33:00.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:00 smithi122 ceph-mon[32959]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:33:00.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:00 smithi045 ceph-mon[29194]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:33:01.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:00 smithi071 ceph-mon[32667]: pgmap v342: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T00:33:01.227 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T00:33:01.227 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T00:33:01.227 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-25T00:33:01.227 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:33:01.227 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:33:01.228 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T00:33:01.239 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:33:01.239 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:33:01.239 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:01.261 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T00:33:01.261 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:33:01.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 -- 172.21.15.45:0/2114594256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 msgr2=0x7fbadc10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 --2- 172.21.15.45:0/2114594256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc10b7d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009900 tx=0x7fbad002a510 comp rx=0 tx=0).stop 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 -- 172.21.15.45:0/2114594256 shutdown_connections 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 --2- 172.21.15.45:0/2114594256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 --2- 172.21.15.45:0/2114594256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc10b7d0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 --2- 172.21.15.45:0/2114594256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 0x7fbadc102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 -- 172.21.15.45:0/2114594256 >> 172.21.15.45:0/2114594256 conn(0x7fbadc077740 msgr2=0x7fbadc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 -- 172.21.15.45:0/2114594256 shutdown_connections 2024-04-25T00:33:01.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.474+0000 7fbae197e640 1 -- 172.21.15.45:0/2114594256 wait complete. 2024-04-25T00:33:01.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 Processor -- start 2024-04-25T00:33:01.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 -- start start 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 0x7fbadc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111b10 con 0x7fbadc1032f0 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111990 con 0x7fbadc1029d0 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbae197e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc111c90 con 0x7fbadc10bd10 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbada7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbada7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49220/0 (socket says 172.21.15.45:49220) 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbada7fc640 1 -- 172.21.15.45:0/3924331644 learned_addr learned my addr 172.21.15.45:0/3924331644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbadaffd640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 0x7fbadc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.475+0000 7fbadb7fe640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 -- 172.21.15.45:0/3924331644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 msgr2=0x7fbadc19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 -- 172.21.15.45:0/3924331644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 msgr2=0x7fbadc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 0x7fbadc198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 -- 172.21.15.45:0/3924331644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc19cff0 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbadb7fe640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbada7fc640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006010 tx=0x7fbad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad003f030 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc19d280 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad0003ce0 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0035070 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.476+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc1a4b70 con 0x7fbadc1032f0 2024-04-25T00:33:01.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.477+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc068560 con 0x7fbadc1032f0 2024-04-25T00:33:01.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.478+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbad0042030 con 0x7fbadc1032f0 2024-04-25T00:33:01.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.479+0000 7fbae097c640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 0x7fbac40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.479+0000 7fbadaffd640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 0x7fbac40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.479+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fbad00efff0 con 0x7fbadc1032f0 2024-04-25T00:33:01.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.480+0000 7fbadaffd640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 0x7fbac40a5240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc009680 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.482+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbad00bc910 con 0x7fbadc1032f0 2024-04-25T00:33:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.668+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbadc199350 con 0x7fbadc1032f0 2024-04-25T00:33:01.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.670+0000 7fbae097c640 1 -- 172.21.15.45:0/3924331644 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+25272 (secure 0 0 0) 0x7fbadc199350 con 0x7fbadc1032f0 2024-04-25T00:33:01.671 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:33:01.671 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":102,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:32:12.003289+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 msgr2=0x7fbac40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 0x7fbac40a5240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc009680 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 msgr2=0x7fbadc198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006010 tx=0x7fbad00042f0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 shutdown_connections 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fbac40a2d80 0x7fbac40a5240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbadc10bd10 0x7fbadc19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbadc1032f0 0x7fbadc198590 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 --2- 172.21.15.45:0/3924331644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbadc1029d0 0x7fbadc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 >> 172.21.15.45:0/3924331644 conn(0x7fbadc077740 msgr2=0x7fbadc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 shutdown_connections 2024-04-25T00:33:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.672+0000 7fbae197e640 1 -- 172.21.15.45:0/3924331644 wait complete. 2024-04-25T00:33:01.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:14:51.716786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:19:31.801167+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:19:32.608231+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:19:36.319099+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:33:01.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:33:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2959644763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100fd0 msgr2=0x7f9b7810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2959644763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100fd0 0x7f9b7810d4e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009cd0 tx=0x7f9b7002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2959644763 shutdown_connections 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2959644763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100fd0 0x7f9b7810d4e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2959644763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b78100610 0x7f9b78100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2959644763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b781066b0 0x7f9b78106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.788+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2959644763 >> 172.21.15.45:0/2959644763 conn(0x7f9b780769c0 msgr2=0x7f9b78076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2959644763 shutdown_connections 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2959644763 wait complete. 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 Processor -- start 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- start start 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b78100fd0 0x7f9b781027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b781066b0 0x7f9b78102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78110a30 con 0x7f9b78100610 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781108b0 con 0x7f9b78100fd0 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.789+0000 7f9b7c91b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78110bb0 con 0x7f9b781066b0 2024-04-25T00:33:01.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49238/0 (socket says 172.21.15.45:49238) 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 -- 172.21.15.45:0/2616165763 learned_addr learned my addr 172.21.15.45:0/2616165763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b75d9b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b78100fd0 0x7f9b781027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b76d9d640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b781066b0 0x7f9b78102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 -- 172.21.15.45:0/2616165763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b781066b0 msgr2=0x7f9b78102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b781066b0 0x7f9b78102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 -- 172.21.15.45:0/2616165763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b78100fd0 msgr2=0x7f9b781027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b78100fd0 0x7f9b781027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 -- 172.21.15.45:0/2616165763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b781a6440 con 0x7f9b78100610 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7659c640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800d770 tx=0x7f9b6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68017070 con 0x7f9b78100610 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b781a6730 con 0x7f9b78100610 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b68009c30 con 0x7f9b78100610 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b781a6c80 con 0x7f9b78100610 2024-04-25T00:33:01.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.790+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68020060 con 0x7f9b78100610 2024-04-25T00:33:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.791+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b78101d50 con 0x7f9b78100610 2024-04-25T00:33:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.793+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9b68003d70 con 0x7f9b78100610 2024-04-25T00:33:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.793+0000 7f9b677fe640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 0x7f9b600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.793+0000 7f9b75d9b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 0x7f9b600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.793+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f9b680cf490 con 0x7f9b78100610 2024-04-25T00:33:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.794+0000 7f9b75d9b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 0x7f9b600a51c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b78103d40 tx=0x7f9b6c005f90 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:01.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.798+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9b68009de0 con 0x7f9b78100610 2024-04-25T00:33:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.985+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b781a6f30 con 0x7f9b78100610 2024-04-25T00:33:01.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.985+0000 7f9b677fe640 1 -- 172.21.15.45:0/2616165763 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f9b6809bcf0 con 0x7f9b78100610 2024-04-25T00:33:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 msgr2=0x7f9b600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 0x7f9b600a51c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b78103d40 tx=0x7f9b6c005f90 comp rx=0 tx=0).stop 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 msgr2=0x7f9b781022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800d770 tx=0x7f9b6800dc30 comp rx=0 tx=0).stop 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 shutdown_connections 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f9b600a2d00 0x7f9b600a51c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b781066b0 0x7f9b78102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b78100fd0 0x7f9b781027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 --2- 172.21.15.45:0/2616165763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b78100610 0x7f9b781022b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 >> 172.21.15.45:0/2616165763 conn(0x7f9b780769c0 msgr2=0x7f9b780ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 shutdown_connections 2024-04-25T00:33:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:01.987+0000 7f9b7c91b640 1 -- 172.21.15.45:0/2616165763 wait complete. 2024-04-25T00:33:01.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:33:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 -- 172.21.15.45:0/353216214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 msgr2=0x7fc3e8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/353216214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e8106a10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009930 tx=0x7fc3d802a420 comp rx=0 tx=0).stop 2024-04-25T00:33:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 -- 172.21.15.45:0/353216214 shutdown_connections 2024-04-25T00:33:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/353216214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100fe0 0x7fc3e810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/353216214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100620 0x7fc3e8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/353216214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e8106a10 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 -- 172.21.15.45:0/353216214 >> 172.21.15.45:0/353216214 conn(0x7fc3e80776a0 msgr2=0x7fc3e8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 -- 172.21.15.45:0/353216214 shutdown_connections 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.103+0000 7fc3ee42a640 1 -- 172.21.15.45:0/353216214 wait complete. 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 Processor -- start 2024-04-25T00:33:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 -- start start 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100620 0x7fc3e81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100fe0 0x7fc3e8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8110a00 con 0x7fc3e8106630 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8110880 con 0x7fc3e8100fe0 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ee42a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8110b80 con 0x7fc3e8100620 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ec9c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ec9c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49268/0 (socket says 172.21.15.45:49268) 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3ec9c7640 1 -- 172.21.15.45:0/1444760787 learned_addr learned my addr 172.21.15.45:0/1444760787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3e77fe640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100fe0 0x7fc3e8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.104+0000 7fc3e7fff640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100620 0x7fc3e81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 -- 172.21.15.45:0/1444760787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100620 msgr2=0x7fc3e81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100620 0x7fc3e81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 -- 172.21.15.45:0/1444760787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100fe0 msgr2=0x7fc3e8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100fe0 0x7fc3e8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 -- 172.21.15.45:0/1444760787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819d0c0 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ec9c7640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000eb10 tx=0x7fc3e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0004b20 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e0004cc0 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819d3b0 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0010760 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.105+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a4c40 con 0x7fc3e8106630 2024-04-25T00:33:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.106+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e80689b0 con 0x7fc3e8106630 2024-04-25T00:33:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.107+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc3e0010900 con 0x7fc3e8106630 2024-04-25T00:33:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.108+0000 7fc3e57fa640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 0x7fc3d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.108+0000 7fc3e7fff640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 0x7fc3d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.108+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fc3e00cffe0 con 0x7fc3e8106630 2024-04-25T00:33:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.108+0000 7fc3e7fff640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 0x7fc3d00a51c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8005a20 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.111+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3e0053740 con 0x7fc3e8106630 2024-04-25T00:33:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.295+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3e8109bc0 con 0x7fc3e8106630 2024-04-25T00:33:02.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.296+0000 7fc3e57fa640 1 -- 172.21.15.45:0/1444760787 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7fc3e009c840 con 0x7fc3e8106630 2024-04-25T00:33:02.296 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:33:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.297+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 msgr2=0x7fc3d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.297+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 0x7fc3d00a51c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8000c00 tx=0x7fc3d8005a20 comp rx=0 tx=0).stop 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.297+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 msgr2=0x7fc3e819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.297+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000eb10 tx=0x7fc3e000c2d0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 shutdown_connections 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fc3d00a2d00 0x7fc3d00a51c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8106630 0x7fc3e819c990 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3e8100fe0 0x7fc3e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 --2- 172.21.15.45:0/1444760787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3e8100620 0x7fc3e81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 >> 172.21.15.45:0/1444760787 conn(0x7fc3e80776a0 msgr2=0x7fc3e806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 shutdown_connections 2024-04-25T00:33:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.298+0000 7fc3ee42a640 1 -- 172.21.15.45:0/1444760787 wait complete. 2024-04-25T00:33:02.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:33:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 -- 172.21.15.45:0/2038778142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 msgr2=0x7f34701104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 --2- 172.21.15.45:0/2038778142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f34701104c0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ca0 tx=0x7f346802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 -- 172.21.15.45:0/2038778142 shutdown_connections 2024-04-25T00:33:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 --2- 172.21.15.45:0/2038778142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f34701104c0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 --2- 172.21.15.45:0/2038778142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100e40 0x7f347010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 --2- 172.21.15.45:0/2038778142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100520 0x7f3470100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 -- 172.21.15.45:0/2038778142 >> 172.21.15.45:0/2038778142 conn(0x7f34700769c0 msgr2=0x7f3470076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 -- 172.21.15.45:0/2038778142 shutdown_connections 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.414+0000 7f3475eda640 1 -- 172.21.15.45:0/2038778142 wait complete. 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 Processor -- start 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 -- start start 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 0x7f347019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 0x7f347019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3470113d30 con 0x7f347010e0d0 2024-04-25T00:33:02.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3470113bb0 con 0x7f3470100e40 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f3475eda640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3470113eb0 con 0x7f3470100520 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49286/0 (socket says 172.21.15.45:49286) 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 -- 172.21.15.45:0/2918195266 learned_addr learned my addr 172.21.15.45:0/2918195266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346f7fe640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 0x7f347019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346effd640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 0x7f347019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 -- 172.21.15.45:0/2918195266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 msgr2=0x7f347019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 0x7f347019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 -- 172.21.15.45:0/2918195266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 msgr2=0x7f347019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 0x7f347019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346ffff640 1 -- 172.21.15.45:0/2918195266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347019f1e0 con 0x7f347010e0d0 2024-04-25T00:33:02.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346effd640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 0x7f347019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.415+0000 7f346f7fe640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 0x7f347019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f346ffff640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f346802ade0 tx=0x7f34680319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468003c30 con 0x7f347010e0d0 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3468003dd0 con 0x7f347010e0d0 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346802fd60 con 0x7f347010e0d0 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347019f470 con 0x7f347010e0d0 2024-04-25T00:33:02.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.416+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701a6d60 con 0x7f347010e0d0 2024-04-25T00:33:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.417+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347006a5a0 con 0x7f347010e0d0 2024-04-25T00:33:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.418+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3468004050 con 0x7f347010e0d0 2024-04-25T00:33:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.419+0000 7f346cff9640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 0x7f34580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.419+0000 7f346f7fe640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 0x7f34580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.419+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f34680f0ba0 con 0x7f347010e0d0 2024-04-25T00:33:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.419+0000 7f346f7fe640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 0x7f34580a51c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3460005fd0 tx=0x7f346000b5f0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.424+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34680bd5a0 con 0x7f347010e0d0 2024-04-25T00:33:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.609+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f347019b540 con 0x7f347010e0d0 2024-04-25T00:33:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.610+0000 7f346cff9640 1 -- 172.21.15.45:0/2918195266 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f34680bcea0 con 0x7f347010e0d0 2024-04-25T00:33:02.611 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 msgr2=0x7f34580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 0x7f34580a51c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3460005fd0 tx=0x7f346000b5f0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 msgr2=0x7f347019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f346802ade0 tx=0x7f34680319f0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 shutdown_connections 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34580a2d00 0x7f34580a51c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f347010e0d0 0x7f347019eab0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3470100e40 0x7f347019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 --2- 172.21.15.45:0/2918195266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470100520 0x7f347019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 >> 172.21.15.45:0/2918195266 conn(0x7f34700769c0 msgr2=0x7f347010f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 shutdown_connections 2024-04-25T00:33:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.612+0000 7f3475eda640 1 -- 172.21.15.45:0/2918195266 wait complete. 2024-04-25T00:33:02.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:33:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 -- 172.21.15.45:0/1642732650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0107720 msgr2=0x7fe3a0107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 --2- 172.21.15.45:0/1642732650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0107720 0x7fe3a0107b00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009930 tx=0x7fe39c02a420 comp rx=0 tx=0).stop 2024-04-25T00:33:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 -- 172.21.15.45:0/1642732650 shutdown_connections 2024-04-25T00:33:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 --2- 172.21.15.45:0/1642732650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 --2- 172.21.15.45:0/1642732650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0101710 0x7fe3a0101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 --2- 172.21.15.45:0/1642732650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0107720 0x7fe3a0107b00 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 -- 172.21.15.45:0/1642732650 >> 172.21.15.45:0/1642732650 conn(0x7fe3a00fd180 msgr2=0x7fe3a00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 -- 172.21.15.45:0/1642732650 shutdown_connections 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.729+0000 7fe3a7410640 1 -- 172.21.15.45:0/1642732650 wait complete. 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 Processor -- start 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 -- start start 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 0x7fe3a019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a0111b30 con 0x7fe3a0101710 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01119b0 con 0x7fe3a0107720 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a7410640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a0111cb0 con 0x7fe3a01020d0 2024-04-25T00:33:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49302/0 (socket says 172.21.15.45:49302) 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 -- 172.21.15.45:0/3661623344 learned_addr learned my addr 172.21.15.45:0/3661623344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a49ab640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a59ad640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 0x7fe3a019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 -- 172.21.15.45:0/3661623344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 msgr2=0x7fe3a01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 -- 172.21.15.45:0/3661623344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 msgr2=0x7fe3a019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 0x7fe3a019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 -- 172.21.15.45:0/3661623344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a019d050 con 0x7fe3a0101710 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a49ab640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a59ad640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 0x7fe3a019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:02.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.730+0000 7fe3a51ac640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009230 tx=0x7fe39c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.731+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c03f030 con 0x7fe3a0101710 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.731+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a019d2e0 con 0x7fe3a0101710 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.731+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c003ad0 con 0x7fe3a0101710 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.731+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c035070 con 0x7fe3a0101710 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.731+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a01a4bd0 con 0x7fe3a0101710 2024-04-25T00:33:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.732+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a00689b0 con 0x7fe3a0101710 2024-04-25T00:33:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.733+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe39c042030 con 0x7fe3a0101710 2024-04-25T00:33:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.733+0000 7fe3967fc640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 0x7fe38c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.733+0000 7fe3a49ab640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 0x7fe38c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.733+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fe39c0f0180 con 0x7fe3a0101710 2024-04-25T00:33:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.734+0000 7fe3a49ab640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 0x7fe38c0a51c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00ff7f0 tx=0x7fe390005ec0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.737+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe39c0bc9e0 con 0x7fe3a0101710 2024-04-25T00:33:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:02 smithi045 ceph-mon[29194]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:33:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3924331644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2616165763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:33:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1444760787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:33:02.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:02 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2918195266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:33:02.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.920+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe3a0109c80 con 0x7fe3a0101710 2024-04-25T00:33:02.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.920+0000 7fe3967fc640 1 -- 172.21.15.45:0/3661623344 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) ==== 103+0+11 (secure 0 0 0) 0x7fe39c0bc2e0 con 0x7fe3a0101710 2024-04-25T00:33:02.921 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 msgr2=0x7fe38c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 0x7fe38c0a51c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00ff7f0 tx=0x7fe390005ec0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 msgr2=0x7fe3a01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009230 tx=0x7fe39c004270 comp rx=0 tx=0).stop 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 shutdown_connections 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe38c0a2d00 0x7fe38c0a51c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a0107720 0x7fe3a019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3a01020d0 0x7fe3a01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 --2- 172.21.15.45:0/3661623344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a0101710 0x7fe3a01980b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 >> 172.21.15.45:0/3661623344 conn(0x7fe3a00fd180 msgr2=0x7fe3a00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 shutdown_connections 2024-04-25T00:33:02.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:02.922+0000 7fe3a7410640 1 -- 172.21.15.45:0/3661623344 wait complete. 2024-04-25T00:33:02.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 -- 172.21.15.45:0/2662598218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 msgr2=0x7fe948107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 --2- 172.21.15.45:0/2662598218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe948107b00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe940009cd0 tx=0x7fe94002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 -- 172.21.15.45:0/2662598218 shutdown_connections 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 --2- 172.21.15.45:0/2662598218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe94810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 --2- 172.21.15.45:0/2662598218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 0x7fe948101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 --2- 172.21.15.45:0/2662598218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe948107b00 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 -- 172.21.15.45:0/2662598218 >> 172.21.15.45:0/2662598218 conn(0x7fe9480fd180 msgr2=0x7fe9480ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 -- 172.21.15.45:0/2662598218 shutdown_connections 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 -- 172.21.15.45:0/2662598218 wait complete. 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.037+0000 7fe9501e7640 1 Processor -- start 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 -- start start 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 0x7fe948073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe948073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe948111b80 con 0x7fe948107720 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe948111a00 con 0x7fe9481020d0 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe9501e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe948111d00 con 0x7fe948101710 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49316/0 (socket says 172.21.15.45:49316) 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 -- 172.21.15.45:0/163536852 learned_addr learned my addr 172.21.15.45:0/163536852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94df83640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 0x7fe948073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94d782640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe948073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 -- 172.21.15.45:0/163536852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 msgr2=0x7fe948073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 0x7fe948073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 -- 172.21.15.45:0/163536852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 msgr2=0x7fe948073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe948073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 -- 172.21.15.45:0/163536852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9481a6540 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94d782640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe948073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe94e784640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d750 tx=0x7fe93800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.038+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017070 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.039+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9481a6830 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.039+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938005500 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.039+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938004d90 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.039+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481a6d80 con 0x7fe948107720 2024-04-25T00:33:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.040+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9480689b0 con 0x7fe948107720 2024-04-25T00:33:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.041+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe938004f30 con 0x7fe948107720 2024-04-25T00:33:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.041+0000 7fe946ffd640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 0x7fe9300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.042+0000 7fe94df83640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 0x7fe9300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.042+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7fe9380cf630 con 0x7fe948107720 2024-04-25T00:33:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.042+0000 7fe94df83640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 0x7fe9300a5240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe940000f80 tx=0x7fe940031550 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.045+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe93809beb0 con 0x7fe948107720 2024-04-25T00:33:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:02 smithi122 ceph-mon[32959]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:33:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3924331644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2616165763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1444760787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:02 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2918195266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:02 smithi071 ceph-mon[32667]: pgmap v343: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:33:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3924331644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2616165763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1444760787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:02 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2918195266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.241+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe94802d170 con 0x7fe948107720 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.241+0000 7fe946ffd640 1 -- 172.21.15.45:0/163536852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+943 (secure 0 0 0) 0x7fe93809b7b0 con 0x7fe948107720 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3305016128 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/937209566 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1588770344 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:33:03.242 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3305016128 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2421422338 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1299966585 2024-04-26T00:12:25.175459+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3254971976 2024-04-26T00:12:25.175459+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3329362383 2024-04-26T00:21:57.361646+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1950515838 2024-04-26T00:11:58.120353+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2906156154 2024-04-26T00:11:58.120353+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3376575480 2024-04-26T00:12:25.175459+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/200010672 2024-04-26T00:11:58.120353+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2002798989 2024-04-26T00:11:58.120353+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2389946368 2024-04-26T00:12:25.175459+0000 2024-04-25T00:33:03.243 INFO:teuthology.orchestra.run.smithi045.stderr:listed 16 entries 2024-04-25T00:33:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.243+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 msgr2=0x7fe9300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.243+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 0x7fe9300a5240 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe940000f80 tx=0x7fe940031550 comp rx=0 tx=0).stop 2024-04-25T00:33:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.243+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 msgr2=0x7fe9480744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.243+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d750 tx=0x7fe93800dc10 comp rx=0 tx=0).stop 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 shutdown_connections 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7fe9300a2d80 0x7fe9300a5240 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe948107720 0x7fe9480744c0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9481020d0 0x7fe948073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 --2- 172.21.15.45:0/163536852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe948101710 0x7fe948073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 >> 172.21.15.45:0/163536852 conn(0x7fe9480fd180 msgr2=0x7fe9480fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 shutdown_connections 2024-04-25T00:33:03.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.244+0000 7fe9501e7640 1 -- 172.21.15.45:0/163536852 wait complete. 2024-04-25T00:33:03.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:33:03.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:03.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:33:03.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:33:03.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:03.295 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:33:03.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:03.351 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T00:33:03.407 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:33:03.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:03.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:33:03.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:33:03.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T00:33:03.565 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T00:33:03.575 INFO:tasks.cephadm:Teardown begin 2024-04-25T00:33:03.575 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:33:03.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:03.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.678+0000 7f5ee4514640 1 -- 172.21.15.45:0/2319001487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 msgr2=0x7f5edc10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.678+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2319001487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc10d510 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009cd0 tx=0x7f5ed402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- 172.21.15.45:0/2319001487 shutdown_connections 2024-04-25T00:33:03.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2319001487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc10d510 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2319001487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc100690 0x7f5edc100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2319001487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc106650 0x7f5edc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- 172.21.15.45:0/2319001487 >> 172.21.15.45:0/2319001487 conn(0x7f5edc0777a0 msgr2=0x7f5edc077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- 172.21.15.45:0/2319001487 shutdown_connections 2024-04-25T00:33:03.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- 172.21.15.45:0/2319001487 wait complete. 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 Processor -- start 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- start start 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc100690 0x7f5edc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc106650 0x7f5edc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc110a60 con 0x7f5edc101050 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc1108e0 con 0x7f5edc106650 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.679+0000 7f5ee4514640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc110be0 con 0x7f5edc100690 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49332/0 (socket says 172.21.15.45:49332) 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 -- 172.21.15.45:0/2623718708 learned_addr learned my addr 172.21.15.45:0/2623718708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee22b0640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc100690 0x7f5edc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee2ab1640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc106650 0x7f5edc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 -- 172.21.15.45:0/2623718708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc100690 msgr2=0x7f5edc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc100690 0x7f5edc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 -- 172.21.15.45:0/2623718708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc106650 msgr2=0x7f5edc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc106650 0x7f5edc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 -- 172.21.15.45:0/2623718708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc19cf40 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee1aaf640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c920 tx=0x7f5ecc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc007c70 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ecc007e10 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc19d230 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc018890 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.680+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc1a4ac0 con 0x7f5edc101050 2024-04-25T00:33:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.681+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc0689b0 con 0x7f5edc101050 2024-04-25T00:33:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.682+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5ecc018a70 con 0x7f5edc101050 2024-04-25T00:33:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.683+0000 7f5ed37fe640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 0x7f5ec80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.683+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f5ecc0cf8b0 con 0x7f5edc101050 2024-04-25T00:33:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.683+0000 7f5ee22b0640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 0x7f5ec80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.683+0000 7f5ee22b0640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 0x7f5ec80a51c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80058c0 tx=0x7f5ed800a250 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:03.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.688+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ecc014030 con 0x7f5edc101050 2024-04-25T00:33:03.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3661623344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:33:03.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:03 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/163536852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:33:03.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.911+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5edc1a4dc0 con 0x7f5edc101050 2024-04-25T00:33:03.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.912+0000 7f5ed37fe640 1 -- 172.21.15.45:0/2623718708 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f5ecc09c110 con 0x7f5edc101050 2024-04-25T00:33:03.913 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:33:03.913 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:33:03.913 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 msgr2=0x7f5ec80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 0x7f5ec80a51c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80058c0 tx=0x7f5ed800a250 comp rx=0 tx=0).stop 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 msgr2=0x7f5edc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c920 tx=0x7f5ecc00cde0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 shutdown_connections 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f5ec80a2d00 0x7f5ec80a51c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5edc106650 0x7f5edc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5edc101050 0x7f5edc198490 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 --2- 172.21.15.45:0/2623718708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc100690 0x7f5edc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 >> 172.21.15.45:0/2623718708 conn(0x7f5edc0777a0 msgr2=0x7f5edc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 shutdown_connections 2024-04-25T00:33:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:03.914+0000 7f5ee4514640 1 -- 172.21.15.45:0/2623718708 wait complete. 2024-04-25T00:33:03.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:33:04.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 -- 172.21.15.45:0/2824757634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 msgr2=0x7f00ac10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 --2- 172.21.15.45:0/2824757634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac10d510 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009cd0 tx=0x7f00a402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 -- 172.21.15.45:0/2824757634 shutdown_connections 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 --2- 172.21.15.45:0/2824757634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac10d510 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 --2- 172.21.15.45:0/2824757634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac100690 0x7f00ac100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 --2- 172.21.15.45:0/2824757634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac106650 0x7f00ac106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 -- 172.21.15.45:0/2824757634 >> 172.21.15.45:0/2824757634 conn(0x7f00ac0777a0 msgr2=0x7f00ac077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 -- 172.21.15.45:0/2824757634 shutdown_connections 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.028+0000 7f00b191b640 1 -- 172.21.15.45:0/2824757634 wait complete. 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 Processor -- start 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 -- start start 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac100690 0x7f00ac198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac106650 0x7f00ac1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac110a20 con 0x7f00ac101050 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1108a0 con 0x7f00ac106650 2024-04-25T00:33:04.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00b191b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac110ba0 con 0x7f00ac100690 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34184/0 (socket says 172.21.15.45:34184) 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 -- 172.21.15.45:0/1776763372 learned_addr learned my addr 172.21.15.45:0/1776763372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00ab7fe640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac106650 0x7f00ac1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00aaffd640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac100690 0x7f00ac198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 -- 172.21.15.45:0/1776763372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac100690 msgr2=0x7f00ac198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac100690 0x7f00ac198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 -- 172.21.15.45:0/1776763372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac106650 msgr2=0x7f00ac1a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac106650 0x7f00ac1a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.029+0000 7f00a3fff640 1 -- 172.21.15.45:0/1776763372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1a43e0 con 0x7f00ac101050 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00a3fff640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f009400e9f0 tx=0x7f009400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009400cd80 con 0x7f00ac101050 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0094010910 con 0x7f00ac101050 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1a45b0 con 0x7f00ac101050 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094018990 con 0x7f00ac101050 2024-04-25T00:33:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.030+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1a4a70 con 0x7f00ac101050 2024-04-25T00:33:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.031+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac101d80 con 0x7f00ac101050 2024-04-25T00:33:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.032+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0094010ac0 con 0x7f00ac101050 2024-04-25T00:33:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.033+0000 7f00b0919640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 0x7f00980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.033+0000 7f00aaffd640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 0x7f00980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.033+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f00940d0010 con 0x7f00ac101050 2024-04-25T00:33:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.033+0000 7f00aaffd640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 0x7f00980a51c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f009c00cac0 tx=0x7f009c00c4a0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.036+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f009409c870 con 0x7f00ac101050 2024-04-25T00:33:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:03 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3661623344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:33:04.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:03 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/163536852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:33:04.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:03 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3661623344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:33:04.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:03 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/163536852' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:33:04.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.259+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00ac0689b0 con 0x7f00ac101050 2024-04-25T00:33:04.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.260+0000 7f00b0919640 1 -- 172.21.15.45:0/1776763372 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11932 (secure 0 0 0) 0x7f00ac0689b0 con 0x7f00ac101050 2024-04-25T00:33:04.261 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:33:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3465528843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3465528843},{"type":"v1","addr":"172.21.15.45:6835","nonce":3465528843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/2745663812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":2745663812},{"type":"v1","addr":"172.21.15.45:6837","nonce":2745663812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:19:33.221862+0000","modified":"2024-04-25T00:21:55.552420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24493,"mds_2":24478,"mds_3":24467,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/2848612989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":2848612989},{"type":"v1","addr":"172.21.15.45:6839","nonce":2848612989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6841/3316648689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3316648689},{"type":"v1","addr":"172.21.15.45:6841","nonce":3316648689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/1511686553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1511686553},{"type":"v1","addr":"172.21.15.71:6833","nonce":1511686553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/3066271880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3066271880},{"type":"v1","addr":"172.21.15.122:6833","nonce":3066271880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/741071549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":741071549},{"type":"v1","addr":"172.21.15.71:6835","nonce":741071549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.122:6835/1025393130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1025393130},{"type":"v1","addr":"172.21.15.122:6835","nonce":1025393130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.71:6837/2635562743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2635562743},{"type":"v1","addr":"172.21.15.71:6837","nonce":2635562743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.122:6837/3810708134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3810708134},{"type":"v1","addr":"172.21.15.122:6837","nonce":3810708134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.122:6839/2298542043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":2298542043},{"type":"v1","addr":"172.21.15.122:6839","nonce":2298542043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/102345855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":102345855},{"type":"v1","addr":"172.21.15.71:6839","nonce":102345855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24493,24500,24478,24467,24460]},"id":1}]} 2024-04-25T00:33:04.261 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T00:33:04.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.262+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 msgr2=0x7f00980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.262+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 0x7f00980a51c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f009c00cac0 tx=0x7f009c00c4a0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.262+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 msgr2=0x7f00ac199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.262+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f009400e9f0 tx=0x7f009400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 shutdown_connections 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f00980a2d00 0x7f00980a51c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00ac106650 0x7f00ac1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00ac101050 0x7f00ac199360 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 --2- 172.21.15.45:0/1776763372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00ac100690 0x7f00ac198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 >> 172.21.15.45:0/1776763372 conn(0x7f00ac0777a0 msgr2=0x7f00ac1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 shutdown_connections 2024-04-25T00:33:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.263+0000 7f00b191b640 1 -- 172.21.15.45:0/1776763372 wait complete. 2024-04-25T00:33:04.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:33:04.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.375+0000 7faa5dbf9640 1 -- 172.21.15.45:0/416022962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 msgr2=0x7faa58100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.375+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/416022962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa58100aa0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009930 tx=0x7faa4c02a420 comp rx=0 tx=0).stop 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 -- 172.21.15.45:0/416022962 shutdown_connections 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/416022962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58100fe0 0x7faa5810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/416022962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa58100aa0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/416022962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58106630 0x7faa58106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 -- 172.21.15.45:0/416022962 >> 172.21.15.45:0/416022962 conn(0x7faa58077740 msgr2=0x7faa58077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 -- 172.21.15.45:0/416022962 shutdown_connections 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 -- 172.21.15.45:0/416022962 wait complete. 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 Processor -- start 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 -- start start 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.376+0000 7faa5dbf9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58100fe0 0x7faa5819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 0x7faa5819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa58110a40 con 0x7faa58100620 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa581108c0 con 0x7faa58100fe0 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa58110bc0 con 0x7faa58106630 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34204/0 (socket says 172.21.15.45:34204) 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 -- 172.21.15.45:0/4098391437 learned_addr learned my addr 172.21.15.45:0/4098391437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa56ffd640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58100fe0 0x7faa5819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa57fff640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 0x7faa5819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 -- 172.21.15.45:0/4098391437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 msgr2=0x7faa5819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 0x7faa5819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 -- 172.21.15.45:0/4098391437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58100fe0 msgr2=0x7faa5819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58100fe0 0x7faa5819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 -- 172.21.15.45:0/4098391437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5819f190 con 0x7faa58100620 2024-04-25T00:33:04.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa57fff640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 0x7faa5819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa577fe640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7faa4800d950 tx=0x7faa4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48017070 con 0x7faa58100620 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa48009c30 con 0x7faa58100620 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5819f480 con 0x7faa58100620 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800f6b0 con 0x7faa58100620 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.377+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa581a6d10 con 0x7faa58100620 2024-04-25T00:33:04.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.378+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa58068560 con 0x7faa58100620 2024-04-25T00:33:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.380+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faa480109d0 con 0x7faa58100620 2024-04-25T00:33:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.380+0000 7faa54ff9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 0x7faa400a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.380+0000 7faa56ffd640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 0x7faa400a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.380+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7faa480cf340 con 0x7faa58100620 2024-04-25T00:33:04.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.381+0000 7faa56ffd640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 0x7faa400a5240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009900 tx=0x7faa4c0058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.384+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa48009de0 con 0x7faa58100620 2024-04-25T00:33:04.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.571+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa581a6f80 con 0x7faa58100620 2024-04-25T00:33:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.573+0000 7faa54ff9640 1 -- 172.21.15.45:0/4098391437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+25272 (secure 0 0 0) 0x7faa4809bc20 con 0x7faa58100620 2024-04-25T00:33:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T00:33:04.574 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":102,"fsid":"24bb2e7a-0298-11ef-bc93-c7b262605968","created":"2024-04-25T00:11:31.853921+0000","modified":"2024-04-25T00:32:12.003289+0000","last_up_change":"2024-04-25T00:18:40.539308+0000","last_in_change":"2024-04-25T00:18:20.179674+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:14:51.716786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:19:31.801167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:19:32.608231+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:19:36.319099+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"47ff99bd-1736-4c56-8965-bd5199120838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6803","nonce":2748889100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6805","nonce":2748889100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6809","nonce":2748889100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2748889100},{"type":"v1","addr":"172.21.15.45:6807","nonce":2748889100}]},"public_addr":"172.21.15.45:6803/2748889100","cluster_addr":"172.21.15.45:6805/2748889100","heartbeat_back_addr":"172.21.15.45:6809/2748889100","heartbeat_front_addr":"172.21.15.45:6807/2748889100","state":["exists","up"]},{"osd":1,"uuid":"f0c603f9-9f3e-4af1-9a59-e3e887208af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6801","nonce":3734145643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6803","nonce":3734145643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6807","nonce":3734145643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":3734145643},{"type":"v1","addr":"172.21.15.71:6805","nonce":3734145643}]},"public_addr":"172.21.15.71:6801/3734145643","cluster_addr":"172.21.15.71:6803/3734145643","heartbeat_back_addr":"172.21.15.71:6807/3734145643","heartbeat_front_addr":"172.21.15.71:6805/3734145643","state":["exists","up"]},{"osd":2,"uuid":"e528381f-f657-4e55-a279-3f45e35fa585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6801","nonce":3829837658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6803","nonce":3829837658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6807","nonce":3829837658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3829837658},{"type":"v1","addr":"172.21.15.122:6805","nonce":3829837658}]},"public_addr":"172.21.15.122:6801/3829837658","cluster_addr":"172.21.15.122:6803/3829837658","heartbeat_back_addr":"172.21.15.122:6807/3829837658","heartbeat_front_addr":"172.21.15.122:6805/3829837658","state":["exists","up"]},{"osd":3,"uuid":"d0f7644b-671b-4e42-b161-8895d46c48a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6811","nonce":2950579455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6813","nonce":2950579455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6817","nonce":2950579455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2950579455},{"type":"v1","addr":"172.21.15.45:6815","nonce":2950579455}]},"public_addr":"172.21.15.45:6811/2950579455","cluster_addr":"172.21.15.45:6813/2950579455","heartbeat_back_addr":"172.21.15.45:6817/2950579455","heartbeat_front_addr":"172.21.15.45:6815/2950579455","state":["exists","up"]},{"osd":4,"uuid":"40f29fc7-dd8a-45cc-b8b5-d837364a6001","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6809","nonce":2155408782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6811","nonce":2155408782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6815","nonce":2155408782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2155408782},{"type":"v1","addr":"172.21.15.71:6813","nonce":2155408782}]},"public_addr":"172.21.15.71:6809/2155408782","cluster_addr":"172.21.15.71:6811/2155408782","heartbeat_back_addr":"172.21.15.71:6815/2155408782","heartbeat_front_addr":"172.21.15.71:6813/2155408782","state":["exists","up"]},{"osd":5,"uuid":"1fb2eebb-1460-46c7-bdf2-aa80df2b595d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6809","nonce":2119403630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6811","nonce":2119403630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6815","nonce":2119403630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2119403630},{"type":"v1","addr":"172.21.15.122:6813","nonce":2119403630}]},"public_addr":"172.21.15.122:6809/2119403630","cluster_addr":"172.21.15.122:6811/2119403630","heartbeat_back_addr":"172.21.15.122:6815/2119403630","heartbeat_front_addr":"172.21.15.122:6813/2119403630","state":["exists","up"]},{"osd":6,"uuid":"6624e936-d5b3-40f7-9a81-cdca44201b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6819","nonce":4007679117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6821","nonce":4007679117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6825","nonce":4007679117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":4007679117},{"type":"v1","addr":"172.21.15.45:6823","nonce":4007679117}]},"public_addr":"172.21.15.45:6819/4007679117","cluster_addr":"172.21.15.45:6821/4007679117","heartbeat_back_addr":"172.21.15.45:6825/4007679117","heartbeat_front_addr":"172.21.15.45:6823/4007679117","state":["exists","up"]},{"osd":7,"uuid":"1e86e97d-8982-4535-9be0-961d692f1472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6817","nonce":2471128651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6819","nonce":2471128651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6823","nonce":2471128651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2471128651},{"type":"v1","addr":"172.21.15.71:6821","nonce":2471128651}]},"public_addr":"172.21.15.71:6817/2471128651","cluster_addr":"172.21.15.71:6819/2471128651","heartbeat_back_addr":"172.21.15.71:6823/2471128651","heartbeat_front_addr":"172.21.15.71:6821/2471128651","state":["exists","up"]},{"osd":8,"uuid":"4a36bdfc-d527-4419-a2d2-03a09912a395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6817","nonce":2592930089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6819","nonce":2592930089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6823","nonce":2592930089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2592930089},{"type":"v1","addr":"172.21.15.122:6821","nonce":2592930089}]},"public_addr":"172.21.15.122:6817/2592930089","cluster_addr":"172.21.15.122:6819/2592930089","heartbeat_back_addr":"172.21.15.122:6823/2592930089","heartbeat_front_addr":"172.21.15.122:6821/2592930089","state":["exists","up"]},{"osd":9,"uuid":"8856abc5-be19-468e-9205-df339f75e018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6827","nonce":3783590854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6829","nonce":3783590854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6833","nonce":3783590854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3783590854},{"type":"v1","addr":"172.21.15.45:6831","nonce":3783590854}]},"public_addr":"172.21.15.45:6827/3783590854","cluster_addr":"172.21.15.45:6829/3783590854","heartbeat_back_addr":"172.21.15.45:6833/3783590854","heartbeat_front_addr":"172.21.15.45:6831/3783590854","state":["exists","up"]},{"osd":10,"uuid":"d7602a6e-0267-4732-90f6-794201613999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6825","nonce":2371437490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6827","nonce":2371437490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6831","nonce":2371437490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2371437490},{"type":"v1","addr":"172.21.15.71:6829","nonce":2371437490}]},"public_addr":"172.21.15.71:6825/2371437490","cluster_addr":"172.21.15.71:6827/2371437490","heartbeat_back_addr":"172.21.15.71:6831/2371437490","heartbeat_front_addr":"172.21.15.71:6829/2371437490","state":["exists","up"]},{"osd":11,"uuid":"f89083ac-5fb2-4e72-ad19-20637464344f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6825","nonce":1901931479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6827","nonce":1901931479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6831","nonce":1901931479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1901931479},{"type":"v1","addr":"172.21.15.122:6829","nonce":1901931479}]},"public_addr":"172.21.15.122:6825/1901931479","cluster_addr":"172.21.15.122:6827/1901931479","heartbeat_back_addr":"172.21.15.122:6831/1901931479","heartbeat_front_addr":"172.21.15.122:6829/1901931479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:01.715331+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:24.986589+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:49.127626+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:14.260436+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:38.533189+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:03.251807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:28.246711+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:16:53.573456+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:19.675040+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:17:46.176479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:12.025288+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:18:38.457753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/937209566":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1588770344":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6801/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6801/3305016128":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/2421422338":"2024-04-26T00:21:57.361646+0000","172.21.15.45:6800/1299966585":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3254971976":"2024-04-26T00:12:25.175459+0000","172.21.15.45:0/3329362383":"2024-04-26T00:21:57.361646+0000","172.21.15.45:0/1950515838":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2906156154":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/3376575480":"2024-04-26T00:12:25.175459+0000","172.21.15.45:6800/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:6801/200010672":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2002798989":"2024-04-26T00:11:58.120353+0000","172.21.15.45:0/2389946368":"2024-04-26T00:12:25.175459+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:33:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 msgr2=0x7faa400a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 0x7faa400a5240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009900 tx=0x7faa4c0058d0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 msgr2=0x7faa5819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7faa4800d950 tx=0x7faa4800de10 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 shutdown_connections 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7faa400a2d80 0x7faa400a5240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa58106630 0x7faa5819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa58100fe0 0x7faa5819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 --2- 172.21.15.45:0/4098391437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa58100620 0x7faa5819a1a0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 >> 172.21.15.45:0/4098391437 conn(0x7faa58077740 msgr2=0x7faa580688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 shutdown_connections 2024-04-25T00:33:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.575+0000 7faa5dbf9640 1 -- 172.21.15.45:0/4098391437 wait complete. 2024-04-25T00:33:04.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:33:04.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- 172.21.15.45:0/420110201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281029c0 msgr2=0x7f3428102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- 172.21.15.45:0/420110201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281029c0 0x7f3428102da0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3424009900 tx=0x7f342402a510 comp rx=0 tx=0).stop 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- 172.21.15.45:0/420110201 shutdown_connections 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- 172.21.15.45:0/420110201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342810bd00 0x7f342810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- 172.21.15.45:0/420110201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34281032e0 0x7f342810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- 172.21.15.45:0/420110201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281029c0 0x7f3428102da0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- 172.21.15.45:0/420110201 >> 172.21.15.45:0/420110201 conn(0x7f3428077740 msgr2=0x7f3428077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- 172.21.15.45:0/420110201 shutdown_connections 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- 172.21.15.45:0/420110201 wait complete. 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 Processor -- start 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- start start 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34281029c0 0x7f3428198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 0x7f342819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3428111b60 con 0x7f34281032e0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34281119e0 con 0x7f34281029c0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.691+0000 7f3430163640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3428111ce0 con 0x7f342810bd00 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34224/0 (socket says 172.21.15.45:34224) 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 -- 172.21.15.45:0/489566033 learned_addr learned my addr 172.21.15.45:0/489566033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342deff640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34281029c0 0x7f3428198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342e700640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 0x7f342819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 -- 172.21.15.45:0/489566033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 msgr2=0x7f342819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 0x7f342819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 -- 172.21.15.45:0/489566033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34281029c0 msgr2=0x7f3428198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34281029c0 0x7f3428198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 -- 172.21.15.45:0/489566033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342819d070 con 0x7f34281032e0 2024-04-25T00:33:04.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342e700640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 0x7f342819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f342d6fe640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f341800d770 tx=0x7f341800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418017070 con 0x7f34281032e0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342819d360 con 0x7f34281032e0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3418005500 con 0x7f34281032e0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418004e00 con 0x7f34281032e0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.692+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a4bf0 con 0x7f34281032e0 2024-04-25T00:33:04.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.693+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3428068560 con 0x7f34281032e0 2024-04-25T00:33:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.694+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3418004fa0 con 0x7f34281032e0 2024-04-25T00:33:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.695+0000 7f341effd640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 0x7f34140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.695+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9475+0+0 (secure 0 0 0) 0x7f34180cf290 con 0x7f34281032e0 2024-04-25T00:33:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.695+0000 7f342deff640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 0x7f34140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:04.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.696+0000 7f342deff640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 0x7f34140a51c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3424009230 tx=0x7f34240058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:04.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.700+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f341809baf0 con 0x7f34281032e0 2024-04-25T00:33:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mon[29194]: pgmap v344: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T00:33:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2623718708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1776763372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/4098391437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.924+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34281068f0 con 0x7f34281032e0 2024-04-25T00:33:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.927+0000 7f341effd640 1 -- 172.21.15.45:0/489566033 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f341809b3f0 con 0x7f34281032e0 2024-04-25T00:33:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:33:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.930+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 msgr2=0x7f34140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.930+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 0x7f34140a51c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3424009230 tx=0x7f34240058d0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.930+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 msgr2=0x7f34281985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:04.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.930+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f341800d770 tx=0x7f341800dc30 comp rx=0 tx=0).stop 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 shutdown_connections 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f34140a2d00 0x7f34140a51c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f342810bd00 0x7f342819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34281032e0 0x7f34281985c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 --2- 172.21.15.45:0/489566033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34281029c0 0x7f3428198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 >> 172.21.15.45:0/489566033 conn(0x7f3428077740 msgr2=0x7f34280fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 shutdown_connections 2024-04-25T00:33:04.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:04.931+0000 7f3430163640 1 -- 172.21.15.45:0/489566033 wait complete. 2024-04-25T00:33:04.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:33:04.974 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mon[32667]: pgmap v344: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T00:33:04.974 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2623718708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.974 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1776763372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.974 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/4098391437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.974 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mon[32959]: pgmap v344: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T00:33:04.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2623718708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1776763372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:33:04.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/4098391437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 -- 172.21.15.45:0/606075151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538074410 msgr2=0x7f953810da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 --2- 172.21.15.45:0/606075151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538074410 0x7f953810da90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9520009900 tx=0x7f952002a480 comp rx=0 tx=0).stop 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 -- 172.21.15.45:0/606075151 shutdown_connections 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 --2- 172.21.15.45:0/606075151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f95381103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 --2- 172.21.15.45:0/606075151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538074410 0x7f953810da90 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 --2- 172.21.15.45:0/606075151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538073af0 0x7f9538073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 -- 172.21.15.45:0/606075151 >> 172.21.15.45:0/606075151 conn(0x7f95380fd180 msgr2=0x7f95380ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 -- 172.21.15.45:0/606075151 shutdown_connections 2024-04-25T00:33:05.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.055+0000 7f953cea5640 1 -- 172.21.15.45:0/606075151 wait complete. 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 Processor -- start 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 -- start start 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538074410 0x7f953819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f953819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9538113cb0 con 0x7f9538073af0 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9538113b30 con 0x7f9538074410 2024-04-25T00:33:05.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953cea5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9538113e30 con 0x7f953810dfd0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.056+0000 7f953659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34286/0 (socket says 172.21.15.45:34286) 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 -- 172.21.15.45:0/2554872461 learned_addr learned my addr 172.21.15.45:0/2554872461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f952dd9b640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538074410 0x7f953819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f9536d9d640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f953819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 -- 172.21.15.45:0/2554872461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 msgr2=0x7f953819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f953819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 -- 172.21.15.45:0/2554872461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538074410 msgr2=0x7f953819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538074410 0x7f953819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 -- 172.21.15.45:0/2554872461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953819f260 con 0x7f9538073af0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f9536d9d640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f953819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953659c640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f952800d950 tx=0x7f952800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528017070 con 0x7f9538073af0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9528009c30 con 0x7f9538073af0 2024-04-25T00:33:05.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953819f550 con 0x7f9538073af0 2024-04-25T00:33:05.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952800f6b0 con 0x7f9538073af0 2024-04-25T00:33:05.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.057+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a6de0 con 0x7f9538073af0 2024-04-25T00:33:05.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.058+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95380689b0 con 0x7f9538073af0 2024-04-25T00:33:05.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.059+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95280109d0 con 0x7f9538073af0 2024-04-25T00:33:05.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.060+0000 7f952ffff640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 0x7f95240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.060+0000 7f952dd9b640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 0x7f95240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.060+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9817+0+0 (secure 0 0 0) 0x7f95280cf570 con 0x7f9538073af0 2024-04-25T00:33:05.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.061+0000 7f952dd9b640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 0x7f95240a51c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9520009230 tx=0x7f95200058d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.064+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9528009de0 con 0x7f9538073af0 2024-04-25T00:33:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.287+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f953802d170 con 0x7f9538073af0 2024-04-25T00:33:05.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.307 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.307 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mds[63830]: -- 172.21.15.122:0/3081511529 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c48d821a40 con 0x55c48d7f4800 2024-04-25T00:33:05.307 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: starting mds.i at 2024-04-25T00:33:05.308 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.308 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.308 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mds[63522]: -- 172.21.15.122:0/1423982633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c84eefa40 con 0x556c84ec2c00 2024-04-25T00:33:05.308 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: starting mds.c at 2024-04-25T00:33:05.309 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.309 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.309 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mds[63215]: -- 172.21.15.122:0/3121418153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5565f8bfc4e0 con 0x5565f8bd2800 2024-04-25T00:33:05.309 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mds[63215]: -- 172.21.15.122:0/3121418153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565f8bffa40 con 0x5565f8bd2800 2024-04-25T00:33:05.309 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: starting mds.f at 2024-04-25T00:33:05.310 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.310 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.310 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-mds[62898]: -- 172.21.15.122:0/3138092950 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b2fd41a40 con 0x555b2fd15000 2024-04-25T00:33:05.310 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:33:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: starting mds.l at 2024-04-25T00:33:05.369 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.369 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.369 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: starting mds.h at 2024-04-25T00:33:05.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.370 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mds[62876]: -- 172.21.15.71:0/1229464492 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b4c627a40 con 0x558b4c5fac00 2024-04-25T00:33:05.370 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: starting mds.b at 2024-04-25T00:33:05.370 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.370 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mds[62267]: -- 172.21.15.71:0/1059090379 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555cf95404e0 con 0x555cf9517000 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mds[62267]: -- 172.21.15.71:0/1059090379 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cf9543a40 con 0x555cf9517000 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: starting mds.e at 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mds[61945]: -- 172.21.15.71:0/1171540736 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d8bd91a1a0 con 0x55d8bd8f0800 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-mds[61945]: -- 172.21.15.71:0/1171540736 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8bd91da40 con 0x55d8bd8f0800 2024-04-25T00:33:05.371 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:33:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: starting mds.k at 2024-04-25T00:33:05.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a[29190]: 2024-04-25T00:33:04.925+0000 7f197d4cb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:33:05.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mds[68922]: -- 172.21.15.45:0/674214436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55939dfb5a40 con 0x55939df89000 2024-04-25T00:33:05.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: starting mds.a at 2024-04-25T00:33:05.400 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:05.401 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:05.401 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-mds[68568]: -- 172.21.15.45:0/3622090351 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a23739a40 con 0x559a2370c800 2024-04-25T00:33:05.401 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:33:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: starting mds.g at 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:33:05.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:33:05.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:05 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:33:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.932+0000 7f952ffff640 1 -- 172.21.15.45:0/2554872461 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f952809bc80 con 0x7f9538073af0 2024-04-25T00:33:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.935+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 msgr2=0x7f95240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.935+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 0x7f95240a51c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9520009230 tx=0x7f95200058d0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.935+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 msgr2=0x7f953819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.935+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f952800d950 tx=0x7f952800de10 comp rx=0 tx=0).stop 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 shutdown_connections 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f95240a2d00 0x7f95240a51c0 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9520009230 tx=0x7f95200058d0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f953810dfd0 0x7f953819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9538074410 0x7f953819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 --2- 172.21.15.45:0/2554872461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9538073af0 0x7f953819a270 secure :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f952800d950 tx=0x7f952800de10 comp rx=0 tx=0).stop 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 >> 172.21.15.45:0/2554872461 conn(0x7f95380fd180 msgr2=0x7f95380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 shutdown_connections 2024-04-25T00:33:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:05.936+0000 7f953cea5640 1 -- 172.21.15.45:0/2554872461 wait complete. 2024-04-25T00:33:05.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:33:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:33:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:33:06.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:33:06.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:33:06.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:33:06.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:33:06.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:05 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:33:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 -- 172.21.15.45:0/3691159686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c102100 msgr2=0x7ff58c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 --2- 172.21.15.45:0/3691159686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c102100 0x7ff58c10e610 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff584009cd0 tx=0x7ff58402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 -- 172.21.15.45:0/3691159686 shutdown_connections 2024-04-25T00:33:06.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 --2- 172.21.15.45:0/3691159686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c102100 0x7ff58c10e610 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 --2- 172.21.15.45:0/3691159686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c101740 0x7ff58c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 --2- 172.21.15.45:0/3691159686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c107750 0x7ff58c107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 -- 172.21.15.45:0/3691159686 >> 172.21.15.45:0/3691159686 conn(0x7ff58c078bd0 msgr2=0x7ff58c078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 -- 172.21.15.45:0/3691159686 shutdown_connections 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.066+0000 7ff592483640 1 -- 172.21.15.45:0/3691159686 wait complete. 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 Processor -- start 2024-04-25T00:33:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 -- start start 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c101740 0x7ff58c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c102100 0x7ff58c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111b60 con 0x7ff58c107750 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1119e0 con 0x7ff58c102100 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff592483640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111ce0 con 0x7ff58c101740 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff590a20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff590a20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34298/0 (socket says 172.21.15.45:34298) 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.067+0000 7ff590a20640 1 -- 172.21.15.45:0/2700019286 learned_addr learned my addr 172.21.15.45:0/2700019286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff58bfff640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c101740 0x7ff58c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff58b7fe640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c102100 0x7ff58c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 -- 172.21.15.45:0/2700019286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c101740 msgr2=0x7ff58c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c101740 0x7ff58c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 -- 172.21.15.45:0/2700019286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c102100 msgr2=0x7ff58c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c102100 0x7ff58c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 -- 172.21.15.45:0/2700019286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c19f220 con 0x7ff58c107750 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff590a20640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff584005cc0 tx=0x7ff5840093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584003e00 con 0x7ff58c107750 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff584004520 con 0x7ff58c107750 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c19f4b0 con 0x7ff58c107750 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584030db0 con 0x7ff58c107750 2024-04-25T00:33:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.068+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a6da0 con 0x7ff58c107750 2024-04-25T00:33:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.069+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c068560 con 0x7ff58c107750 2024-04-25T00:33:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.070+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff5840046d0 con 0x7ff58c107750 2024-04-25T00:33:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.071+0000 7ff5897fa640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 0x7ff5740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.071+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9817+0+0 (secure 0 0 0) 0x7ff58402e030 con 0x7ff58c107750 2024-04-25T00:33:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.071+0000 7ff58bfff640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 0x7ff5740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.072+0000 7ff58bfff640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 0x7ff5740a51c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005fd0 tx=0x7ff57c00b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:06.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.076+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff58404b060 con 0x7ff58c107750 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/489566033' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T00:33:06.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:05 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:33:06.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.261+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff58c1a7040 con 0x7ff58c107750 2024-04-25T00:33:06.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: pgmap v346: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: Cluster is now healthy 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6838/782319705,v1:172.21.15.45:6839/782319705] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6832/3544401609,v1:172.21.15.122:6834/3544401609] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6833/387769484,v1:172.21.15.122:6835/387769484] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6832/3544690111,v1:172.21.15.71:6834/3544690111] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.45:6840/212825801,v1:172.21.15.45:6841/212825801] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6833/3750142776,v1:172.21.15.71:6835/3750142776] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6836/2300773009,v1:172.21.15.71:6837/2300773009] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.71:6838/2100728254,v1:172.21.15.71:6839/2100728254] up:boot 2024-04-25T00:33:06.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6836/4072570055,v1:172.21.15.122:6838/4072570055] up:boot 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: mds.? [v2:172.21.15.122:6837/16510795,v1:172.21.15.122:6839/16510795] up:boot 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: fsmap 12 up:standby 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:33:06.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:33:06.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:06 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:06.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.933+0000 7ff5897fa640 1 -- 172.21.15.45:0/2700019286 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7ff5840bd6a0 con 0x7ff58c107750 2024-04-25T00:33:06.934 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:33:06.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 msgr2=0x7ff5740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 0x7ff5740a51c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005fd0 tx=0x7ff57c00b390 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 msgr2=0x7ff58c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff584005cc0 tx=0x7ff5840093d0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 shutdown_connections 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7ff5740a2d00 0x7ff5740a51c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff58c107750 0x7ff58c19eaf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff58c102100 0x7ff58c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 --2- 172.21.15.45:0/2700019286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c101740 0x7ff58c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:06.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 >> 172.21.15.45:0/2700019286 conn(0x7ff58c078bd0 msgr2=0x7ff58c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:06.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 shutdown_connections 2024-04-25T00:33:06.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:06.937+0000 7ff592483640 1 -- 172.21.15.45:0/2700019286 wait complete. 2024-04-25T00:33:06.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:33:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: pgmap v346: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-25T00:33:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:33:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:33:07.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: Cluster is now healthy 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6838/782319705,v1:172.21.15.45:6839/782319705] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6832/3544401609,v1:172.21.15.122:6834/3544401609] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6833/387769484,v1:172.21.15.122:6835/387769484] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6832/3544690111,v1:172.21.15.71:6834/3544690111] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.45:6840/212825801,v1:172.21.15.45:6841/212825801] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6833/3750142776,v1:172.21.15.71:6835/3750142776] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6836/2300773009,v1:172.21.15.71:6837/2300773009] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.71:6838/2100728254,v1:172.21.15.71:6839/2100728254] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6836/4072570055,v1:172.21.15.122:6838/4072570055] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: mds.? [v2:172.21.15.122:6837/16510795,v1:172.21.15.122:6839/16510795] up:boot 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: fsmap 12 up:standby 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:33:07.057 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:33:07.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:06 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 -- 172.21.15.45:0/712426777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 msgr2=0x7f1cfc10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 --2- 172.21.15.45:0/712426777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc10e1a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009ca0 tx=0x7f1cf402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 -- 172.21.15.45:0/712426777 shutdown_connections 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 --2- 172.21.15.45:0/712426777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc10e1a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 --2- 172.21.15.45:0/712426777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 0x7f1cfc10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 --2- 172.21.15.45:0/712426777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 0x7f1cfc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 -- 172.21.15.45:0/712426777 >> 172.21.15.45:0/712426777 conn(0x7f1cfc077740 msgr2=0x7f1cfc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 -- 172.21.15.45:0/712426777 shutdown_connections 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.058+0000 7f1d03370640 1 -- 172.21.15.45:0/712426777 wait complete. 2024-04-25T00:33:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 Processor -- start 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 -- start start 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 0x7f1cfc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 0x7f1cfc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc111b00 con 0x7f1cfc10bd00 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc111980 con 0x7f1cfc1029c0 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d03370640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc111c80 con 0x7f1cfc1032e0 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d0190d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d0190d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34316/0 (socket says 172.21.15.45:34316) 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d0190d640 1 -- 172.21.15.45:0/1855948675 learned_addr learned my addr 172.21.15.45:0/1855948675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d0090b640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 0x7f1cfc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.059+0000 7f1d0110c640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 0x7f1cfc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 -- 172.21.15.45:0/1855948675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 msgr2=0x7f1cfc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 0x7f1cfc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 -- 172.21.15.45:0/1855948675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 msgr2=0x7f1cfc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 0x7f1cfc19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 -- 172.21.15.45:0/1855948675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc19f1b0 con 0x7f1cfc10bd00 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d0190d640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006010 tx=0x7f1cf4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4047070 con 0x7f1cfc10bd00 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc19f440 con 0x7f1cfc10bd00 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf402fb20 con 0x7f1cfc10bd00 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4033050 con 0x7f1cfc10bd00 2024-04-25T00:33:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.060+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1a6d30 con 0x7f1cfc10bd00 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.061+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cfc068560 con 0x7f1cfc10bd00 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.062+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1cf404a070 con 0x7f1cfc10bd00 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.063+0000 7f1cf27fc640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 0x7f1ce80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.063+0000 7f1d0110c640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 0x7f1ce80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.063+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9255+0+0 (secure 0 0 0) 0x7f1cf40f0260 con 0x7f1cfc10bd00 2024-04-25T00:33:07.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.063+0000 7f1d0110c640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 0x7f1ce80a51c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800bbe0 tx=0x7f1cf800b390 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.067+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1cf407e740 con 0x7f1cfc10bd00 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: pgmap v346: 129 pgs: 129 active+clean; 2.2 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: Cluster is now healthy 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2554872461' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6838/782319705,v1:172.21.15.45:6839/782319705] up:boot 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6832/3544401609,v1:172.21.15.122:6834/3544401609] up:boot 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6833/387769484,v1:172.21.15.122:6835/387769484] up:boot 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6832/3544690111,v1:172.21.15.71:6834/3544690111] up:boot 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.45:6840/212825801,v1:172.21.15.45:6841/212825801] up:boot 2024-04-25T00:33:07.119 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6833/3750142776,v1:172.21.15.71:6835/3750142776] up:boot 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6836/2300773009,v1:172.21.15.71:6837/2300773009] up:boot 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.71:6838/2100728254,v1:172.21.15.71:6839/2100728254] up:boot 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6836/4072570055,v1:172.21.15.122:6838/4072570055] up:boot 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: mds.? [v2:172.21.15.122:6837/16510795,v1:172.21.15.122:6839/16510795] up:boot 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: fsmap 12 up:standby 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:33:07.120 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:33:07.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='mgr.25163 172.21.15.45:0/1558514340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:33:07.121 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:06 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:07.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.252+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1cfc1092b0 con 0x7f1cfc10bd00 2024-04-25T00:33:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.939+0000 7f1cf27fc640 1 -- 172.21.15.45:0/1855948675 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f1cf40bc550 con 0x7f1cfc10bd00 2024-04-25T00:33:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 msgr2=0x7f1ce80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 0x7f1ce80a51c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800bbe0 tx=0x7f1cf800b390 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 msgr2=0x7f1cfc19ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006010 tx=0x7f1cf4003ba0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 shutdown_connections 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f1ce80a2d00 0x7f1ce80a51c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cfc10bd00 0x7f1cfc19ea80 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cfc1032e0 0x7f1cfc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 --2- 172.21.15.45:0/1855948675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cfc1029c0 0x7f1cfc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.943+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 >> 172.21.15.45:0/1855948675 conn(0x7f1cfc077740 msgr2=0x7f1cfc101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.944+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 shutdown_connections 2024-04-25T00:33:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:07.944+0000 7f1d03370640 1 -- 172.21.15.45:0/1855948675 wait complete. 2024-04-25T00:33:07.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T00:33:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 -- 172.21.15.45:0/2355587622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc100990 msgr2=0x7f20dc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 --2- 172.21.15.45:0/2355587622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc100990 0x7f20dc100df0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009900 tx=0x7f20d002a410 comp rx=0 tx=0).stop 2024-04-25T00:33:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 -- 172.21.15.45:0/2355587622 shutdown_connections 2024-04-25T00:33:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 --2- 172.21.15.45:0/2355587622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 0x7f20dc10d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 --2- 172.21.15.45:0/2355587622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc100990 0x7f20dc100df0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 --2- 172.21.15.45:0/2355587622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc106690 0x7f20dc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 -- 172.21.15.45:0/2355587622 >> 172.21.15.45:0/2355587622 conn(0x7f20dc076440 msgr2=0x7f20dc076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 -- 172.21.15.45:0/2355587622 shutdown_connections 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.072+0000 7f20e24a5640 1 -- 172.21.15.45:0/2355587622 wait complete. 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 Processor -- start 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 -- start start 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc100990 0x7f20dc196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 0x7f20dc1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.073+0000 7f20e24a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1109e0 con 0x7f20dc106690 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e24a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc110860 con 0x7f20dc100990 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e24a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc110b60 con 0x7f20dc101330 2024-04-25T00:33:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34336/0 (socket says 172.21.15.45:34336) 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 -- 172.21.15.45:0/3749210758 learned_addr learned my addr 172.21.15.45:0/3749210758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20dbfff640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc100990 0x7f20dc196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20db7fe640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 0x7f20dc1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 -- 172.21.15.45:0/3749210758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 msgr2=0x7f20dc1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 0x7f20dc1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 -- 172.21.15.45:0/3749210758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc100990 msgr2=0x7f20dc196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc100990 0x7f20dc196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 -- 172.21.15.45:0/3749210758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc19be40 con 0x7f20dc106690 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e0a42640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f20d400cad0 tx=0x7f20d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400ae20 con 0x7f20dc106690 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4005020 con 0x7f20dc106690 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc19c130 con 0x7f20dc106690 2024-04-25T00:33:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400f6f0 con 0x7f20dc106690 2024-04-25T00:33:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.074+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1a39c0 con 0x7f20dc106690 2024-04-25T00:33:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.075+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc101d80 con 0x7f20dc106690 2024-04-25T00:33:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.076+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20d40051d0 con 0x7f20dc106690 2024-04-25T00:33:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.077+0000 7f20d97fa640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 0x7f20c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.077+0000 7f20dbfff640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 0x7f20c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.078+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8754+0+0 (secure 0 0 0) 0x7f20d40cf850 con 0x7f20dc106690 2024-04-25T00:33:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.078+0000 7f20dbfff640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 0x7f20c40a51c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0058c0 tx=0x7f20cc00a2d0 comp rx=0 tx=0).ready entity=mgr.25163 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.082+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20d4014030 con 0x7f20dc106690 2024-04-25T00:33:08.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.267+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20dc02d170 con 0x7f20dc106690 2024-04-25T00:33:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:07 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:07 smithi122 ceph-mon[32959]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:33:08.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:07 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:07 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:08.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:07 smithi071 ceph-mon[32667]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:33:08.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:07 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:07 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/2700019286' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:07 smithi045 ceph-mon[29194]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:33:08.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:07 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.946+0000 7f20d97fa640 1 -- 172.21.15.45:0/3749210758 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v106) ==== 178+0+0 (secure 0 0 0) 0x7f20d409bcd0 con 0x7f20dc106690 2024-04-25T00:33:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2024-04-25T00:33:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.948+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 msgr2=0x7f20c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 0x7f20c40a51c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0058c0 tx=0x7f20cc00a2d0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 msgr2=0x7f20dc19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f20d400cad0 tx=0x7f20d400cf90 comp rx=0 tx=0).stop 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 shutdown_connections 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:6800/1314659631,v1:172.21.15.45:6801/1314659631] conn(0x7f20c40a2d00 0x7f20c40a51c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20dc106690 0x7f20dc19b710 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc101330 0x7f20dc1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 --2- 172.21.15.45:0/3749210758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20dc100990 0x7f20dc196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 >> 172.21.15.45:0/3749210758 conn(0x7f20dc076440 msgr2=0x7f20dc0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 shutdown_connections 2024-04-25T00:33:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T00:33:08.949+0000 7f20e24a5640 1 -- 172.21.15.45:0/3749210758 wait complete. 2024-04-25T00:33:08.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:33:08.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:33:09.025 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:33:09.057 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T00:33:09.058 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:33:09.074 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:33:09.090 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:33:09.115 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T00:33:09.115 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T00:33:09.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:33:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:08 smithi122 ceph-mon[32959]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:33:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:08 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:09.306 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:08 smithi122 ceph-mon[32959]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:33:09.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:08 smithi122 ceph-mon[32959]: from='client.? 172.21.15.45:0/3749210758' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:09.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:08 smithi071 ceph-mon[32667]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:33:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:08 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:08 smithi071 ceph-mon[32667]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:33:09.369 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:08 smithi071 ceph-mon[32667]: from='client.? 172.21.15.45:0/3749210758' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:08 smithi045 ceph-mon[29194]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:33:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:08 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/1855948675' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:33:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:08 smithi045 ceph-mon[29194]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:33:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:08 smithi045 ceph-mon[29194]: from='client.? 172.21.15.45:0/3749210758' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:33:09.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:09 smithi045 systemd[1]: Stopping Ceph mon.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:09.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a[29190]: 2024-04-25T00:33:09.555+0000 7f1982cd6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:09.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a[29190]: 2024-04-25T00:33:09.555+0000 7f1982cd6700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T00:33:09.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:09 smithi045 podman[99131]: 2024-04-25 00:33:09.689245921 +0000 UTC m=+0.372882430 container died 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:33:10.348 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:10 smithi045 podman[99131]: 2024-04-25 00:33:10.064077386 +0000 UTC m=+0.747713897 container cleanup 6599841ffb1abc6d284df04f9b5a518cac1a36d41ffdb485b389cc31272cda5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:33:10.348 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 00:33:10 smithi045 bash[99131]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:33:10.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T00:33:10.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:10.550 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T00:33:10.550 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T00:33:10.550 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.b 2024-04-25T00:33:10.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:10 smithi071 systemd[1]: Stopping Ceph mon.b for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:11.338 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-b[32663]: 2024-04-25T00:33:10.894+0000 7fe5d436b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:11.338 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-b[32663]: 2024-04-25T00:33:10.894+0000 7fe5d436b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:33:11.338 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:11 smithi071 podman[68371]: 2024-04-25 00:33:11.05791999 +0000 UTC m=+0.317566288 container died bfaa5703b9bf86dbbc3ab02812ed59f6d2365c2825baac8c0248e5138b68d656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:33:11.618 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:11 smithi071 podman[68371]: 2024-04-25 00:33:11.453992147 +0000 UTC m=+0.713638431 container cleanup bfaa5703b9bf86dbbc3ab02812ed59f6d2365c2825baac8c0248e5138b68d656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T00:33:11.619 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 25 00:33:11 smithi071 bash[68371]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-b 2024-04-25T00:33:11.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T00:33:11.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:11.911 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T00:33:11.911 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T00:33:11.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.c 2024-04-25T00:33:12.221 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:11 smithi122 systemd[1]: Stopping Ceph mon.c for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:12.476 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c[32955]: 2024-04-25T00:33:12.220+0000 7f87312e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:12.476 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c[32955]: 2024-04-25T00:33:12.220+0000 7f87312e9700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:33:12.781 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:12 smithi122 podman[69270]: 2024-04-25 00:33:12.47614033 +0000 UTC m=+0.381090668 container died d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:33:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:12 smithi122 podman[69270]: 2024-04-25 00:33:12.88059892 +0000 UTC m=+0.785549252 container cleanup d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T00:33:13.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:12 smithi122 bash[69270]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c 2024-04-25T00:33:13.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T00:33:13.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:13 smithi122 podman[69284]: 2024-04-25 00:33:13.149083194 +0000 UTC m=+0.672961651 container remove d47d25f3654b08bfd2c288372220a02aeda6e042ba4ad1e25af2f251d0b7f3c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:33:13.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:13 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-25T00:33:13.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:13 smithi122 systemd[1]: Stopped Ceph mon.c for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:33:13.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 25 00:33:13 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mon.c.service: Consumed 12.451s CPU time. 2024-04-25T00:33:13.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:13.357 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T00:33:13.357 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T00:33:13.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:33:13.649 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:33:13 smithi045 systemd[1]: Stopping Ceph mgr.x for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:14.111 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:33:13 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x[29425]: 2024-04-25T00:33:13.684+0000 7fe420d22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:14.111 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:33:13 smithi045 podman[99339]: 2024-04-25 00:33:13.715688749 +0000 UTC m=+0.152525907 container died 69afc426db4913fe6747be993bdcc3eb17c4ca8fedd549436ba6e9f2869dd866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:33:14.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:33:14 smithi045 podman[99339]: 2024-04-25 00:33:14.201739598 +0000 UTC m=+0.638576750 container cleanup 69afc426db4913fe6747be993bdcc3eb17c4ca8fedd549436ba6e9f2869dd866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:33:14.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 00:33:14 smithi045 bash[99339]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-x 2024-04-25T00:33:14.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T00:33:14.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:14.672 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T00:33:14.672 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T00:33:14.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.y 2024-04-25T00:33:14.967 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:33:14 smithi071 systemd[1]: Stopping Ceph mgr.y for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:15.369 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:33:15 smithi071 podman[68584]: 2024-04-25 00:33:15.196898811 +0000 UTC m=+0.326919125 container died bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T00:33:15.861 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:33:15 smithi071 podman[68584]: 2024-04-25 00:33:15.59276506 +0000 UTC m=+0.722785365 container cleanup bc1d8e561dbd975a39d7dee90a6a88a72c304b7b33a2241d1c76e86dac7c9da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:33:15.861 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Apr 25 00:33:15 smithi071 bash[68584]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-y 2024-04-25T00:33:16.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T00:33:16.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:16.046 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T00:33:16.046 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T00:33:16.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.z 2024-04-25T00:33:16.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:33:16 smithi122 systemd[1]: Stopping Ceph mgr.z for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:16.679 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:33:16 smithi122 podman[69404]: 2024-04-25 00:33:16.375487282 +0000 UTC m=+0.152945392 container died e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:33:17.021 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:33:16 smithi122 podman[69404]: 2024-04-25 00:33:16.761815999 +0000 UTC m=+0.539274099 container cleanup e2985e6f3a91194b75b35843c9d9cfb5473fa7d6764d7b0d082a6ef82f7097a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:33:17.022 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 25 00:33:16 smithi122 bash[69404]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mgr-z 2024-04-25T00:33:17.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T00:33:17.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:17.215 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T00:33:17.216 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T00:33:17.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.0 2024-04-25T00:33:17.531 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:17 smithi045 systemd[1]: Stopping Ceph osd.0 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:17.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0[41753]: 2024-04-25T00:33:17.530+0000 7f1f9b0d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:17.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0[41753]: 2024-04-25T00:33:17.530+0000 7f1f9b0d7700 -1 osd.0 106 *** Got signal Terminated *** 2024-04-25T00:33:17.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0[41753]: 2024-04-25T00:33:17.530+0000 7f1f9b0d7700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:23.149 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:22 smithi045 podman[99458]: 2024-04-25 00:33:22.881921165 +0000 UTC m=+5.464739240 container died 5d857de0ac378ec910a443aadbeba80766adc9adedd1319ff6207041f88fad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T00:33:23.590 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:23 smithi045 podman[99458]: 2024-04-25 00:33:23.322273374 +0000 UTC m=+5.905091452 container cleanup 5d857de0ac378ec910a443aadbeba80766adc9adedd1319ff6207041f88fad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:33:23.590 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:23 smithi045 bash[99458]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0 2024-04-25T00:33:23.899 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:23 smithi045 podman[99475]: 2024-04-25 00:33:23.590222585 +0000 UTC m=+0.702357358 container remove 5d857de0ac378ec910a443aadbeba80766adc9adedd1319ff6207041f88fad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:33:24.163 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:23 smithi045 podman[99565]: 2024-04-25 00:33:23.898799343 +0000 UTC m=+0.018725437 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:33:24.163 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:24 smithi045 podman[99565]: 2024-04-25 00:33:24.024543634 +0000 UTC m=+0.144469766 container create 6aba5d36f710d0e6f86f20a22768ca24c92808d40b25266da3cc2986242d6843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T00:33:24.598 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:24 smithi045 podman[99565]: 2024-04-25 00:33:24.296261768 +0000 UTC m=+0.416187865 container init 6aba5d36f710d0e6f86f20a22768ca24c92808d40b25266da3cc2986242d6843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T00:33:24.598 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:24 smithi045 podman[99565]: 2024-04-25 00:33:24.299426419 +0000 UTC m=+0.419352506 container start 6aba5d36f710d0e6f86f20a22768ca24c92808d40b25266da3cc2986242d6843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:33:24.598 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:24 smithi045 podman[99565]: 2024-04-25 00:33:24.402250164 +0000 UTC m=+0.522176267 container attach 6aba5d36f710d0e6f86f20a22768ca24c92808d40b25266da3cc2986242d6843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-25T00:33:24.896 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 00:33:24 smithi045 podman[99565]: 2024-04-25 00:33:24.597901999 +0000 UTC m=+0.717828101 container died 6aba5d36f710d0e6f86f20a22768ca24c92808d40b25266da3cc2986242d6843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:33:25.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T00:33:25.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:25.280 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T00:33:25.280 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T00:33:25.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.1 2024-04-25T00:33:25.618 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:25 smithi071 systemd[1]: Stopping Ceph osd.1 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:25.972 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1[37297]: 2024-04-25T00:33:25.637+0000 7feb302dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:25.972 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1[37297]: 2024-04-25T00:33:25.637+0000 7feb302dc700 -1 osd.1 106 *** Got signal Terminated *** 2024-04-25T00:33:25.972 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1[37297]: 2024-04-25T00:33:25.637+0000 7feb302dc700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:31.337 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:31 smithi071 podman[68719]: 2024-04-25 00:33:31.009386583 +0000 UTC m=+5.533375210 container died 2c3a1248b7d65b5db0e14c816560470b2ae692e23ca1442117a57cf794ddef00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:33:31.618 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:31 smithi071 podman[68719]: 2024-04-25 00:33:31.438342153 +0000 UTC m=+5.962330775 container cleanup 2c3a1248b7d65b5db0e14c816560470b2ae692e23ca1442117a57cf794ddef00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T00:33:31.619 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:31 smithi071 bash[68719]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1 2024-04-25T00:33:32.050 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:31 smithi071 podman[68736]: 2024-04-25 00:33:31.648103618 +0000 UTC m=+0.642513402 container remove 2c3a1248b7d65b5db0e14c816560470b2ae692e23ca1442117a57cf794ddef00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:33:32.310 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:32.049024245 +0000 UTC m=+0.113445239 container create 16f31a880376a60d36ff00bc2539cbe9321fd5cf6106d0273dbd43bba11b6789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-25T00:33:32.310 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:31.953593289 +0000 UTC m=+0.018014248 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:33:32.600 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:32.309918462 +0000 UTC m=+0.374339402 container init 16f31a880376a60d36ff00bc2539cbe9321fd5cf6106d0273dbd43bba11b6789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T00:33:32.600 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:32.313153478 +0000 UTC m=+0.377574417 container start 16f31a880376a60d36ff00bc2539cbe9321fd5cf6106d0273dbd43bba11b6789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:33:32.600 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:32.476713169 +0000 UTC m=+0.541134114 container attach 16f31a880376a60d36ff00bc2539cbe9321fd5cf6106d0273dbd43bba11b6789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:33:32.868 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 25 00:33:32 smithi071 podman[68826]: 2024-04-25 00:33:32.60048256 +0000 UTC m=+0.664903508 container died 16f31a880376a60d36ff00bc2539cbe9321fd5cf6106d0273dbd43bba11b6789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:33:33.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T00:33:33.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:33.329 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T00:33:33.330 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T00:33:33.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.2 2024-04-25T00:33:33.642 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:33 smithi122 systemd[1]: Stopping Ceph osd.2 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:34.056 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2[38137]: 2024-04-25T00:33:33.640+0000 7f14725f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:34.056 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2[38137]: 2024-04-25T00:33:33.640+0000 7f14725f4700 -1 osd.2 106 *** Got signal Terminated *** 2024-04-25T00:33:34.056 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2[38137]: 2024-04-25T00:33:33.640+0000 7f14725f4700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:39.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:39 smithi122 podman[69527]: 2024-04-25 00:33:39.033685233 +0000 UTC m=+5.507932971 container died 7c33b5f8504fc2959980c90cb5e28ceb2ceebd990e3f8682383838d18e55d4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T00:33:39.693 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:39 smithi122 podman[69527]: 2024-04-25 00:33:39.424810807 +0000 UTC m=+5.899058550 container cleanup 7c33b5f8504fc2959980c90cb5e28ceb2ceebd990e3f8682383838d18e55d4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:33:39.693 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:39 smithi122 bash[69527]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2 2024-04-25T00:33:40.043 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:39 smithi122 podman[69544]: 2024-04-25 00:33:39.693168157 +0000 UTC m=+0.655604541 container remove 7c33b5f8504fc2959980c90cb5e28ceb2ceebd990e3f8682383838d18e55d4f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:33:40.305 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:39.946688651 +0000 UTC m=+0.017808927 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:33:40.305 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:40.069207124 +0000 UTC m=+0.140327384 container create 318917ec7a7b9ab05d611723c01605d8060f12840f10d7c1287fa2ad5259ede0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:33:40.306 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:40.305637995 +0000 UTC m=+0.376758255 container init 318917ec7a7b9ab05d611723c01605d8060f12840f10d7c1287fa2ad5259ede0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:33:40.556 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:40.309066703 +0000 UTC m=+0.380186972 container start 318917ec7a7b9ab05d611723c01605d8060f12840f10d7c1287fa2ad5259ede0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-25T00:33:40.556 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:40.397191446 +0000 UTC m=+0.468311707 container attach 318917ec7a7b9ab05d611723c01605d8060f12840f10d7c1287fa2ad5259ede0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:33:40.909 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 25 00:33:40 smithi122 podman[69631]: 2024-04-25 00:33:40.602003916 +0000 UTC m=+0.673124176 container died 318917ec7a7b9ab05d611723c01605d8060f12840f10d7c1287fa2ad5259ede0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T00:33:41.249 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T00:33:41.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:41.286 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T00:33:41.286 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T00:33:41.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.3 2024-04-25T00:33:41.608 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:41 smithi045 systemd[1]: Stopping Ceph osd.3 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:41.882 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:33:41.607+0000 7fccdc4b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:41.882 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:33:41.607+0000 7fccdc4b4700 -1 osd.3 106 *** Got signal Terminated *** 2024-04-25T00:33:41.882 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:41 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:33:41.607+0000 7fccdc4b4700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:44.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:44 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:44.054+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:44.535 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:44.161+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:44.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:44.534+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:45.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:45 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:45.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:45.522 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:45.153+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:45.649 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:45 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:33:45.378+0000 7fccd8aa8700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.589448+0000 front 2024-04-25T00:33:20.589419+0000 (oldest deadline 2024-04-25T00:33:45.289585+0000) 2024-04-25T00:33:45.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:45.522+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:46.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:45 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:45.793+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:46.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:46 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:46.011+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:46.500 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:46.164+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:46.649 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:46 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3[48319]: 2024-04-25T00:33:46.368+0000 7fccd8aa8700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.589448+0000 front 2024-04-25T00:33:20.589419+0000 (oldest deadline 2024-04-25T00:33:45.289585+0000) 2024-04-25T00:33:46.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:46.500+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:47.111 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:46 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:46.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:47.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:47 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:47.045+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:47.399 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:47 smithi045 podman[99674]: 2024-04-25 00:33:47.111671428 +0000 UTC m=+5.620726844 container died ddffbb85fc27811b377593e21d901451514221dd39c9d4aeb6dae73b8dee368a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:33:47.519 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:47.134+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:47.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:47.519+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:47.850 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:47 smithi045 podman[99674]: 2024-04-25 00:33:47.457936346 +0000 UTC m=+5.966991752 container cleanup ddffbb85fc27811b377593e21d901451514221dd39c9d4aeb6dae73b8dee368a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:33:47.850 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:47 smithi045 bash[99674]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3 2024-04-25T00:33:47.850 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:47 smithi045 podman[99704]: 2024-04-25 00:33:47.667688185 +0000 UTC m=+0.556183937 container remove ddffbb85fc27811b377593e21d901451514221dd39c9d4aeb6dae73b8dee368a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:33:48.135 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:47.995536322 +0000 UTC m=+0.018900686 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:33:48.135 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:47 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:47.848+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:48.370 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:48 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:48.044+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:48.399 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:48.135365709 +0000 UTC m=+0.158730112 container create 9010118dc1dd6647a2b972e12ab51b9bdc60a6dd0dc0ba6527cbe83fd925d2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:33:48.400 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:48.362835705 +0000 UTC m=+0.386200066 container init 9010118dc1dd6647a2b972e12ab51b9bdc60a6dd0dc0ba6527cbe83fd925d2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:33:48.400 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:48.365980166 +0000 UTC m=+0.389344534 container start 9010118dc1dd6647a2b972e12ab51b9bdc60a6dd0dc0ba6527cbe83fd925d2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T00:33:48.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:48.101+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:48.667 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:48.429295891 +0000 UTC m=+0.452660261 container attach 9010118dc1dd6647a2b972e12ab51b9bdc60a6dd0dc0ba6527cbe83fd925d2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T00:33:48.975 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 00:33:48 smithi045 podman[99798]: 2024-04-25 00:33:48.668008946 +0000 UTC m=+0.691373308 container died 9010118dc1dd6647a2b972e12ab51b9bdc60a6dd0dc0ba6527cbe83fd925d2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T00:33:48.975 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:48 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:48.817+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:49.056 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:48.562+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:49.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T00:33:49.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:49.351 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T00:33:49.352 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T00:33:49.352 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.4 2024-04-25T00:33:49.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:49 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:49.075+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:49.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:49.133+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:49.666 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:49 smithi071 systemd[1]: Stopping Ceph osd.4 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:49.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:49.525+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:50.118 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:49 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:49.666+0000 7f5a5fb3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:50.118 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:49 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:49.666+0000 7f5a5fb3f700 -1 osd.4 106 *** Got signal Terminated *** 2024-04-25T00:33:50.118 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:49 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:49.666+0000 7f5a5fb3f700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:50.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:49 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:49.792+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:50.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:50 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:50.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:50.510 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:50.179+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:50.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:50.510+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:51.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:50 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:50.790+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:51.469 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:51 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:51.197+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:51.469 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:51 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:51.197+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:51.618 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:51 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:51.154+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:51.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:51 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:51.468+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:51.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:51.771+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:51.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:51 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:51.551+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:52.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:52.176+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:52.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:52.176+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:52.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:52.021+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:52.369 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:52.097+0000 7f5a5812b700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.731844+0000 front 2024-04-25T00:33:25.731948+0000 (oldest deadline 2024-04-25T00:33:51.632140+0000) 2024-04-25T00:33:52.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:52 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:52.170+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:52.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:52 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:52.469+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:52.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:52.538+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:52.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:52 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:52.788+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:53.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:53.027+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:53.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:53.170+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:53.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:53.170+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:53.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:53 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:53.149+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:53.369 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:53 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:53.060+0000 7f5a5812b700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.731844+0000 front 2024-04-25T00:33:25.731948+0000 (oldest deadline 2024-04-25T00:33:51.632140+0000) 2024-04-25T00:33:53.794 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:53.524+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:53.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:53 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:53.488+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:53.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:53 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:53.547+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:54.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:53.793+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:54.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:53.793+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:54.368 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4[44074]: 2024-04-25T00:33:54.054+0000 7f5a5812b700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.731844+0000 front 2024-04-25T00:33:25.731948+0000 (oldest deadline 2024-04-25T00:33:51.632140+0000) 2024-04-25T00:33:54.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:54.168+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:54.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:54.168+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:54.460 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:54.070+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:54.460 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:54.211+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:54.460 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:54.211+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:54.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:54 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:54.459+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:54.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:54 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:54.504+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:54.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:54.548+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:54.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:54.755+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:54.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:54 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:54.755+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:55.369 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:55 smithi071 podman[68944]: 2024-04-25 00:33:55.054596695 +0000 UTC m=+5.504534542 container died 07e76dab08005a3d3c35efd1e96aa402ba48ac8fa90877de455d8165476525e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:33:55.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:55.142+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:55.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:55.142+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:55.458 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:55.100+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:55.459 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:55.237+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:55.459 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:55.237+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:55.701 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:55 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:55.455+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:55.701 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:55 smithi071 podman[68944]: 2024-04-25 00:33:55.474521634 +0000 UTC m=+5.924459492 container cleanup 07e76dab08005a3d3c35efd1e96aa402ba48ac8fa90877de455d8165476525e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:33:55.701 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:55 smithi071 bash[68944]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4 2024-04-25T00:33:55.804 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:55.533+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:55.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:55 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:55.457+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:56.103 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:55 smithi071 podman[68961]: 2024-04-25 00:33:55.700730304 +0000 UTC m=+0.647880613 container remove 07e76dab08005a3d3c35efd1e96aa402ba48ac8fa90877de455d8165476525e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T00:33:56.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:55.804+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:56.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:55 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:55.804+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:56.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:56 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:56.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:56.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:56 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:56.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:56.369 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.005912205 +0000 UTC m=+0.017808663 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:33:56.369 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.126710997 +0000 UTC m=+0.138607437 container create dddc9ae05dd5e713df206a63c58d5c4aa83a8b2f56dffd16f4d3422b49b7fa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T00:33:56.416 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:56.081+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:56.416 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:56.284+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:56.416 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:56.284+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:56.667 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:56 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:56.504+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:56.667 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.370647555 +0000 UTC m=+0.382543996 container init dddc9ae05dd5e713df206a63c58d5c4aa83a8b2f56dffd16f4d3422b49b7fa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-25T00:33:56.667 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.373662993 +0000 UTC m=+0.385559434 container start dddc9ae05dd5e713df206a63c58d5c4aa83a8b2f56dffd16f4d3422b49b7fa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T00:33:56.667 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.479398578 +0000 UTC m=+0.491295038 container attach dddc9ae05dd5e713df206a63c58d5c4aa83a8b2f56dffd16f4d3422b49b7fa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T00:33:56.801 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:56.533+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:56.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:56.415+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:56.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:56 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:56.415+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:33:56.957 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 25 00:33:56 smithi071 podman[69050]: 2024-04-25 00:33:56.667180399 +0000 UTC m=+0.679076836 container died dddc9ae05dd5e713df206a63c58d5c4aa83a8b2f56dffd16f4d3422b49b7fa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:33:57.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:56.801+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:57.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:56 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:56.801+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:57.266 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:57 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:57.181+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:57.266 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:57 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:57.181+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:57.281 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T00:33:57.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:33:57.361 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T00:33:57.361 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T00:33:57.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.5 2024-04-25T00:33:57.369 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:57.091+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:57.370 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:57.267+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:57.370 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:57.267+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:57.618 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:57 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:57.510+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:57.672 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:57.369+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:57.672 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:57.369+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:33:57.672 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 systemd[1]: Stopping Ceph osd.5 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:33:57.839 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:57.536+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:58.057 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:57.671+0000 7f54a8421700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:33:58.057 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:57.671+0000 7f54a8421700 -1 osd.5 106 *** Got signal Terminated *** 2024-04-25T00:33:58.057 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:57 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:57.671+0000 7f54a8421700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:33:58.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:57.839+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:58.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:57 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:57.839+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:58.545 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:58 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:58.224+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:58.545 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:58 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:58.224+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:58.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:58 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:58.136+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:58.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:58 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:58.330+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:58.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:58 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:58.330+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:33:58.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:58 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:58.241+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:58.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:58 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:58.241+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:58.814 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:58.518+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:58.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:58 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:58.544+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:33:59.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:58.813+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:33:59.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:58.813+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:33:59.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:33:59 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:33:59.143+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:33:59.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:59 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:59.343+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:33:59.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:33:59 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:33:59.343+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:33:59.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:59 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:59.245+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:33:59.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:33:59 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:33:59.245+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:33:59.564 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:59 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:59.204+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:33:59.564 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:33:59 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:33:59.204+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:33:59.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:33:59 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:33:59.469+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:33:59.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:33:59 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:33:59.564+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:00.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:59 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:59.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:00.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:33:59 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:33:59.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:00.518 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:00 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:00.187+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:00.519 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:00 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:00.187+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:00.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:00.110+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:00.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:00.110+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:00.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:00.334+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:34:00.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:00.334+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:34:00.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:00.258+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:00.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:00 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:00.258+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:00.774 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:00.442+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:00.775 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:00.442+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:00.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:00 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:00.518+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:01.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:00.774+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:01.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:00.774+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:01.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:00 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:00.774+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:01.554 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:01 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:01.173+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:01.554 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:01 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:01.173+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:01.554 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:01 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:01.173+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:01.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:01.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:01.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:01.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:01.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:01.380+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:34:01.556 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:01.380+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:34:01.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:01.242+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:01.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:01.242+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:01.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:01 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:01.242+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:01.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:01.395+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:01.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:01.395+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:01.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:01 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:01.553+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:02.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:01.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:02.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:01.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:02.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:01.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:02.412 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:02.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:02.412 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:02.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:02.413 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:02.198+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:02.413 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:02.198+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:02.413 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:02.198+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:02.583 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:02.164+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:02.583 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:02.164+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:02.583 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:02.164+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:02.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:02.351+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:02.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:02.351+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:02.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:02.411+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.465638+0000 front 2024-04-25T00:33:21.465604+0000 (oldest deadline 2024-04-25T00:33:44.365330+0000) 2024-04-25T00:34:02.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:02.411+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.266500+0000 front 2024-04-25T00:33:30.266533+0000 (oldest deadline 2024-04-25T00:33:55.566641+0000) 2024-04-25T00:34:02.806 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:02 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5[45054]: 2024-04-25T00:34:02.411+0000 7f54a0a0d700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:35.567038+0000 front 2024-04-25T00:33:35.567062+0000 (oldest deadline 2024-04-25T00:34:01.467234+0000) 2024-04-25T00:34:02.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:02.583+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:02.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:02 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:02.583+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:03.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:02.719+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:03.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:02.719+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:03.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:02 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:02.719+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:03.306 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:03 smithi122 podman[69750]: 2024-04-25 00:34:03.04698659 +0000 UTC m=+5.490593392 container died b64ea0c5c04cf1988c26e57768c8a7591546f54519523ce2b9b36be2eaac27ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T00:34:03.310 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:03 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:03.149+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:03.310 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:03 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:03.149+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:03.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:03 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:03.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:03.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:03 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:03.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:03.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:03 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:03.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:03.596 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:03 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:03.134+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:03.597 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:03 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:03.134+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:03.597 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:03 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:03.134+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:03.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:03.377+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:03.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:03.377+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:03.731 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:03 smithi122 podman[69750]: 2024-04-25 00:34:03.496592781 +0000 UTC m=+5.940199565 container cleanup b64ea0c5c04cf1988c26e57768c8a7591546f54519523ce2b9b36be2eaac27ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:34:03.731 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:03 smithi122 bash[69750]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5 2024-04-25T00:34:03.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:03 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:03.596+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:03.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:03 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:03.596+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:04.056 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:03 smithi122 podman[69768]: 2024-04-25 00:34:03.731181381 +0000 UTC m=+0.671787999 container remove b64ea0c5c04cf1988c26e57768c8a7591546f54519523ce2b9b36be2eaac27ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:34:04.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:03.733+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:04.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:03.733+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:04.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:03 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:03.733+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:04.410 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:04.103+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:04.410 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:04.103+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:04.411 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:04.181+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:04.411 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:04.181+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:04.411 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:04 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:04.181+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:04.411 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.074874579 +0000 UTC m=+0.018064808 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:04.411 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.190821052 +0000 UTC m=+0.134011268 container create b3322a24fc452edc26b101d8561d4d7d4f057ae8c7422c94641eefa228ca5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:34:04.576 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:04.165+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:04.576 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:04.165+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:04.576 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:04.165+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:04.700 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.410505383 +0000 UTC m=+0.353695607 container init b3322a24fc452edc26b101d8561d4d7d4f057ae8c7422c94641eefa228ca5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:34:04.701 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.414095538 +0000 UTC m=+0.357285754 container start b3322a24fc452edc26b101d8561d4d7d4f057ae8c7422c94641eefa228ca5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T00:34:04.701 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.510081678 +0000 UTC m=+0.453271918 container attach b3322a24fc452edc26b101d8561d4d7d4f057ae8c7422c94641eefa228ca5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:34:04.776 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:04.406+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:04.776 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:04.406+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:04.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:04.575+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:04.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:04 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:04.575+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:05.012 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 25 00:34:04 smithi122 podman[69856]: 2024-04-25 00:34:04.701122971 +0000 UTC m=+0.644313180 container died b3322a24fc452edc26b101d8561d4d7d4f057ae8c7422c94641eefa228ca5f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:34:05.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:04.775+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:05.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:04.775+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:05.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:04 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:04.775+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:05.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:05 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:05.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:05.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:05 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:05.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:05.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:05 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:05.152+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:05.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:05 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:05.152+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:05.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:05 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:05.152+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:05.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T00:34:05.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:05.419 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T00:34:05.419 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T00:34:05.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.6 2024-04-25T00:34:05.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:05 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:05.209+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:05.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:05 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:05.209+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:05.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:05 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:05.209+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:05.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:05.384+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:05.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:05.384+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:05.649 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 systemd[1]: Stopping Ceph osd.6 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:05.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:05 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:05.580+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:05.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:05 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:05.580+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:06.149 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:05.734+0000 7f8456f8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:06.149 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:05.734+0000 7f8456f8f700 -1 osd.6 106 *** Got signal Terminated *** 2024-04-25T00:34:06.149 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:05.734+0000 7f8456f8f700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:06.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:05.768+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:06.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:05.768+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:06.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:05 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:05.768+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:06.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:06 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:06.113+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:06.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:06 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:06.113+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:06.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:06 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:06.113+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:06.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:06 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:06.096+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:06.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:06 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:06.096+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:06.615 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:06 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:06.161+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:06.615 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:06 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:06.161+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:06.615 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:06 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:06.161+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:06.733 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:06.426+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:06.733 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:06.426+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:06.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:06 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:06.614+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:06.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:06 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:06.614+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:07.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:06.732+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:07.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:06.732+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:07.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:06.732+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:07.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:07 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:07.136+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:07.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:07 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:07.136+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:07.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:07 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:07.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:07.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:07 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:07.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:07.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:07 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:07.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:07.573 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:07.167+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:07.573 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:07.167+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:07.574 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:07.167+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:07.737 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:07 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:07.437+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:07.737 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:07 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:07.437+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:07.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:07.573+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:07.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:07 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:07.573+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:08.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:07 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:07.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:08.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:07 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:07.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:08.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:07 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:07.736+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:08.555 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:08.160+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:08.555 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:08.160+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:08.555 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:08.160+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:08.555 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:08.160+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:08.126+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:08.126+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:08.126+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:08.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:08.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:08.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:08 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:08.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:08.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:08.554+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:08.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:08.554+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:08.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:08 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:08.554+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:08.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:08 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:08.464+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:08.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:08 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:08.464+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:08.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:08 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:08.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:08.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:08 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:08.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:08.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:08 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:08.707+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:09.538 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:09.128+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:09.538 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:09.128+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:09.538 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:09.128+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:09.538 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:09.128+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:09.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:09.147+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:09.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:09.147+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:09.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:09.147+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:09.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:09.094+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:09.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:09.094+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:09.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:09 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:09.094+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:09.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:09.537+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:09.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:09.537+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:09.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:09 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:09.537+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:09.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:09.667+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:09.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:09.667+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:09.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:09.667+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:09.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:09.491+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:09.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:09.491+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:09.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:09 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:09.491+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:43.837236+0000 front 2024-04-25T00:33:43.837272+0000 (oldest deadline 2024-04-25T00:34:09.137441+0000) 2024-04-25T00:34:10.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:10.105+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:10.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:10.105+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:10.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:10.105+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:10.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:10.140+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:10.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:10.140+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:10.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:10 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:10.140+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:10.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:10.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:10.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:10.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:10.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:10.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:10.581 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:10.131+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:10.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:10.581+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:10.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:10.581+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:10.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:10.581+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:10.452+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.534551+0000 front 2024-04-25T00:33:30.534692+0000 (oldest deadline 2024-04-25T00:33:51.034424+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:10.452+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.535553+0000 front 2024-04-25T00:33:34.535771+0000 (oldest deadline 2024-04-25T00:33:59.835425+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6[54873]: 2024-04-25T00:34:10.452+0000 7f844f57b700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:43.837236+0000 front 2024-04-25T00:33:43.837272+0000 (oldest deadline 2024-04-25T00:34:09.137441+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:10.662+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:10.662+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:10.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:10 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:10.662+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:11.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:11.092+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:11.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:11.092+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:11.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:11.092+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:11.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:11.092+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:11.399 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:11 smithi045 podman[99905]: 2024-04-25 00:34:11.065928354 +0000 UTC m=+5.445777721 container died 07cddafa62498c796bf9268150a52a41a4e1c93a2eb2240c1bcc608dac08e86a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T00:34:11.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:11.122+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:11.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:11.122+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:11.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:11.122+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:11.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:11.065+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:11.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:11.065+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:11.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:11 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:11.065+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:11.678 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:11 smithi045 podman[99905]: 2024-04-25 00:34:11.584755736 +0000 UTC m=+5.964605093 container cleanup 07cddafa62498c796bf9268150a52a41a4e1c93a2eb2240c1bcc608dac08e86a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:34:11.678 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:11 smithi045 bash[99905]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6 2024-04-25T00:34:12.102 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:11.621+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:12.103 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:11.621+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:12.103 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:11 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:11.621+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:12.149 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:11 smithi045 podman[99922]: 2024-04-25 00:34:11.811171607 +0000 UTC m=+0.746493330 container remove 07cddafa62498c796bf9268150a52a41a4e1c93a2eb2240c1bcc608dac08e86a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:34:12.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:11 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:11.676+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:12.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:11 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:11.676+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:12.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:11 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:11.676+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:12.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:11 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:11.676+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:12.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.019+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:12.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.019+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:12.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.019+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:12.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:12.144+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:12.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:12.144+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:12.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:12.144+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:12.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:12.102+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:12.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:12.102+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:12.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:12.102+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:12.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:12.102+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:12.614 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.174321141 +0000 UTC m=+0.018385789 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:12.614 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.303645879 +0000 UTC m=+0.147710536 container create 372b5dc20d6b7bbd5b193ef0946b771ba559fedcdaf43c889eb3ac0844941783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:34:12.614 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.514170652 +0000 UTC m=+0.358235300 container init 372b5dc20d6b7bbd5b193ef0946b771ba559fedcdaf43c889eb3ac0844941783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:34:12.614 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.517339126 +0000 UTC m=+0.361403777 container start 372b5dc20d6b7bbd5b193ef0946b771ba559fedcdaf43c889eb3ac0844941783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:34:12.614 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.614140079 +0000 UTC m=+0.458204739 container attach 372b5dc20d6b7bbd5b193ef0946b771ba559fedcdaf43c889eb3ac0844941783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:34:12.899 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 00:34:12 smithi045 podman[100012]: 2024-04-25 00:34:12.814205919 +0000 UTC m=+0.658270569 container died 372b5dc20d6b7bbd5b193ef0946b771ba559fedcdaf43c889eb3ac0844941783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:34:12.899 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:12 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:12.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:12.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:12 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:12.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:12.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:12 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:12.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:12.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:12 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:12.706+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:13.074 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:12.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:13.074 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:12.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:13.074 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:12 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:12.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:13.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:13.164+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:13.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:13.164+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:13.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:13.164+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:13.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.993+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:13.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.993+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:13.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:12 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:12.993+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:13.370 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:13.074+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:13.370 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:13.074+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:13.371 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:13.074+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:13.371 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:13.074+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:13.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T00:34:13.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:13.657 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T00:34:13.658 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T00:34:13.658 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.7 2024-04-25T00:34:13.884 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:13 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:13.756+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:13.884 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:13 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:13.756+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:13.884 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:13 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:13.756+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:13.884 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:13 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:13.756+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:13.977 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.630+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:13.977 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.630+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:13.977 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.630+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:13.978 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 systemd[1]: Stopping Ceph osd.7 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:14.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:14 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:14.199+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:14.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:14 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:14.199+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:14.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:14 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:14.199+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:14.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:13.986+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:14.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:13.986+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:14.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:13 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:13.986+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.976+0000 7fbc76825700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.976+0000 7fbc76825700 -1 osd.7 106 *** Got signal Terminated *** 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:13 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:13.976+0000 7fbc76825700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:14.088+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:14.088+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:14.088+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:14.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:14.088+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:15.119 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:14.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:15.119 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:14.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:15.119 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:14.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:15.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:14.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:15.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:14.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:15.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:14.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:15.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:14 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:14.776+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:15.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:15.029+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:15.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:15.029+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:15.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:15.029+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:15.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:15.151+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:15.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:15.151+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:15.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:15 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:15.151+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:15.551 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:15.130+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:15.552 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:15.130+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:15.552 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:15.130+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:15.552 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:15.130+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:15.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:15.608+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:15.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:15.608+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:15.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:15 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:15.608+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:16.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:15 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:15.799+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:16.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:15 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:15.799+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:16.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:15 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:15.799+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:16.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:15 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:15.799+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:16.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:16.104+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:16.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:16.104+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:16.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:16.104+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:16.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:16.104+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:16.403 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:16.077+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:16.404 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:16.077+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:16.404 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:16.077+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:16.404 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:16.194+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:16.404 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:16.194+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:16.404 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:16 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:16.194+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:16.868 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:16.613+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:16.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:16.613+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:16.869 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:16 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:16.613+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:17.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:16 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:16.802+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:17.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:16 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:16.802+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:17.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:16 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:16.802+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:17.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:16 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:16.802+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:17.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:17.097+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:17.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:17.097+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:17.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:17.097+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:17.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:17.097+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:17.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:17.097+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:17.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:17.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:17.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:17.148+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:17.148+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:17.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:17.148+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:17.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:17 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:17.148+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:18.064 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:17.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:18.064 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:17.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:18.065 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:17 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:17.650+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:18.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:17.778+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:18.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:17.778+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:18.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:17.778+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:18.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:17.778+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:18.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:17 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:17.778+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:18.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:18.064+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:18.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:18.064+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:18.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:18.064+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:18.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:18.064+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:18.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:18.064+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:18.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:18.137+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:18.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:18.137+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:18.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:18.137+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:18.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:18.121+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:18.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:18.121+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:18.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:18.121+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:18.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:18 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:18.121+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:19.044 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:18.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:28.556986+0000 front 2024-04-25T00:33:28.557045+0000 (oldest deadline 2024-04-25T00:33:53.257197+0000) 2024-04-25T00:34:19.044 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:18.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:38.458561+0000 front 2024-04-25T00:33:38.458602+0000 (oldest deadline 2024-04-25T00:34:02.459331+0000) 2024-04-25T00:34:19.044 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7[50649]: 2024-04-25T00:34:18.644+0000 7fbc6ee11700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:42.459857+0000 front 2024-04-25T00:33:42.459916+0000 (oldest deadline 2024-04-25T00:34:08.359948+0000) 2024-04-25T00:34:19.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:18.779+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:19.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:18.779+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:19.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:18.779+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:19.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:18.779+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:19.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:18 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:18.779+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:19.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:19 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:19.043+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:19.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:19 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:19.043+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:19.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:19 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:19.043+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:19.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:19 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:19.043+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:19.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:19 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:19.043+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:19.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:19.180+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:19.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:19.180+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:19.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:19.180+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:19.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:19.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:19.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:19.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:19.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:19.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:19.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:19 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:19.128+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:19.729 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:19 smithi071 podman[69154]: 2024-04-25 00:34:19.394459199 +0000 UTC m=+5.548885349 container died a59733ba9b66ffe4cb4b9285986d00d2a839ffba7d3c4b00b431c08d033f2c74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:34:20.037 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:19 smithi071 podman[69154]: 2024-04-25 00:34:19.836505714 +0000 UTC m=+5.990931861 container cleanup a59733ba9b66ffe4cb4b9285986d00d2a839ffba7d3c4b00b431c08d033f2c74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T00:34:20.038 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:19 smithi071 bash[69154]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7 2024-04-25T00:34:20.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:19 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:19.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:20.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:19 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:19.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:20.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:19 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:19.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:20.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:19 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:19.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:20.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:19 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:19.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:20.368 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69184]: 2024-04-25 00:34:20.037585235 +0000 UTC m=+0.632734338 container remove a59733ba9b66ffe4cb4b9285986d00d2a839ffba7d3c4b00b431c08d033f2c74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-25T00:34:20.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:20 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:20.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:20.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:20 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:20.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:20.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:20 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:20.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:20.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:20 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:20.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:20.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:20 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:20.039+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:20.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:20.123+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:20.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:20.123+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:20.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:20.123+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:20.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:20.123+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:20.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:20.179+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:20.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:20.179+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:20.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:20.179+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:20.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:20 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:20.179+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:20.724 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69279]: 2024-04-25 00:34:20.348027032 +0000 UTC m=+0.017961100 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:20.725 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69279]: 2024-04-25 00:34:20.480362121 +0000 UTC m=+0.150296213 container create fcf97fb46f65c95003698900685bb6c39c04dfae3e67f5124dd43bbd6d9b558f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-25T00:34:20.725 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69279]: 2024-04-25 00:34:20.72465321 +0000 UTC m=+0.394587269 container init fcf97fb46f65c95003698900685bb6c39c04dfae3e67f5124dd43bbd6d9b558f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:34:21.018 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69279]: 2024-04-25 00:34:20.727708098 +0000 UTC m=+0.397642157 container start fcf97fb46f65c95003698900685bb6c39c04dfae3e67f5124dd43bbd6d9b558f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:34:21.019 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:20 smithi071 podman[69279]: 2024-04-25 00:34:20.833189027 +0000 UTC m=+0.503123090 container attach fcf97fb46f65c95003698900685bb6c39c04dfae3e67f5124dd43bbd6d9b558f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:34:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:20.791+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:20.791+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:20.791+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:20.791+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:20 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:20.791+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:21.328 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 25 00:34:21 smithi071 podman[69279]: 2024-04-25 00:34:21.021792539 +0000 UTC m=+0.691726590 container died fcf97fb46f65c95003698900685bb6c39c04dfae3e67f5124dd43bbd6d9b558f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:34:21.328 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:21 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:21.017+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:21.328 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:21 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:21.017+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:21.328 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:21 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:21.017+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:21.329 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:21 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:21.017+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:21.329 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:21 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:21.017+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.097+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.097+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.097+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.097+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:21.213+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:21.213+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:21.213+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:21.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:21 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:21.213+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:21.652 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T00:34:21.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:21.696 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T00:34:21.696 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T00:34:21.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.8 2024-04-25T00:34:22.000 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:21 smithi122 systemd[1]: Stopping Ceph osd.8 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:22.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:21.749+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:22.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:21.749+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:22.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:21.749+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:22.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:21.749+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:22.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:21 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:21.749+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:22.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:22.190+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:22.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:22.190+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:22.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:22.190+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:22.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:22.190+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:22.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.999+0000 7ff04c7e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.999+0000 7ff04c7e8700 -1 osd.8 106 *** Got signal Terminated *** 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:21.999+0000 7ff04c7e8700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:22.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:22.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:22.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:22.307 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:22 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:22.141+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:22.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:22.034+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:22.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:22.034+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:22.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:22.034+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:22.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:22.034+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:22.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:22.034+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:23.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:22.789+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:23.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:22.789+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:23.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:22.789+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:23.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:22.789+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:23.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:22 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:22.789+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:23.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:23 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:23.040+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:23.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:23 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:23.040+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:23.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:23 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:23.040+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:23.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:23 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:23.040+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:23.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:23 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:23.040+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:23.235+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:23.235+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:23.235+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:23.235+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:23.114+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:23.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:23.114+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:23.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:23.114+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:23.557 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:23 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:23.114+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:24.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:23.806+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:24.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:23.806+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:24.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:23.806+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:24.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:23.806+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:24.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:23 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:23.806+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:24.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:24 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:24.067+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:24.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:24 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:24.067+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:24.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:24 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:24.067+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:24.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:24 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:24.067+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:24.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:24 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:24.067+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:24.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:24.088+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:24.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:24.089+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:24.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:24.089+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:24.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:24.089+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:24.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:24.089+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.316531+0000 front 2024-04-25T00:34:02.316488+0000 (oldest deadline 2024-04-25T00:34:24.016567+0000) 2024-04-25T00:34:24.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:24.218+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:24.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:24.218+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:24.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:24.218+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:24.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:24.218+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:24.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:24 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:24.218+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:25.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:24.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:25.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:24.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:25.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:24.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:25.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:24.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:25.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:24 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:24.796+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:25.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:25 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:25.024+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:25.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:25.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:25.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:25.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:25.115+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.316531+0000 front 2024-04-25T00:34:02.316488+0000 (oldest deadline 2024-04-25T00:34:24.016567+0000) 2024-04-25T00:34:25.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:25.206+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:25.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:25.206+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:25.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:25.206+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:25.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:25.206+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:25.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:25 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:25.206+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:26.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:25.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:26.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:25.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:26.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:25.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:26.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:25.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:26.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:25 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:25.807+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:26.203 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:26.202+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:26.203 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:26.202+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:26.203 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:26.153+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:25.809232+0000 front 2024-04-25T00:33:25.809254+0000 (oldest deadline 2024-04-25T00:33:51.109415+0000) 2024-04-25T00:34:26.203 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:26.153+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.411605+0000 front 2024-04-25T00:33:37.411626+0000 (oldest deadline 2024-04-25T00:33:59.711958+0000) 2024-04-25T00:34:26.204 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:26.153+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.313199+0000 front 2024-04-25T00:33:44.313275+0000 (oldest deadline 2024-04-25T00:34:07.213291+0000) 2024-04-25T00:34:26.204 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:26.153+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:53.015027+0000 front 2024-04-25T00:33:53.014863+0000 (oldest deadline 2024-04-25T00:34:16.514852+0000) 2024-04-25T00:34:26.204 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8[51925]: 2024-04-25T00:34:26.153+0000 7ff044dd4700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.316531+0000 front 2024-04-25T00:34:02.316488+0000 (oldest deadline 2024-04-25T00:34:24.016567+0000) 2024-04-25T00:34:26.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:26.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:26.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:26.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:26.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:26.370 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.007+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:26.474 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:26.202+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:26.474 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:26.202+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:26.474 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:26.202+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:27.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:26 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:26.829+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:27.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:26 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:26.829+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:27.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:26 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:26.829+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:27.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:26 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:26.829+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:27.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:26 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:26.829+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:27.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:26 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:26.965+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:27.528 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:27 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:27.168+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:27.528 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:27 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:27.168+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:27.528 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:27 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:27.168+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:27.528 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:27 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:27.168+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:27.528 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:27 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:27.168+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:27.801 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:27 smithi122 podman[69973]: 2024-04-25 00:34:27.528291041 +0000 UTC m=+5.645018470 container died 28c82bc731d9e0597cb57e475a61c053e42bb58d971d4586fba2acfe8b8aa6d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:34:28.056 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:27 smithi122 podman[69973]: 2024-04-25 00:34:27.875123599 +0000 UTC m=+5.991851040 container cleanup 28c82bc731d9e0597cb57e475a61c053e42bb58d971d4586fba2acfe8b8aa6d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T00:34:28.056 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:27 smithi122 bash[69973]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8 2024-04-25T00:34:28.056 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[69990]: 2024-04-25 00:34:28.034726894 +0000 UTC m=+0.501063268 container remove 28c82bc731d9e0597cb57e475a61c053e42bb58d971d4586fba2acfe8b8aa6d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:28.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:27 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:27.865+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:28.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:28.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:28.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:28.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:28.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:28.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:27 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:27.994+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:28.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:28 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:28.151+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:28.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:28 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:28.151+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:28.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:28 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:28.151+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:28.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:28 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:28.151+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:28.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:28 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:28.151+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:28.696 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.342142718 +0000 UTC m=+0.018301803 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:28.696 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.444047977 +0000 UTC m=+0.120207108 container create 5f999d64dd27e24a15da70f5e5466ccdde39c94aa84e51952ee83f4a911adece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:34:28.697 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.696780833 +0000 UTC m=+0.372939918 container init 5f999d64dd27e24a15da70f5e5466ccdde39c94aa84e51952ee83f4a911adece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:34:28.983 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.699933402 +0000 UTC m=+0.376092488 container start 5f999d64dd27e24a15da70f5e5466ccdde39c94aa84e51952ee83f4a911adece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T00:34:28.983 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.780059015 +0000 UTC m=+0.456218100 container attach 5f999d64dd27e24a15da70f5e5466ccdde39c94aa84e51952ee83f4a911adece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T00:34:29.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:29.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:29.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:29.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:29.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:29.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:28 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:28.845+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:29.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 25 00:34:28 smithi122 podman[70080]: 2024-04-25 00:34:28.983475132 +0000 UTC m=+0.659634220 container died 5f999d64dd27e24a15da70f5e5466ccdde39c94aa84e51952ee83f4a911adece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:34:29.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:29 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:29.147+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:29.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:29 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:29.147+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:29.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:29 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:29.147+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:29.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:29 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:29.147+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:29.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:29 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:29.147+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:29.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:29.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:29.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:29.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:29.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:29.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:29 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:29.022+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:29.673 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T00:34:29.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:29.711 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T00:34:29.711 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T00:34:29.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.9 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 systemd[1]: Stopping Ceph osd.9 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:30.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:29 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:29.871+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:30.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:30.100+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:30.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:30.100+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:30.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:30.100+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:30.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:30.100+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:30.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:30.100+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:30.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:30 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:30.058+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:30.649 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.169+0000 7f96c2b0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:30.649 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.169+0000 7f96c2b0d700 -1 osd.9 106 *** Got signal Terminated *** 2024-04-25T00:34:30.649 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.169+0000 7f96c2b0d700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:31.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:30 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:30.847+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:31.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:31 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:31.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:31.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:31 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:31.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:31.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:31 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:31.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:31.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:31 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:31.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:31.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:31 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:31.082+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:31.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:31 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:31.069+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:32.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:32.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:32.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:32.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:32.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:32.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:31 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:31.846+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:32.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:32 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:32.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:32.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:32 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:32.116+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:33.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:33.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:33.149 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:33.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:33.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:33.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:32 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:32.892+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:33.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:33.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:33.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:33.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:33.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:33.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:33 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:33.080+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:33.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:33.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:33.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:33.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:33.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:33.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:33 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:33.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:34.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:34.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:34.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:34.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:34.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:34.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:33 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:33.934+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:34.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:34.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:34.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:34.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:34.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:34.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:34.154+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:34.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:34 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:34.118+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:35.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:22.176989+0000 front 2024-04-25T00:33:22.177030+0000 (oldest deadline 2024-04-25T00:33:45.077183+0000) 2024-04-25T00:34:35.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:26.778314+0000 front 2024-04-25T00:33:26.778348+0000 (oldest deadline 2024-04-25T00:33:53.179285+0000) 2024-04-25T00:34:35.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:34.180864+0000 front 2024-04-25T00:33:34.180883+0000 (oldest deadline 2024-04-25T00:34:00.080900+0000) 2024-04-25T00:34:35.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:45.982069+0000 front 2024-04-25T00:33:45.982003+0000 (oldest deadline 2024-04-25T00:34:11.282147+0000) 2024-04-25T00:34:35.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:51.282609+0000 front 2024-04-25T00:33:51.282763+0000 (oldest deadline 2024-04-25T00:34:17.083305+0000) 2024-04-25T00:34:35.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:34 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9[61917]: 2024-04-25T00:34:34.904+0000 7f96bb0f9700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:02.384601+0000 front 2024-04-25T00:34:02.384496+0000 (oldest deadline 2024-04-25T00:34:27.684412+0000) 2024-04-25T00:34:35.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:35.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:35.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:35.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:35.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:35.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:35 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:35.117+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:35.618 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:35.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:35.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:35.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:35.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:35.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:35 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:35.123+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:36.062 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:35 smithi045 podman[100129]: 2024-04-25 00:34:35.690798698 +0000 UTC m=+5.779227426 container died 4dc97ebc6ecf870a1696a93fe23a0c70818c29bb6457582bfc6954c50af5782b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:36.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:36 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:36.108+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:36.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:36 smithi045 podman[100129]: 2024-04-25 00:34:36.152650245 +0000 UTC m=+6.241078979 container cleanup 4dc97ebc6ecf870a1696a93fe23a0c70818c29bb6457582bfc6954c50af5782b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T00:34:36.399 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:36 smithi045 bash[100129]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:36.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:36 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:36.074+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:36.741 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:36 smithi045 podman[100146]: 2024-04-25 00:34:36.420183818 +0000 UTC m=+0.723269079 container remove 4dc97ebc6ecf870a1696a93fe23a0c70818c29bb6457582bfc6954c50af5782b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:34:36.995 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:36 smithi045 podman[100241]: 2024-04-25 00:34:36.645402294 +0000 UTC m=+0.018551284 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:36.995 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:36 smithi045 podman[100241]: 2024-04-25 00:34:36.896041742 +0000 UTC m=+0.269190719 container create 851748c82cea38bbb76d031e6eea495b4f9ab450681f61e7a8bd2770b04c04f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T00:34:37.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:37 smithi045 podman[100241]: 2024-04-25 00:34:37.115094287 +0000 UTC m=+0.488243279 container init 851748c82cea38bbb76d031e6eea495b4f9ab450681f61e7a8bd2770b04c04f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:34:37.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:37 smithi045 podman[100241]: 2024-04-25 00:34:37.118639643 +0000 UTC m=+0.491788620 container start 851748c82cea38bbb76d031e6eea495b4f9ab450681f61e7a8bd2770b04c04f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:34:37.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:37 smithi045 podman[100241]: 2024-04-25 00:34:37.206915936 +0000 UTC m=+0.580064918 container attach 851748c82cea38bbb76d031e6eea495b4f9ab450681f61e7a8bd2770b04c04f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:34:37.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:37.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:37.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:37.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:37.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:37.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:37 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:37.104+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:37.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:37 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:37.144+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:37.681 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 00:34:37 smithi045 podman[100241]: 2024-04-25 00:34:37.435296927 +0000 UTC m=+0.808445899 container died 851748c82cea38bbb76d031e6eea495b4f9ab450681f61e7a8bd2770b04c04f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T00:34:38.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T00:34:38.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:38.059 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T00:34:38.059 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T00:34:38.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.10 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 systemd[1]: Stopping Ceph osd.10 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:38.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.159+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:38.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:38.127+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:38.869 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.429+0000 7fded7b0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:38.869 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.429+0000 7fded7b0a700 -1 osd.10 106 *** Got signal Terminated *** 2024-04-25T00:34:38.869 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:38 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:38.429+0000 7fded7b0a700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:39.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:39 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:39.116+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:39.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:39 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:39.102+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:40.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:40.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:40 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:40.093+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:40.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:40.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:40.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:40.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:40.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:40.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:40 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:40.126+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:41.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:41.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:41.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:41.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:41.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:41.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:41.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:41 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:41.166+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:41.618 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:41.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:41 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:41.124+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:42.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:42 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:42.084+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:42.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:42.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:42.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:42.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:42.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:42.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:42.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:42 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:42.216+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:43.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:20.973585+0000 front 2024-04-25T00:33:20.973530+0000 (oldest deadline 2024-04-25T00:33:43.174213+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:30.576874+0000 front 2024-04-25T00:33:30.576845+0000 (oldest deadline 2024-04-25T00:33:54.076995+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:37.578061+0000 front 2024-04-25T00:33:37.578075+0000 (oldest deadline 2024-04-25T00:34:00.477950+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T00:33:44.578806+0000 front 2024-04-25T00:33:44.578707+0000 (oldest deadline 2024-04-25T00:34:07.478820+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.480252+0000 front 2024-04-25T00:33:54.480325+0000 (oldest deadline 2024-04-25T00:34:16.780362+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.281393+0000 front 2024-04-25T00:34:00.281500+0000 (oldest deadline 2024-04-25T00:34:24.381354+0000) 2024-04-25T00:34:43.369 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10[57526]: 2024-04-25T00:34:43.117+0000 7fded40fe700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:06.082517+0000 front 2024-04-25T00:34:06.082555+0000 (oldest deadline 2024-04-25T00:34:35.483025+0000) 2024-04-25T00:34:43.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:43.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:43.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:43.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:43.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:43.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:43.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:43 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:43.261+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:44.119 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:43 smithi071 podman[69383]: 2024-04-25 00:34:43.82353862 +0000 UTC m=+5.570364051 container died fda36a70a651dba1108c77a31cb21fc1bf6ba2ce7fb385098b7c41097d452c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:44.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:44.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:44 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:44.271+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:44.618 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69383]: 2024-04-25 00:34:44.146686456 +0000 UTC m=+5.893511901 container cleanup fda36a70a651dba1108c77a31cb21fc1bf6ba2ce7fb385098b7c41097d452c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:34:44.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 bash[69383]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10 2024-04-25T00:34:44.619 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69400]: 2024-04-25 00:34:44.35639029 +0000 UTC m=+0.531341585 container remove fda36a70a651dba1108c77a31cb21fc1bf6ba2ce7fb385098b7c41097d452c06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T00:34:45.068 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69492]: 2024-04-25 00:34:44.667054549 +0000 UTC m=+0.017915094 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:45.068 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69492]: 2024-04-25 00:34:44.773919036 +0000 UTC m=+0.124779569 container create 740ea31b8c58cb71ac4c24dad70d6288996fee6e0d8e7d78c3188a54cf82a529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True) 2024-04-25T00:34:45.068 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69492]: 2024-04-25 00:34:44.976145747 +0000 UTC m=+0.327006280 container init 740ea31b8c58cb71ac4c24dad70d6288996fee6e0d8e7d78c3188a54cf82a529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:34:45.068 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:44 smithi071 podman[69492]: 2024-04-25 00:34:44.979506996 +0000 UTC m=+0.330367532 container start 740ea31b8c58cb71ac4c24dad70d6288996fee6e0d8e7d78c3188a54cf82a529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T00:34:45.068 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:45 smithi071 podman[69492]: 2024-04-25 00:34:45.067835702 +0000 UTC m=+0.418696250 container attach 740ea31b8c58cb71ac4c24dad70d6288996fee6e0d8e7d78c3188a54cf82a529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:34:45.368 INFO:journalctl@ceph.osd.10.smithi071.stdout:Apr 25 00:34:45 smithi071 podman[69492]: 2024-04-25 00:34:45.267394249 +0000 UTC m=+0.618254783 container died 740ea31b8c58cb71ac4c24dad70d6288996fee6e0d8e7d78c3188a54cf82a529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:34:45.640 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:45.640 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:45.640 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:45.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:45.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:45.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:45.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:45.307+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:45.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T00:34:45.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:45.890 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T00:34:45.890 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T00:34:45.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.11 2024-04-25T00:34:46.187 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:45 smithi122 systemd[1]: Stopping Ceph osd.11 for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.186+0000 7f1198b4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.186+0000 7f1198b4c700 -1 osd.11 106 *** Got signal Terminated *** 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.186+0000 7f1198b4c700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:46.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:46.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:46.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:46.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:46.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:46 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:46.298+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:47.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:47.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:47.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:47.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:47.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:47.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:47.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:47 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:47.270+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:48.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:48.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:48.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:48.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:48.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:48.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:48.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:48 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:48.260+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:49.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:49.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:49.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:49.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:49.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:49.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:49.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:49 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:49.223+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:50.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T00:33:21.850876+0000 front 2024-04-25T00:33:21.850860+0000 (oldest deadline 2024-04-25T00:33:44.150944+0000) 2024-04-25T00:34:50.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-04-25T00:33:29.352425+0000 front 2024-04-25T00:33:29.352371+0000 (oldest deadline 2024-04-25T00:33:51.052397+0000) 2024-04-25T00:34:50.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-25T00:33:36.254080+0000 front 2024-04-25T00:33:36.254103+0000 (oldest deadline 2024-04-25T00:34:00.954139+0000) 2024-04-25T00:34:50.556 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-04-25T00:33:54.158019+0000 front 2024-04-25T00:33:54.158121+0000 (oldest deadline 2024-04-25T00:34:20.057997+0000) 2024-04-25T00:34:50.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-25T00:34:00.058391+0000 front 2024-04-25T00:34:00.058438+0000 (oldest deadline 2024-04-25T00:34:24.158480+0000) 2024-04-25T00:34:50.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T00:34:10.059697+0000 front 2024-04-25T00:34:10.059717+0000 (oldest deadline 2024-04-25T00:34:31.759551+0000) 2024-04-25T00:34:50.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-04-25T00:34:17.060883+0000 front 2024-04-25T00:34:17.060963+0000 (oldest deadline 2024-04-25T00:34:41.160753+0000) 2024-04-25T00:34:50.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:50 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11[58520]: 2024-04-25T00:34:50.177+0000 7f1191138700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-25T00:34:23.362314+0000 front 2024-04-25T00:34:23.362343+0000 (oldest deadline 2024-04-25T00:34:49.262403+0000) 2024-04-25T00:34:51.806 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:51 smithi122 podman[70210]: 2024-04-25 00:34:51.533429123 +0000 UTC m=+5.459043138 container died 5c0f846674773c03f444d5e6f7dac0ab6d9dd8014934a59f39f289e0e9a444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:34:52.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:51 smithi122 podman[70210]: 2024-04-25 00:34:51.855589799 +0000 UTC m=+5.781203814 container cleanup 5c0f846674773c03f444d5e6f7dac0ab6d9dd8014934a59f39f289e0e9a444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:34:52.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:51 smithi122 bash[70210]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11 2024-04-25T00:34:52.306 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70228]: 2024-04-25 00:34:52.040019212 +0000 UTC m=+0.504643806 container remove 5c0f846674773c03f444d5e6f7dac0ab6d9dd8014934a59f39f289e0e9a444ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:34:52.776 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.364515149 +0000 UTC m=+0.017860661 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:34:52.777 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.474253189 +0000 UTC m=+0.127598694 container create 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:34:52.777 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.693434274 +0000 UTC m=+0.346779778 container init 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:34:52.777 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.696603701 +0000 UTC m=+0.349949206 container start 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:34:52.777 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.776674191 +0000 UTC m=+0.430019698 container attach 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:34:53.056 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:52 smithi122 podman[70316]: 2024-04-25 00:34:52.991841165 +0000 UTC m=+0.645186677 container died 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T00:34:53.509 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T00:34:53.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:53 smithi122 podman[70342]: 2024-04-25 00:34:53.488486171 +0000 UTC m=+0.491783774 container remove 5adf25e98ad34323d5d77e8e13bd448e5c1bf4240c8e5d581e54af5717177181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T00:34:53.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:53 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-25T00:34:53.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:53 smithi122 systemd[1]: Stopped Ceph osd.11 for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:34:53.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 25 00:34:53 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@osd.11.service: Consumed 17.145s CPU time. 2024-04-25T00:34:53.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:53.545 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T00:34:53.546 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T00:34:53.546 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.a 2024-04-25T00:34:53.880 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:53 smithi045 systemd[1]: Stopping Ceph mds.a for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:54.149 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: 2024-04-25T00:34:53.879+0000 7f150c5e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:54.149 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:53 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a[68918]: 2024-04-25T00:34:53.879+0000 7f150c5e7700 -1 mds.a *** got signal Terminated *** 2024-04-25T00:34:57.247 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:56 smithi045 podman[100370]: 2024-04-25 00:34:56.990513282 +0000 UTC m=+3.242590228 container died 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:34:57.513 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:57 smithi045 podman[100370]: 2024-04-25 00:34:57.336683315 +0000 UTC m=+3.588760267 container cleanup 36bc0840dc9814f33209f5068251fb185c4f6c5f6de1f37cc556c7f3ecd50833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T00:34:57.513 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 00:34:57 smithi045 bash[100370]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-a 2024-04-25T00:34:57.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T00:34:57.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:34:57.767 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T00:34:57.768 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T00:34:57.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.d 2024-04-25T00:34:58.090 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:34:57 smithi045 systemd[1]: Stopping Ceph mds.d for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:34:58.399 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:34:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d[67966]: 2024-04-25T00:34:58.089+0000 7fb4ac349700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:34:58.399 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:34:58 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d[67966]: 2024-04-25T00:34:58.089+0000 7fb4ac349700 -1 mds.d *** got signal Terminated *** 2024-04-25T00:35:01.031 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:35:00 smithi045 podman[100493]: 2024-04-25 00:35:00.726490729 +0000 UTC m=+2.735485840 container died 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:35:01.032 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:35:01 smithi045 podman[100493]: 2024-04-25 00:35:01.031781241 +0000 UTC m=+3.040776361 container cleanup 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:35:01.371 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:35:01 smithi045 bash[100493]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d 2024-04-25T00:35:01.371 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 00:35:01 smithi045 podman[100507]: 2024-04-25 00:35:01.249537497 +0000 UTC m=+0.520472956 container remove 8b9832431b5cf82df8e2ed363c44bd82352e47617fca7584fe8c958ac1d80703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:35:01.388 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T00:35:01.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:01.422 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T00:35:01.422 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T00:35:01.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.g 2024-04-25T00:35:01.649 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:01 smithi045 systemd[1]: Stopping Ceph mds.g for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:02.149 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: 2024-04-25T00:35:01.734+0000 7f929bb98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:02.150 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:01 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g[68564]: 2024-04-25T00:35:01.734+0000 7f929bb98700 -1 mds.g *** got signal Terminated *** 2024-04-25T00:35:05.293 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:05 smithi045 podman[100612]: 2024-04-25 00:35:05.001496594 +0000 UTC m=+3.362814391 container died eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True) 2024-04-25T00:35:05.639 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:05 smithi045 podman[100612]: 2024-04-25 00:35:05.293320562 +0000 UTC m=+3.654638376 container cleanup eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:35:05.639 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:05 smithi045 bash[100612]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g 2024-04-25T00:35:05.640 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 00:35:05 smithi045 podman[100627]: 2024-04-25 00:35:05.519523421 +0000 UTC m=+0.516348059 container remove eb39981a19709d7fc3de4b06e0f66338837bd1541a1924fe3374a03936da4f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:35:05.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T00:35:05.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:05.694 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T00:35:05.694 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T00:35:05.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.j 2024-04-25T00:35:06.030 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:05 smithi045 systemd[1]: Stopping Ceph mds.j for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:06.399 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-j[67616]: 2024-04-25T00:35:06.030+0000 7f2be6037700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:06.399 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:06 smithi045 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-j[67616]: 2024-04-25T00:35:06.030+0000 7f2be6037700 -1 mds.j *** got signal Terminated *** 2024-04-25T00:35:10.149 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:09 smithi045 podman[100734]: 2024-04-25 00:35:09.81071644 +0000 UTC m=+3.898116291 container died 7475b49d5dfb5e307e27a22364a23c1297ebe273df931cc3e756df703c9b5c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:35:10.430 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:10 smithi045 podman[100734]: 2024-04-25 00:35:10.171520967 +0000 UTC m=+4.258920817 container cleanup 7475b49d5dfb5e307e27a22364a23c1297ebe273df931cc3e756df703c9b5c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T00:35:10.430 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 00:35:10 smithi045 bash[100734]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-j 2024-04-25T00:35:10.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T00:35:10.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:10.587 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T00:35:10.588 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T00:35:10.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.b 2024-04-25T00:35:10.869 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:10 smithi071 systemd[1]: Stopping Ceph mds.b for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:11.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:35:10.987+0000 7ff4335fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:11.369 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:10 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b[62872]: 2024-04-25T00:35:10.987+0000 7ff4335fe700 -1 mds.b *** got signal Terminated *** 2024-04-25T00:35:13.368 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:12 smithi071 podman[69608]: 2024-04-25 00:35:12.996229836 +0000 UTC m=+2.219218245 container died 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-25T00:35:13.746 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:13 smithi071 podman[69608]: 2024-04-25 00:35:13.46189503 +0000 UTC m=+2.684883428 container cleanup 9efd29d082b00f0bafc79b68f0592c205691305a9990a636fc11bb281a3101cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:35:13.746 INFO:journalctl@ceph.mds.b.smithi071.stdout:Apr 25 00:35:13 smithi071 bash[69608]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-b 2024-04-25T00:35:13.891 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T00:35:13.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:13.925 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T00:35:13.925 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T00:35:13.925 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.e 2024-04-25T00:35:14.368 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:14 smithi071 systemd[1]: Stopping Ceph mds.e for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:14.369 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: 2024-04-25T00:35:14.256+0000 7ffb58294700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:14.369 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:14 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e[62263]: 2024-04-25T00:35:14.256+0000 7ffb58294700 -1 mds.e *** got signal Terminated *** 2024-04-25T00:35:17.267 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:16 smithi071 podman[69729]: 2024-04-25 00:35:16.993132914 +0000 UTC m=+2.844006957 container died 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:35:17.608 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:17 smithi071 podman[69729]: 2024-04-25 00:35:17.366184272 +0000 UTC m=+3.217058313 container cleanup 1e27693faf6fb89fc26cb5aa192a9f25de8e3e278baec6a8146141edd0339a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T00:35:17.609 INFO:journalctl@ceph.mds.e.smithi071.stdout:Apr 25 00:35:17 smithi071 bash[69729]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-e 2024-04-25T00:35:17.734 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T00:35:17.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:17.767 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T00:35:17.767 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T00:35:17.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.h 2024-04-25T00:35:17.869 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:17 smithi071 systemd[1]: Stopping Ceph mds.h for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:18.368 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: 2024-04-25T00:35:18.086+0000 7f2c78fe7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:18.369 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:18 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h[62567]: 2024-04-25T00:35:18.086+0000 7f2c78fe7700 -1 mds.h *** got signal Terminated *** 2024-04-25T00:35:21.368 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:20 smithi071 podman[69862]: 2024-04-25 00:35:20.995863277 +0000 UTC m=+3.014033498 container died 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T00:35:21.829 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:21 smithi071 podman[69862]: 2024-04-25 00:35:21.636230078 +0000 UTC m=+3.654400287 container cleanup 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:35:21.829 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:21 smithi071 bash[69862]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h 2024-04-25T00:35:22.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T00:35:22.080 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:21 smithi071 podman[69876]: 2024-04-25 00:35:21.82928277 +0000 UTC m=+0.826664191 container remove 907d4bec42838bf055e87b6fad02cfa350cd830096306f62b6c8941ac2d8fb82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T00:35:22.080 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:22 smithi071 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.h.service: Deactivated successfully. 2024-04-25T00:35:22.080 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:22 smithi071 systemd[1]: Stopped Ceph mds.h for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:35:22.080 INFO:journalctl@ceph.mds.h.smithi071.stdout:Apr 25 00:35:22 smithi071 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.h.service: Consumed 1.903s CPU time. 2024-04-25T00:35:22.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:22.091 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T00:35:22.091 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T00:35:22.091 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.k 2024-04-25T00:35:22.368 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:22 smithi071 systemd[1]: Stopping Ceph mds.k for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:22.869 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: 2024-04-25T00:35:22.431+0000 7fec3e3c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:22.869 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:22 smithi071 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k[61941]: 2024-04-25T00:35:22.431+0000 7fec3e3c7700 -1 mds.k *** got signal Terminated *** 2024-04-25T00:35:25.369 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:25 smithi071 podman[69987]: 2024-04-25 00:35:25.003399487 +0000 UTC m=+2.693061572 container died 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T00:35:25.824 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:25 smithi071 podman[69987]: 2024-04-25 00:35:25.556500415 +0000 UTC m=+3.246162502 container cleanup 1a2b2438e67246baf8506643453a8d28c177db52a2d9dc322248b0866a947cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:35:25.824 INFO:journalctl@ceph.mds.k.smithi071.stdout:Apr 25 00:35:25 smithi071 bash[69987]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-k 2024-04-25T00:35:25.979 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T00:35:26.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:26.012 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T00:35:26.012 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T00:35:26.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.c 2024-04-25T00:35:26.306 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:26 smithi122 systemd[1]: Stopping Ceph mds.c for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:26.806 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: 2024-04-25T00:35:26.429+0000 7f23e2f49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:26.806 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:26 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c[63518]: 2024-04-25T00:35:26.429+0000 7f23e2f49700 -1 mds.c *** got signal Terminated *** 2024-04-25T00:35:29.306 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:29 smithi122 podman[70421]: 2024-04-25 00:35:29.00167631 +0000 UTC m=+2.794094911 container died 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:35:29.556 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:29 smithi122 podman[70421]: 2024-04-25 00:35:29.405305783 +0000 UTC m=+3.197724393 container cleanup 05ccd812718f274684731923327ca71a296779bb782c00f0bed39b6a11d005cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:35:29.556 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 25 00:35:29 smithi122 bash[70421]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-c 2024-04-25T00:35:29.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T00:35:29.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:29.854 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T00:35:29.854 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T00:35:29.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.f 2024-04-25T00:35:30.306 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:29 smithi122 systemd[1]: Stopping Ceph mds.f for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:30.306 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: 2024-04-25T00:35:30.175+0000 7f2cc092f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:30.306 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:30 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f[63211]: 2024-04-25T00:35:30.175+0000 7f2cc092f700 -1 mds.f *** got signal Terminated *** 2024-04-25T00:35:33.306 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 podman[70553]: 2024-04-25 00:35:33.002505456 +0000 UTC m=+2.926221152 container died e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:35:33.753 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 podman[70553]: 2024-04-25 00:35:33.526974847 +0000 UTC m=+3.450690534 container cleanup e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:35:33.753 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 bash[70553]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f 2024-04-25T00:35:33.998 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T00:35:34.018 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 podman[70567]: 2024-04-25 00:35:33.753315457 +0000 UTC m=+0.750707181 container remove e4126ecd158084c90b1abe781917c98c3b008a3024c0704b5506f2eeaedbf1fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:35:34.018 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-25T00:35:34.019 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 systemd[1]: Stopped Ceph mds.f for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:35:34.019 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 25 00:35:33 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.970s CPU time. 2024-04-25T00:35:34.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:34.030 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T00:35:34.031 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T00:35:34.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.i 2024-04-25T00:35:34.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:34 smithi122 systemd[1]: Stopping Ceph mds.i for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:34.806 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: 2024-04-25T00:35:34.371+0000 7f408adf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:34.806 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:34 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i[63826]: 2024-04-25T00:35:34.371+0000 7f408adf4700 -1 mds.i *** got signal Terminated *** 2024-04-25T00:35:37.306 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:36 smithi122 podman[70674]: 2024-04-25 00:35:36.999339317 +0000 UTC m=+2.744177236 container died c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:35:37.583 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 podman[70674]: 2024-04-25 00:35:37.414225168 +0000 UTC m=+3.159063096 container cleanup c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:35:37.583 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 bash[70674]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i 2024-04-25T00:35:37.833 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 podman[70688]: 2024-04-25 00:35:37.583186885 +0000 UTC m=+0.584136728 container remove c547891e9449b49d866d6d109d0d402437296743cb5d2319cf048441e25c3f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-i, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:35:37.833 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T00:35:37.833 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 systemd[1]: Stopped Ceph mds.i for 24bb2e7a-0298-11ef-bc93-c7b262605968. 2024-04-25T00:35:37.833 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 25 00:35:37 smithi122 systemd[1]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.i.service: Consumed 1.976s CPU time. 2024-04-25T00:35:37.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T00:35:37.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:37.866 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T00:35:37.867 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T00:35:37.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.l 2024-04-25T00:35:38.169 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:37 smithi122 systemd[1]: Stopping Ceph mds.l for 24bb2e7a-0298-11ef-bc93-c7b262605968... 2024-04-25T00:35:38.556 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: 2024-04-25T00:35:38.168+0000 7fc7cba88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:35:38.556 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:38 smithi122 ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l[62894]: 2024-04-25T00:35:38.168+0000 7fc7cba88700 -1 mds.l *** got signal Terminated *** 2024-04-25T00:35:41.306 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:41 smithi122 podman[70794]: 2024-04-25 00:35:41.006621731 +0000 UTC m=+2.918370766 container died 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:35:41.629 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:41 smithi122 podman[70794]: 2024-04-25 00:35:41.419759557 +0000 UTC m=+3.331508592 container cleanup 358d2787ef1f9cba08384b5bbc2838e3698b5cbbb1435ab58b4890946e5eeb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T00:35:41.630 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 25 00:35:41 smithi122 bash[70794]: ceph-24bb2e7a-0298-11ef-bc93-c7b262605968-mds-l 2024-04-25T00:35:41.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24bb2e7a-0298-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T00:35:41.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:35:41.800 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T00:35:41.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:35:41.999 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:35:44.574 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:35:44.778 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:35:47.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:35:47.295 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:35:49.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:35:49.608 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:35:49.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:35:49.666 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T00:35:49.667 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi045/crash 2024-04-25T00:35:49.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:35:49.707 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:35:49.707 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:35:49.709 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi071/crash 2024-04-25T00:35:49.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:35:49.814 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:35:49.814 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:35:49.816 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi122/crash 2024-04-25T00:35:49.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:35:49.852 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T00:35:49.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T00:35:49.898 INFO:tasks.cephadm:Compressing logs... 2024-04-25T00:35:49.898 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:35:49.941 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:35:49.944 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:35:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:35:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-04-25T00:35:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:35:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:35:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T00:35:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T00:35:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T00:35:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T00:35:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T00:35:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T00:35:49.972 INFO:teuthology.orchestra.run.smithi045.stderr: 87.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:35:49.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T00:35:49.973 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr: 94.6% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:35:49.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.a.log: 84.1% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:35:49.984 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T00:35:49.988 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:35:49.988 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.f.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% 90.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:35:49.989 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:35:49.990 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T00:35:49.990 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T00:35:49.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:35:49.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T00:35:49.993 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T00:35:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T00:35:49.994 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.8.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T00:35:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T00:35:49.997 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:35:49.997 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-25T00:35:50.001 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T00:35:50.003 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:35:50.010 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T00:35:50.013 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:35:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-25T00:35:50.036 INFO:teuthology.orchestra.run.smithi045.stderr: 94.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T00:35:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.k.log: 92.8% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T00:35:50.037 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T00:35:50.040 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T00:35:50.042 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T00:35:50.059 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T00:35:50.070 INFO:teuthology.orchestra.run.smithi071.stderr: 94.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T00:35:50.081 INFO:teuthology.orchestra.run.smithi122.stderr: 94.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T00:35:50.084 INFO:teuthology.orchestra.run.smithi071.stderr: 93.9% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T00:35:50.354 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T00:35:50.396 INFO:teuthology.orchestra.run.smithi122.stderr: 92.0% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T00:35:50.403 INFO:teuthology.orchestra.run.smithi071.stderr: 92.9% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:35:50.487 INFO:teuthology.orchestra.run.smithi071.stderr: 91.9% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T00:35:50.986 INFO:teuthology.orchestra.run.smithi045.stderr: 90.6% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T00:35:52.011 INFO:teuthology.orchestra.run.smithi071.stderr: 92.3% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T00:35:52.226 INFO:teuthology.orchestra.run.smithi071.stderr: 92.7% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T00:35:52.582 INFO:teuthology.orchestra.run.smithi045.stderr: 92.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T00:35:52.821 INFO:teuthology.orchestra.run.smithi071.stderr: 92.6% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T00:35:52.981 INFO:teuthology.orchestra.run.smithi122.stderr: 92.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T00:35:53.060 INFO:teuthology.orchestra.run.smithi122.stderr: 92.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T00:35:53.217 INFO:teuthology.orchestra.run.smithi045.stderr: 92.5% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T00:35:53.523 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T00:35:53.606 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T00:35:53.655 INFO:teuthology.orchestra.run.smithi071.stderr: 92.1% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:35:53.657 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-25T00:35:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m3.699s 2024-04-25T00:35:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m11.595s 2024-04-25T00:35:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.397s 2024-04-25T00:35:54.347 INFO:teuthology.orchestra.run.smithi122.stderr: 92.2% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T00:35:54.348 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-25T00:35:54.348 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m4.390s 2024-04-25T00:35:54.348 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m14.274s 2024-04-25T00:35:54.348 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.458s 2024-04-25T00:35:55.753 INFO:teuthology.orchestra.run.smithi045.stderr: 91.7% -- replaced with /var/log/ceph/24bb2e7a-0298-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T00:35:55.755 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T00:35:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m5.800s 2024-04-25T00:35:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m16.348s 2024-04-25T00:35:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.474s 2024-04-25T00:35:55.756 INFO:tasks.cephadm:Archiving logs... 2024-04-25T00:35:55.756 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi045/log 2024-04-25T00:35:55.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:35:58.422 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi071/log 2024-04-25T00:35:58.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:36:00.243 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi122/log 2024-04-25T00:36:00.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:36:02.869 INFO:tasks.cephadm:Removing cluster... 2024-04-25T00:36:02.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force 2024-04-25T00:36:03.070 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:36:03.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force 2024-04-25T00:36:03.549 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:36:03.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24bb2e7a-0298-11ef-bc93-c7b262605968 --force 2024-04-25T00:36:04.046 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 24bb2e7a-0298-11ef-bc93-c7b262605968 2024-04-25T00:36:04.314 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T00:36:04.314 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:36:04.337 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:36:04.363 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:36:04.383 INFO:tasks.cephadm:Teardown complete 2024-04-25T00:36:04.384 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:36:04.503 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:36:04.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:36:04.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:36:04.513 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:36:04.632 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:36:04.700 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:36:04.700 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:36:04.709 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:36:04.715 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:36:04.721 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:^- hv01.front.sepia.ceph.com 2 6 377 48 +960us[ +960us] +/- 584ms 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -41us[ -41us] +/- 50ms 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv03.front.sepia.ceph.com 3 6 377 53 +47us[ +51us] +/- 74ms 2024-04-25T00:36:04.728 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:36:04.729 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:36:04.731 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:^- hv01.front.sepia.ceph.com 2 6 377 56 +967us[ +973us] +/- 584ms 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -40us[ -40us] +/- 50ms 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv03.front.sepia.ceph.com 3 6 377 54 +40us[ +45us] +/- 74ms 2024-04-25T00:36:04.732 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:^- hv01.front.sepia.ceph.com 2 6 377 54 +958us[ +958us] +/- 584ms 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 7 377 121 -36us[ -40us] +/- 81ms 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +58us[ +58us] +/- 74ms 2024-04-25T00:36:04.734 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:36:04.735 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:36:05.194 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:36:05.195 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:36:05.278 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:36:05.364 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:36:05.436 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:36:05.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:36:05.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:05.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:36:05.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:05.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:36:05.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:05.628 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:36:06.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:36:07.079 INFO:teuthology.task.internal:Duration was 1909.832271 seconds 2024-04-25T00:36:07.079 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:36:07.255 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:36:07.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:36:07.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:36:07.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:36:07.308 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:36:07.310 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:36:07.314 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:36:07.690 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:36:07.691 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T00:36:07.691 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:36:07.755 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-25T00:36:07.755 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:36:07.786 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-25T00:36:07.787 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:36:07.820 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:36:07.820 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:36:07.823 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:36:07.830 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:36:07.888 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:36:07.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:36:07.895 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:36:07.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:36:08.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:36:08.322 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:36:08.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:36:08.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:36:08.371 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:36:08.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:36:08.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.429 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-25T00:36:08.433 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-25T00:36:08.456 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-25T00:36:08.476 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:08.495 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:08.527 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:36:08.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:36:08.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:36:08.554 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:36:08.555 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi045 2024-04-25T00:36:08.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:36:08.597 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi071 2024-04-25T00:36:08.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:36:08.632 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672244/remote/smithi122 2024-04-25T00:36:08.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:36:08.667 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:36:08.667 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:36:08.669 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:36:08.671 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:36:08.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:36:08.730 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:36:08.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:36:08.770 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:36:08.771 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:36:08.773 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:36:08.775 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:36:08.786 INFO:teuthology.orchestra.run.smithi045.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:36 /home/ubuntu/cephtest 2024-04-25T00:36:08.787 INFO:teuthology.orchestra.run.smithi071.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:36 /home/ubuntu/cephtest 2024-04-25T00:36:08.790 INFO:teuthology.orchestra.run.smithi122.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:36 /home/ubuntu/cephtest 2024-04-25T00:36:08.791 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:36:08.821 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:36:08.927 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} duration: 1909.8322706222534 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:36:08.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:36:09.007 INFO:teuthology.run:pass